New requirements for Ubuntu Certification

As part of looking at how we ensure that Ubuntu Certification delivers a great experience to consumers, we are revising our coverage of Suspend/Resume and boot testing. This is a proposal at the moment , so please feel free to comment on the suggested tests or additional ones that you would fine worth including.

Here is the list of proposed tests that a certify system would need to pass for 11.10 with respect suspend and resume, this include  existing and new tests:

Test Description
Suspend_once Main sleep test, all “after” tests depend on this one. Triggered manually, auto-wakes after 30-60 seconds (not all systems support automatic trigger and wake up)
hibernate_once Triggered manually, auto-wakes after 5 minutes
CPUs Checking before/after suspend Check in /proc/interrupt before and after suspend how many cores are online. Fail ‘after’ test if different.
Memory before/after suspend Check in /proc/meminfo that all system memory is still available.
USB Before and After suspend One per port, write to USB storage device (thumb drive) and verify.
Display Ensure that the display is working after resume (N/A for servers)
resolution_*_suspend ensure that resolution is same before/after suspend (N/A for servers)
cycle_resolutions_after_suspend Can cycle through resolutions after a suspend (N/A for servers)
*wireless_*_suspend Checks that wireless network is still available and can connect to it
network_*_suspend Checks that wired network is still available and can connect to it
Wake-on-lan put system to sleep then send an IP packet from a remote system to wake it (only applicable to servers)
audio_*_suspend Audio device still works after detected (N/A for servers)
record_playback_after_suspend records and playback after suspend (N/A for servers)
bluetooth_obex_*_suspend Bluetooth obex object can be sent to another device after suspend (N/A for servers)
stress/suspend Does 30 S3/Resume cycles,can only fail once
30 Soft Reboots Restart during testing 30 times
About these ads

11 Responses to New requirements for Ubuntu Certification

  1. michael says:

    stress/suspend Does 30 S3/Resume cycles,can only fail once

    why is it allowed to fail suspending?

    • Victor Palau says:

      it is allow to fail 1 out of 30, basically that is the rate that we think is acceptable. Some time race conditions might happen that will mean that is the case.

  2. Shane Fagan says:

    Hibernate isnt really a massive priority and there was talk about removing it from the interface since its causing so much problems on most machines. Id say that should be removed from the list.

  3. jspaleta says:

    I realize these are proposed tests. Are the final agreed on tests going to be archived on the public wiki for later reference? Are the current list of certification tests available on the public wiki?

    -jef

  4. Even with light use, one failure in 30 suspend cycles will mean “once a month” in user’s hands. People who suspend more than once a day, will see accordingly higher failure rates.

    • Victor Palau says:

      Hi David,

      Not really, this is 30 cycles without powering off. So if you run your computer for a month without reboot and you suspend once a day, that will be true.

      I take the feedback, we might remove the 1 faliure, but that only means that might fail on day 31 ;)

  5. skitterman says:

    For systems that have a hardware wifi switch (airplane mode) you might want to test that resuming with wifi switched off and then enabling wifi will result in working wifi. This is a class of problem that is not rare.

    For systems with just a soft wifi swtich (Fn key) testing that the switch actually disables wifi/bluetooth and then re-enables it would be useful.

    • Victor Palau says:

      we consider this but we have not include it at the moment, as it add the level of manual testing required. We have kept it on the “next round” to be consider for “P” release

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: