Ubuntu Certification – What do we test?
September 13, 2011 7 Comments
We frequently get asked what do we test on the certification program. While we do have a simple page covering this topic, some times we are asked for further details. We have now updated the certification program guide with a more comprehensive description of the test cases. We review and update if necessary the list of test cases for each release:
Note that these test cases only apply to hardware that actually supports the functionality. For example, we do not run the bluetooth tests on a laptop that does not list bluetooth on its specifications.
Here is what the program guide says for Oneiric:
We use three different lists:
- Whitelist, or features that are required for certification. If any of the tests in the whitelist fails, the certification will fail.
- Greylist, or features that are tested, but that don’t block certification. If any of the tests under the greylist fail, a note will be added to the certificate to warn the potential customer or user.
- Blacklist, or features that are not currently tested. We will consider adding more tests as needed.
- ia32 (x86), x86_64 and ARM processors are tested to ensure proper functionality.
- Stress tests are performed to ensure that they work during high utilization as well.
- Proper detection
- General usage
- Stress testing
Hard drive(s) tests are conducted to validate proper operation:
- High load
Optical drives (CD/DVD):
- Primary display (laptop panels or primary video port on desktops)
- Multiple-Monitor (where supported, we test multi-head display (2 heads))
- External video connections (HDMI, DisplayPort, VGA, RGB, etc.)
- Multiple resolutions
- Speakers and Headphones
- Microphone (Built-in, External)
- USB Mic, USB Headphones
USB controllers. Several USB devices are used to ensure all USB ports operate as expected:
Bluetooth controllers. Several bluetooth devices are used to ensure it works
- File transfer
Built-in Web cams
- Lid open
- Lid close
- Internal keyboard
- Touch screens (single touch)
Primary special keys (volume, mute)
Suspend/Resume (30 iterations)
Tested after resume:
- Display resolutions
- USB controllers
External Expansion Port
Firewire external storage devices
Data Card ports
Hibernate/Resume (30 iterations)
Data cards that are not SD or SDHC (for example MMC)
- Hybrid Graphics: if UMA or discreet work out of the box: all ports working
- we will note which card is the one that is certified.
- Whether proprietary drivers are necessary to enable 3D graphics.
- Wi-fi Slider: if the slider to turn the wi-fi on/off is not working, but the wi-fi
- can be disconnected through the UI controls, this failure is accepted (and noted).
Secondary special keys:
- Media Control
- Fingerprint readers
- HDMI/DisplayPort audio
- Surround audio
- Multitouch touchpads
- Multitouch screens
- Specific USB 3.0 devices
- 3G connections