Run Media Performance Class Tests

The Android Compatibility Test Suite (CTS) includes media performance class tests for devices running Android 11 or higher. Because the test process differs between devices running Android 13 and devices running Android 11 or Android 12, be sure to follow the instructions for the version of Android running on your devices.

Download tests

Download the following tests:

Test Android 14 or higher devices

Run Performance Class 14 (PC14)-Video encoding quality (VEQ) tests

Android 14 introduces Video encoding quality (VEQ) requirements for Performance Class 14 (PC14) devices. The PC14-VEQ requirements essentially define a baseline quality target requirement in a measurable manner for the HW video encoders on a PC14 device.

The goal is to ensure that each Android device clustered into the Performance Class 14 delivers premium media experience to end users by creating high quality video content.

The PC14-VEQ requirements are verified by a new CTS test called CtsVideoEncodingQualityHostTest.

This CTS test runs a set of encoding test cases defined by the configuration JSON files. For each test case, the rate-distortion (RD) curve of the encoder being tested is generated by encoding a bitstream using the encoder settings from the JSON file. The reference rate-distortion curve is included in the same JSON file. The Video Multimethod Assessment Fusion (VMAF) vmaf_v0.6.1.json is used to compute the distortion (that is, quality loss).

A test case can pass only when its BDRATE-VMAF result is less than or equal to zero.

The PC14-VEQ CTS test covers encoding configurations including the following:

  • AVC and HEVC HW Encoders
  • VBR rate control mode
  • P-Frame/B-frame encoding
  • 1920x1080p30fps (Landscape) or 1080x1920p30fps (Portrait)

Test Android 13 or higher devices

Before starting a test run, prepare the devices for testing:

  1. Setup the device to advertise media performance class 13 (PC13) by setting the _ro.odm.build.media_performance_class system property value to 33.

  2. Ensure that the device system image is Android 13 or higher. Although you can run Media CTS tests on an Android 11 or higher system image, camera CTS tests require Android 13 features for PC13.

Run camera ITS tests

To run the camera ITS tests only, first set both the device and tablet serial numbers in config.yml, then run the following commands:

python tools/run_all_tests.py camera=[PRIMARY_REAR_CAMERA_ID] scenes=2_c
python tools/run_all_tests.py camera=[PRIMARY_FRONT_CAMERA_ID] scenes=2_c

Run media performance class tests

The cts-media-performance-class test plan verifies the CDD requirements for media performance class requirements (CDD Section 2.2.7.1) and camera requirements (CDD Section 2.2.7.2). Run the cts-media-performance-class test plan using the following command:

cts-tradefed run cts-media-performance-class

To run only the camera CTS tests, use the following command:

cts-tradefed run singleCommand cts --disable-reboot -m CtsCameraTestCases -t android.hardware.camera2.cts.ExtendedCameraCharacteristicsTest#testCameraPerfClassCharacteristics

To run only the media CTS tests, use the following command:

cts-tradefed run singleCommand cts --disable-reboot -m CtsMediaPerformanceClassTestCases

Run MediaDrm CTS tests

MediaDrm CTS tests verify the MediaDrm requirements in CDD Section 2.2.7.1. Run MediaDrm CTS tests using the following commands:

cts-tradefed run singleCommand cts --disable-reboot -c android.mediapc.cts.PerformanceClassTest -m testSecureHwDecodeSupport
cts-tradefed run singleCommand cts --disable-reboot -c android.mediapc.cts.PerformanceClassTest -m testMediaDrmSecurityLevelHwSecureAll

Run File System CTS tests

File System CTS tests verify the Performance requirements in CDD Section 2.2.7.4.

  • To test sequential read and write requirements, use the following command:

    cts-tradefed run singleCommand cts --disable-reboot -m CtsFileSystemTestCases -t android.filesystem.cts.SequentialRWTest

  • To test random read and write requirements, use the following command:

    cts-tradefed run singleCommand cts --disable-reboot -m CtsFileSystemTestCases -t android.filesystem.cts.RandomRWTest

Test Android 11 or 12 devices

To test devices running Android 11 or 12, first set up the test devices (setup differs between Android version and userdebug versus user devices), then run CTS and CTS verifier tests.

Advertise the media performance class

Use the following steps to set up test devices to advertise the correct performance class using the ro.odm.build.media_performance_class system property:

  • For performance class 12 (PC12), set the value of the property to 31.

  • For performance class 11 (PC11), set the value of the property to 30.

Userdebug devices

To set up Android userdebug devices, run the following commands:

  • Android 12 userdebug devices:

    adb root
adb shell setprop ro.odm.build.media_performance_class 31   // use 30 for PC11
adb shell stop && sleep 1 && adb shell start
adb shell pkill -f camera

  • Android 11 userdebug devices:

    adb root
adb shell setenforce 0
adb shell setprop ro.odm.build.media_performance_class 31   // use 30 for PC11
adb shell appops set com.android.cts.verifier MANAGE_EXTERNAL_STORAGE 0

You can set the ro.odm.build.media_performance_class system property only once per reboot (but you don't need to reboot the device before setting it the first time).

User devices

To set up Android userdebug devices, add the ro.odm.build.media_performance_class property with the desired performance class value to the product.prop file. An example is shown below:

ro.odm.build.media_performance_class=31   // use 30 for PC11

For user devices running Android 11, perform the following additional steps:

  1. Cherrypick the [1627181] Add media performance class property to sepolicy patch.

  2. Run the following command:

    adb shell appops set com.android.cts.verifier MANAGE_EXTERNAL_STORAGE 0

Run CTS and ITS tests

Build CTS and CTS Verifier with the downloaded patches applied, then run the appropriate version of Android for your test devices (Android 11 CTS for Android 11 devices, Android 12 CTS for Android 12 devices).

Media

Run media CTS tests using the following command:

cts-tradefed run singleCommand cts --disable-reboot -m CtsMediaPerformanceClassTestCases

Camera

Run camera CTS tests using the following commands:

cts-tradefed run singleCommand cts --disable-reboot -m CtsCameraTestCases -t android.hardware.camera2.cts.ExtendedCameraCharacteristicsTest#testCameraPerfClassCharacteristics

python tools/run_all_tests.py device=DEVICE camera=PRIMARY_CAMERA_ID chart=CHART_ID scenes=2_c

For devices running Android 12, run the following command:

cts-tradefed run singleCommand cts --disable-reboot -m CtsCameraApi31TestCases

Hardware

Run hardware CTS tests using the following commands:

cts-tradefed run singleCommand cts --disable-reboot -m CtsFileSystemTestCases -t android.filesystem.cts.RandomRWTest
cts-tradefed run singleCommand cts --disable-reboot -m CtsFileSystemTestCases -t android.filesystem.cts.SequentialRWTest