WebGL Conformance Tests

An effort to port Khronos's conformance tests to the GPU build bots. This allows developers to immediately find the revision that broke WebGL conformance.

How to Check Conformance

All bots on the GPU waterfall run the WebGL conformance suite under the step named, creatively enough, webgl_conformance_tests. The bots are expected to generally stay green, so it should be easy to spot failures.

The goal is to keep webgl_conformance_tests green. If the test is red, you can click through on individual tests for more information.

Run the Tests Locally

After building chrome (assuming Release mode) run: 

content/test/gpu/run_gpu_integration_test.py webgl_conformance --browser=release

Note that this harness mainly understands --browser=release and --browser=debug, assuming they're generated into out/Release and out/Debug. You can use --browser=exact --browser-executable=[path to chrome] to run a specific other version.

Currently this defaults to running only tests in the 1.0.1 suite. You can specify a different version using the --webgl-conformance-version flag, for example:

content/test/gpu/run_gpu_integration_test.py webgl_conformance --webgl-conformance-version=1.0.3
content/test/gpu/run_gpu_integration_test.py webgl_conformance --webgl-conformance-version=2.0.1

If you only want to run a single test or subset of tests, pass a partial test name into --test-filter flag. For example, if you only wanted to run the tests related to index validation (under conformance/buffers) you could run:

content/test/gpu/run_gpu_integration_test.py webgl_conformance --test-filter=index_validation

Note that all hyphens ("-") and path separators ("/") are converted to underscores when specifying a test filter.

Test Expectations

content/test/gpu/gpu_tests/webgl_conformance_expectations.py and webgl2_conformance_expectations.py contain the lists of all known conformance test failures for WebGL 1.0 and 2.0, specifically. See the comments at the top of the expectations file for a list of valid conditions. 

Format

self.{Fail|Skip}('test/url.html', ['array', 'of', 'conditions'], bug='optional crbug number')

Example

self.Fail('conformance/textures/texture-size.html', ['win', 'intel'], bug=121139)

Updating the Conformance Tests

Since the tests rely on an external repository, the local version must be updated on any change upstream.

See the GPU Pixel Wrangling instructions, section "Updating the WebGL conformance tests", for detailed instructions on updating the snapshot of the WebGL conformance tests in the Chromium repository.
Comments