The Chromium Projects

Search this site

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.

WebGL Conformance Tests

An effort to port Khronos's conformance tests to the GPU build bots. This allows developers to immediately find the revision that broke WebGL conformance.

Contents

  1. 1 How to Check Conformance
  2. 2 Run the Tests Locally
  3. 3 Test Expectations
    1. 3.1 Format
    2. 3.2 Example
  4. 4 Updating the Conformance Tests
    1. 4.1 Workflow
  5. 5 Future Work
    1. 5.1 Increase the granularity of test expectations.
    2. 5.2 Fix currently failing and flaky tests.

How to Check Conformance

Simple. Just navigate to the GPU waterfall. Unfortunately, many listed bots are offline, so pay attention to these three:
  1. GPU Win7 Tests
  2. GPU Mac 10.6 Tests (dbg)
  3. GPU Linux Tests (dbg)(1) (ATI)
  4. GPU Linux Tests (dbg)(2) (NVIDIA)
    5. (View all four here)
The goal is to keep gpu_tests green. If the test is red, you can click through on individual tests for more information.

Run the Tests Locally

After building the chrome and gpu_tests binaries, run:
gpu_tests --gtest_filter=WebGLConformanceTests.*

Test Expectations

Similar to the WebKit Layout Tests, webgl_conformance_test_expectations.txt contains the list of all known conformance test failures. The header-generating script, generate_webgl_test_list.py, automatically prepends DISABLED, FAILS or FLAKY to the defective test names. See Handling a failing test for more information.

Format

If not followed exactly as specified, the test expectation will be ignored.
{BUG#} {MAC,WIN,LINUX} : {TEST_NAME} = {PASS,FAIL,TIMEOUT}

Example

91530 LINUX MAC : conformance_quickCheckAPI = TIMEOUT

Updating the Conformance Tests

Since the tests rely on an external repository, the local version must be updated on any change upstream. Right now, this is manual work (and blocked by Khronos's poor servers).

Workflow

  1. Navigate to the conformance test directory.
  2. Remove old test files (to purge any dead files).
  3. Download the most recent revision.
    4. svn export https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl
  4. Some files may be marked executable. Remove those permissions.
    5. chmod -R a-x *
  5. Update the README file.
  6. Check in revisions.
  7. Update DEPS.

Future Work

Increase the granularity of test expectations.

  1. Expectations for specific GPU cards, based on vendor and device IDs.
  2. Expectations for specific driver versions.

Fix currently failing and flaky tests.

A non-exhaustive list can be found here.
Comments