Layout Test Style Guidelines

Feel free to propose new rules to put here or change to existing ones at blink-dev@. We have a thread for this topic.

See also
https://sites.google.com/a/chromium.org/dev/blink/blink-testing-and-the-w3c

Testing framework

Two testing frameworks are available in Blink layout test files. Consider using either of them to reduce boilerplate code and HTML elements.

js-test.js (From WebKit)

Pros:
  • Helpers like evalAndLog() and shouldBe() produce verbose output, which can make the output self-documenting
Cons:
  • shouldBe family functions use eval(). Values to be checked need to live in global scope (window, self).

testharness.js and testharnessreport.js (From W3C test suites)

Pros:
  • Can be upstreamed to W3C web-platform-tests
  • Passing expectations are not needed.
  • Contains helpers for writing tests with workers and asynchronous steps
Cons:
  • blink-dev doesn't have direct control on testharness.js development; changes can be upstreamed, but process may be slow.
  • Only prints whether each test case has passed, or the first failed assertion.

Comparison (=== vs. ==)

You might want to use the triple-equals operator for comparing actual/expected value strictly by avoiding type conversion.

assert(!"");          // empty string is falsy
assert(!!"0");        // and non-empty string is truthy

assert("0" == 0);     // type coercion: string to number
assert("" == false);  // type coercion: string to boolean
assert("0" == false); // type coercion: string to number, then number to boolean!
assert("0" != "");    // but it's not transitive!

DOCTYPE declaration

Unless the test is explicitly testing quirks mode, include a DOCTYPE declaration at the top of the document.

<html>, <head>, <body>

These tags are normally omitted.

<!DOCTYPE html>
<script src="../resources/js-test.js"></script>
<script>
  ...
</script>

Indentation

Either of 2 space and 4 space indentation.

The usage should be consistent within a test file.

Braces

Either of Blink style or Google C++ style.

The usage should be consistent within a test file.

Quotation

Some tests use double quote for HTML (e.g. <script src="some.js">) and single quote for JavaScript (e.g. var image = 'resources/image.jpg';), but it could be either.

The usage should be consistent within a test file.

Comments

First, try to give a descriptive name to variables and functions. Then, write comments wherever it makes sense to improve readability.

File naming

Give a descriptive name. Separate words by hyphens or underscores. Be consistent.

Special directories

Put files that are used by layout test files in a sub-directory named "resources". See base.py.

Asynchronous tests

Use of Promises may improve readability of tests on asynchronous operations. Use of callbacks is still fine.

TestRunner API

Some tests may require special help by the content_shell. See test_runner.cc for what kind of APIs are available on window.testRunner.

Most Layout Tests can be run in a browser without these additional APIs - which can assist in rapid iteration or quick debugging, and comparing the behavior with other browsers. For tests that do rely on helpers (e.g. observable garbage collection, event generation, etc) it is recommended that you feature-detect for the helpers and fail the test early with a diagnostic message explaining the requirement.

Writing HTTP tests

The test server sets up an alias to LayoutTests/resources directory. You can access testing framework at e.g. "/js-test-resources/js-test.js". Use it instead of putting lots of ".."s.

Stable tests

If --stable-release-mode flag is specified, test-only features (ones listed in RuntimeEnabledFeatures.in with "test") are turned off.

TestExpectation family files

License boilerplate

<Most of layout test files don't contain any license declaration. Is this fine? Should we encourage putting it?>

ServiceWorker tests

ServiceWorker development team has their own style guide. Some of them may apply to your project.

Comments