The Chromium Projects

Search this site

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.

The Chromium OS designs and code are preliminary. Expect them to evolve.
For Developers‎ > ‎

Testing and infrastructure

Contents

  1. 1 Child pages
  2. 2 Overview
  3. 3 Expectations
  4. 4 Inventory of Existing Tests
  5. 5 Running basic tests
    1. 5.1 Windows
    2. 5.2 Linux
    3. 5.3 Mac
    4. 5.4 Running a particular subtest
    5. 5.5 Chrome Frame SSL tests
  6. 6 Layout tests
  7. 7 Inducing a crash

Child pages

Overview

Chromium development places a high premium on tests, tests, tests, tests, and more tests. In order to maintain a rapid rate of development across multiple platforms and an ever increasing set of features, it is imperative that test suites be updated, maintained, executed, and evolved. The Chromium Buildbots are employed to run these tests 24x7.

Expectations

Developers contributing code are expected to always run all tests. Given the volume of tests combined with the multitude of platforms, this can be a daunting task. But fear not, your Chromium teammates will help you too. The goal is to keep the source tree always building so that regressions are minimized and developer productivity is maximized. When contributing code, always consider whether your change has enough testing. If it is a new feature or module, it should almost certainly be accompanied by tests.

After committing a change, you should use the Chromium Buildbot to monitor that the automated tests pass with your change. Please be reachable by email or IRC after check-in in case something unexpected happens.

Inventory of Existing Tests

This list has been moved to the wiki, please make further modifications there.

Test Development Infrastructure
To assist with building tests, several pieces of infrastructure exist.  A truism with testing is that more is always needed.

Here are some tools you might find useful:
  • chrome/test/automation - Chromium includes a mechanism for driving the browser through automation.  This is primarily used with the UI tests.
  • gtest - GTest is Chromium's C++ test harness.
  • image_diff - A mechanism for comparing bitmaps.

Running basic tests

Windows

  1. Build
  2. Set the test as the startup project and press F5 to compile it.
  3. Or in a cmd window, run the test of interest in src\chrome\Debug, e.g. src\chrome\Debug\base_unittests.exe

Linux

  1. Build
  2. cd to src/out
  3. Run the test of interest, e.g. ./Debug/base_unittests

Mac

  1. Build.
  2. Hint: Build the appropriate "Run *tests" target in Xcode (for example, "Run unit_tests" in chrome.xcodeproj.)
  3. Test failures will show up in the build results just like compile errors would.
  4. If you want to see the raw gtest output, press the little square lines-of-text button below the build results to show the build transcript.
If you want to run the tests from Terminal — for instance, to filter sub-tests — do this instead:
  1. Build the appropriate "*tests" target in Xcode (for example, "unit_tests" in chrome.xcodeproj.) This just builds the tests, without running them.
  2. In a shell, cd to src/
  3. Run the appropriate test binary in xcodebuild/Debug (for example, xcodebuild/Debug/unit_tests)

Running a particular subtest

The above test executables are built with gtest, so they accept command line arguments to filter which sub-tests to run.  For instance, base_unittests --gtest_filter=FileUtilTest.*

Chrome Frame SSL tests

On Windows, if you plan to run the Chrome Frame network tests with the host network stack (chrome_frame_net_tests.exe), you need to import the test root CA certificate to your trusted root CA store. This is not necessary for tests that use Chrome's network, such as net_unittests.exe or ui_tests.exe, nor is it necessary for other operating systems such as OS X or Linux.

To do this on Windows XP:
  1. double-click the test root CA certificate file (net/data/ssl/certificates/root_ca_cert.crt in the source tree), this opens the Certificate Windows dialog
  2. click Install Certificate
  3. click Next
  4. select the option 'Place all certificate in the following store'
  5. click Browse.., select 'Trusted Root Certificate Authorities', click OK
  6. click Next and Finish
To do this on Windows Vista or Windows 7:
  1. in the Start menu, type "cmd" in the Start Search box (Note: note the Run... box) and then type ctrl-shift-Enter to run the Command Prompt as Administrator
  2. in the Command Prompt, cd into net\data\ssl\certificates in the source tree, and run the root_ca_cert.crt file
  3. in the "Open File - Security Warning" dialog, click OK
  4. in the Certificate dialog, click Install Certificate
  5. In the Certificate Import Wizard, click Next
  6. select the option "Place all certificates in the following store"
  7. click Browse.., check the "Shown physical stores" box, expand "Trusted Root Certification Authorities", select "Local Computer", click OK
  8. verify that the "Certificate store" field says "Trusted Root Certification Authorities\Local Computer"
  9. click Next and Finish

Layout tests

WebKit has a large suite of tests that typically verify a page is laid out properly.  We use them to verify much of the code that runs within a Chromium renderer.
To run these tests, build test_shell, change into the webkit directory, and run ./tools/layout_tests/run_webkit_tests.sh --debug .

More information about running layout tests or fixing layouts tests can be found on the WebKit Layout Tests page.

Inducing a crash

This can be useful to test breakpad:
  • about:crash - will cause a renderer crash.
  • about:inducebrowsercrashforrealz - will cause a browser crash.
Get dumps for Chromium crashes on Windows

Before running the tests, make sure to run crash_service.exe. We use this program to intercept crashes in chromium and write a crash dump in the Crash Reports folder under the User Data directory of your chromium profile.
If you also want crash_service.exe to intercept crashes for your normal Google Chrome or Chromium build, add the flag --noerrdialogs.
You can also use the flag --enable-dcheck to get assertion errors in release mode.

Sign in  |  Recent Site Activity  |  Terms  |  Report Abuse  |  Print page  |  Powered by Google Sites