The Chromium Projects

Search this site

Except as otherwise noted, the content of this page is licensed under a Creative Commons Attribution 2.5 license, and examples are licensed under the BSD License.
Chromium‎ > ‎DOMUI Testing‎ > ‎

WebUI browser_tests

Contents

  1. 1 Problem
  2. 2 Objective
  3. 3 Solution
  4. 4 How to write a test
    1. 4.1 [maybe] create a new test file
    2. 4.2 Write a test fixture, defining the page to browse to:
    3. 4.3 Mock the Javascript handler:
    4. 4.4 Mock stubs which call a function:
    5. 4.5 Define a test using mock expectations:
    6. 4.6 Conditionally run a test using generated c++ ifdefs:See Handling a failing test for more details on style and how/when to disable a test.
  5. 5 Maintaining
  6. 6 Considerations/FAQs
  7. 7 Caveats
  8. 8 Best practices

Problem

See Also domui-testing.

WebUI contains Javascript, which runs in the renderer and a C++ handler, which runs in the UI thread of the browser process. While this is a necessary part of the design, testing across these boundaries of both language and process/thread is cumbersome at best.

Objective

Make it possible to test the Javascript portion of WebUI in Javascript:
  • Write WebUI tests in Javascript.
  • Run in browser_tests so that the page is loaded in a real chrome browser.
  • Allow WebUI handlers to be mocked in Javascript.

Solution

The solution comes in the following parts:
  • Support libraries: chrome/test/data/webui/test_api.js, chrome/third_party/mock4js/mock4js.js.
  • gyp rules for js2webui generator feeding results into browser_tests.
  • WebUIBrowserTest C++ class in chrome/browser/ui/webui/web_ui_browsertest* .
  • Test source files next to the implementation or in chrome/test/data/webui/ - see chrome/browser/ui/webui/options/options_browsertest.js and chrome/test/data/webui/print_preview.js for reference.

How to write a test

The best reference examples are chrome/test/data/webui/print_preview.js and chrome/browser/ui/webui/options/options_browsertest.js

  • [maybe] create a new test file

    • Create new file chrome/test/data/webui/mytest.js
    • Add chrome/test/data/webui/mytest.js to the sources for browser_tests in chrome/chrome_tests.gypi

  • Write a test fixture, defining the page to browse to:

/**
 * TestFixture for OptionsPage WebUI testing.
 * @extends {testing.Test}
 * @constructor
 **/
function OptionsWebUITest() {}

OptionsWebUITest.prototype = {
  __proto__: testing.Test.prototype,

  /**
   * Browse to the options page & call our preLoad().
   **/
  browsePreload: 'chrome://settings-frame',

  // ...

};

  • Mock the Javascript handler:

OptionsWebUITest.prototype = {
...
  /**
   * Register a mock handler to ensure expectations are met and options pages
   * behave correctly.
   **/
  preLoad: function() {
      this.makeAndRegisterMockHandler(
            ['defaultZoomFactorAction',
             'fetchPrefs',
             'observePrefs',
             'setBooleanPref',
             'setIntegerPref',
             'setDoublePref',
             'setStringPref',
             'setObjectPref',
             'clearPref',
             'coreOptionsUserMetricsAction',
            ]);

      // Register stubs for methods expected to be called before/during tests.
      // Specific expectations can be made in the tests themselves.
      this.mockHandler.stubs().fetchPrefs(ANYTHING);
      this.mockHandler.stubs().observePrefs(ANYTHING);
      this.mockHandler.stubs().coreOptionsUserMetricsAction(ANYTHING);
  },
...
};

  • Mock stubs which call a function:

    mockHandler.stubs().getDefaultPrinter().
        will(callFunction(function() {
          setDefaultPrinter('FooDevice');
        }));

  • Define a test using mock expectations:

TEST_F('OptionsWebUITest', 'testSetBooleanPrefTriggers', function() {
  var showHomeButton = $('toolbarShowHomeButton');
  var trueListValue = [
    'browser.show_home_button',
    true,
    'Options_Homepage_HomeButton',
  ];
  // Note: this expectation is checked in testing::Test::TearDown.
  this.mockHandler.expects(once()).setBooleanPref(trueListValue);

  // Cause the handler to be called.
  showHomeButton.click();
  showHomeButton.blur();
});

  • Conditionally run a test using generated c++ ifdefs:

See Handling a failing test for more details on style and how/when to disable a test.

// Not meant to run on ChromeOS at this time.
// Not finishing in windows. http://crbug.com/81723
GEN('#if defined(OS_CHROMEOS) || defined(OS_MACOSX) || defined(OS_WIN) \\');
GEN('    || defined(TOUCH_UI)');
GEN('#define MAYBE_testRefreshStaysOnCurrentPage \\');
GEN('    DISABLED_testRefreshStaysOnCurrentPage');
GEN('#else');
GEN('#define MAYBE_testRefreshStaysOnCurrentPage ' +
    'testRefreshStaysOnCurrentPage');
GEN('#endif');

TEST_F('OptionsWebUITest', 'MAYBE_testRefreshStaysOnCurrentPage', function() {
  var item = $('advancedPageNav');
  item.onclick();
  window.location.reload();
  var pageInstance = AdvancedOptions.getInstance();
  var topPage = OptionsPage.getTopmostVisiblePage();
  var expectedTitle = pageInstance.title;
  var actualTitle = document.title;
  expectEquals("chrome://settings/advanced", document.location.href);
  expectEquals(expectedTitle, actualTitle);
  expectEquals(pageInstance, topPage);
});

Maintaining

  • Adding more goodies to generator - edit chrome/test/ui/webui/javascript2webui.js and tests in chrome/test/data/webui/.
  • Disabling a test - find tests in chrome/test/data/webui/ & mark FLAKY_, DISABLED_ or use the MAYBE_ trick shown above to conditionally decide.

Considerations/FAQs

  • Isn't mocking in javascript not testing the WebUI message passing?  True, but that should be tested as a unit test and then trusted.  Mocking in JS is easier, less flaky (always recursive - no synchronization challenges) and much better than not having any tests at all.
  • If you already have the page in question why don't you just run all tests without starting a new IN_PROCESS_BROWSER_TEST? You are more than welcome to group tons of expect* calls into a single test; all errors will be reported after the entire test runs. Having tests be separate IN_PROCESS_BROWSER_TEST calls ensures the state is exactly the same at the start of each tests with no pollution from previous tests.

Caveats

  • [crbug.com/88104] The generator relies on v8_shell to be built for the host.  Currently the v8.gyp rules aren't correct for Arm as they have conditionals on the 'target_host' and don't heed the 'toolset". A gyp condition only runs the js2webui rule on non-arm platforms.
  • Use of MAYBE_ to ifdef will have a different run name from GTEST than from test_api.js - this is because MAYBE_xyz will be defined as either xyz or DISABLED_xyz in C++, but not changed in javascript.

Best practices

  • As described in the gtest docs, prefer expect* over assert*, as it will not halt the test, but will register the failure and allow other checks in that particular testcase to run.
  • Since the call is included in the failure error message, the optional message parameter should only include information not available:
// NO
TEST_F('FooTest', 'TestFoo', function() {
  expectEquals(foo, bar, 'foo != bar');
  expectEquals(foo, bar, foo + '!=' + bar);
  var i = ...;
  expectEquals(5, array[i], 'array[i] != 5');
}

// YES
TEST_F('FooTest', 'TestFoo', function() {
  expectEquals(foo, bar);
  expectEquals(foo, bar);
  var i = ...;
  expectEquals(5, array[i], 'i=' + i);
}


Subpages (1): WebUI accessibility audit
Comments