This document describes how to build and run Chrome Frame’s set of tests. This can be handy when a try server complains or a builder breaks unexpectedly.To build and run Chrome Frame’s tests, you need to be able to build Chromium on Windows following the instructions here:
As of r121750, Chrome Frame projects are no longer included in chrome.sln. In order to build Chrome Frame and its associated tests, the current (as of 16/02/2012) recommended approach is to locally edit src/build/some.gyp as follows:
Once the file has been edited, you need to run gclient runhooks to regenerate your project files.
Once that has completed, the solution file src/build/some.sln will contain all the projects needed to build Chrome Frame and its tests.
Note for git users: non-checked in locally edited files and git don't play well together. The ever resourceful evanm@ suggested the following work around to keep git happy:
git update-index --assume-unchanged src/build/some.gyp
This contains a set of integration tests that exercise the full Chrome Frame stack. This means they run and automate one or more Internet Explorer instances that host Chrome Frame which starts up Chrome. They also start up a local web server to serve requests.
Several of the tests in chrome_frame_tests.exe use accessibility APIs (MSAA) to automate IE, doing things like clicking buttons on download dialogs. As such, they make assumptions about the current focus state of IE dialogs and fail if focus changes unexpectedly or if an interactive user clicks buttons on dialogs unexpectedly. As such, it is recommended that you not interact with the machine while chrome_frame_tests are running.
These tests do not currently run on the try servers.
[0216/152532:FATAL:test_utils.cc(126)] DLL registration failed (exit code: 0xa00
04005, command: "chrome_frame_tests.exe" --call-registration-entrypoint "E:\src\
chromium\src\build\Debug\npchrome_frame.dll" DllRegisterServer). Make sure you a
re running as Admin.
When this happens, while the tests will continue to run, they may not be testing the Chrome Frame instance you expect them to. To avoid this, always run the tests as admin.
automated reliability tests. It is little more than a page cycler for IE running chrome frame. You can run it locally if you like, as with chrome_frame_tests.exe it registers CF at startup so needs to be run as admin.
chrome_frame_reliability_tests do not run on the try servers.
This test works by setting up, in the test process, an environment that is as close as possible to that used by Chrome’s net tests except driven via a fake ExternalTab that emulates Chrome’s startup. Once setup, the tests run in a similar fashion to Chrome’s net tests, except that url requests are handled by Chrome Frame running in an IE instance over the automation interface.
Given that CF’s net tests attempt to start up pieces of Chrome outside of the norm (ie. outside of chrome.exe), it is fairly sensitive to new inter-module dependencies or invalid assumptions about startup order of different Chrome components. Most of the time when these tests break, they are being tickled by bad assumptions about startup order (code that assumes and doesn’t check that some subsystem of Chrome is initialized).
If debugging chrome_frame_net_tests, note that the entry point lives in src\chrome_frame\test\fake_external_tab.cc.
chrome_frame_net_tests run on the try servers.
NOTE THESE TESTS CURRENTLY DO NOT RUN ON MACHINES WITH IE9 INSTALLED. THIS IS BEING WORKED ON, BUT IN THE MEAN TIME A VM WITH IE8 OR LESS OR A CODE CHANGE TO main() IN fake_external_tab.cc is required.
If you plan to run the Chrome Frame network tests with the host network stack (chrome_frame_net_tests.exe), you need to import the test root CA certificate to your trusted root CA store.
To do this on Windows XP:
To do this on Windows Vista or Windows 7:
Before anything else, be aware that running a component=shared_library build on another machine is a tricky affair. If you don't want to worry about getting the right version of the CRT on your test machine (hint: if you don't know what the CRT is, follow the advice to come after the close parens right here:), it's much simpler to use the component=static_library build.
Running some of Chrome Frame's tests from a different location than the build folder in the source tree can be annoying. For example, like Chrome's net tests, CF's net tests makes a number of assumptions about the directory structure it lives in. To ease running on a VM, the indomitable joi@ put together a poor man's trybot script that copies all the needed files to a given directory. Running this script from a mounted share that contains your build output directory on a VM is a common use case.
The script can be found at src/chrome_frame/test/poor_mans_trybot.bat. Sample use case from a virtual PC vm:
In addition, if you build a version of Chrome Frame that dynamically links the MSVCRT (aka the component build) then you may get a SxS error. SxS errors are really annoying to work around. You can work around them by:
where <VERSION> is the version number of the MSVCRT you linked against. Note that this may be different from the one included with your copy of MSVC, as many of these can live on a Windows system. To figure out which version your app expects, you can check the Application log in the Event Viewer (type "Event Viewer in the Win7 start menu) to discover the missing version. Then you have to go and find that version on your system and copy the three DLLs listed above into the Microsoft.VC90.DebugCRT directory.