This page explains how to convert a googletest executable into a fully isolated test. This is done by describing runtime dependencies for a test.
For information about the infrastructure itself and the roadmap, see Isolated Testing Infrastructure.
Goal: describe the exact list of files needed to run a executable.
The isolate project is all the file formats, tools and server code to make this work fast and efficiently. Every test executable list the data files it depends on, which allows the testers bots to only download the files they need instead of the whole Chromium source tree.
Goal: distribute tasks fast and efficiently in an hetegeneous fleet of bots.
The swarming project is all the tools, server and bot code to run a step (like a unit test) on a remote bot and get results back. It has native google-test sharding capability.
By reducing the amount of data that we need to transfer to the tester machines, it becomes much easier to increase the number of tester bots that we have using Swarming (part of LUCI).
See the Roadmap for what can or can't be done.
Expectations of the tests:
To create a new isolated test, do the following section in order:
See src/build/isolate.gypi for the gory details.
In general, it is preferable to:
By default a directory is only included if all the files in it are needed. If most of the directory is used and you feel that it makes sense to make the full directory available to simplify the .isolate file, feel free to replace the individual listings with a single directory listing.
Refer to github.com/luci/luci-py/blob/master/appengine/isolate/doc/client/Isolate-User-Guide.md for the up to date user manual.
See src/tools/swarming_client/isolate.py --help for the up-to-date behavior description.
Right now, only users @google.com can use the infrastructure. For others, we'll try to make it available to Chromium committers eventually. Note that the whole Swarming infrastructure is open source so if any other company would help to recreate the same infrastructure internally, send us a note at email@example.com
If you have a chromium checkout, you already have everything you need in
By login first, you have access tokens so that the following commands do not have to constantly prompt for your identity.
If you are running through a text only session on a remote machine, append argument
This is a good sanity check to ensure that everything works:
If this doesn't work, see the FAQ before continuing or ping us, we're friendly.
For example, let's take base_unittests and create out/Release/base_unittests.isolated:
Still scratching your head about what base_unittests_run is? Read the top of this page. It compiles base_unittests.isolate into base_unittests.isolated so it can be archived properly.
Now you built something, time to archives it to the isolate server and request Swarming to run it on your behalf.
First thing it does is to archive the binary. Depending on your connection speed and the size of the executable, it may take up to a minute. Then it triggers the task and wait for results. OS currently available:
It is possible that all the bots are currently fully utilized.
In theory yes, in practice please don't and keep the list to the strict minimum. The goal is not to run the tests more slowly and having the slaves download 20 gb of data. Reasons includes:
The .isolated files are generated when you build the isolation specific version of a target, e.g. out/Debug or out/Release. The isolation target name is just the normal target name with a _run added to the end.
If you have a test that passes locally and fails on some trybots, then it could be related.
This error can be seen when a browser test js file is not found:
TypeError: Cannot read property 'testCaseBodies' of undefined