Chromium uses buildbot to run continuous builds and tests. The most useful is the Chromium Continuous master. From the top section you'll see other masters linked. Each can display a waterfall and console view. The Chromium Buildbot waterfall shows the status of many tests. That waterfall shows a lot of information that can be hard to understand at first, so let's take a quick tour. 

Build Steps 

Now look down one of the columns, starting at the grey box with the slave's name. Each box shows one step in that slave's build/test sequence, with the oldest one at the bottom and the current or latest one just below the name box. Just above the name box is the slave's current activity, and above that is the final result of the last build/test sequence that finished. 

A yellow build step is still in progress, green finished successfully, red finished with errors, orange finished with warnings, and purple had an internal Buildbot error. 

If you click on the "stdio" link for a step, you can see the exact command that was run and the environment it was run in in blue, and any output it produced in black. stderr is in red.

Most of the tests are pretty straightforward, but performance test output can be complicated at times. See the Guide to Perf Test Plots for more about that. 

Builders vs Testers

Changes

Each time someone commits a change to the svn repository, the build master tries to get the slaves to start their build and test sequences. (If they're already busy, the change is queued, which means that more than one change is included in a single run if they're coming in faster than the slaves can test.) The "changes" column at the left of the waterfall shows who committed a patch and when. If you hover over that link, you'll see a summary of the change; click on it to see a little more information. The times at the very left are in Pacific time. 

At the start of each run (that is, at the bottom of each series of steps for a slave), there's a yellow box holding the build number. Clicking on that build-number link shows more information about the run, including in principle the "blamelist" of changes that went into it. Every now and then, though, that list of changes is off by one. If you need to know for sure, look at the results of the "update" step to see exactly what gclient sync pulled in. 

Tree state

Seeing More and Seeing Less

Tucked away down at the very bottom of the waterfall page is a set of small links, of which two are particularly useful. The [next page] link takes you to the next screenful of the waterfall, which is backward in time to earlier builds. The [help] link, among other things, shows you a list of all the slaves attached to this master. You can choose which ones you'd like to see, then bookmark the resulting URL so you can get that view easily next time. 

Now back to the top of the waterfall. At the very top is a banner showing the current state of the tree. If the tree is closed because of build or test failures, it should be mentioned here. If there's an announcement about a new build process, expected downtime, or some other aspect of Chromium development, it'll generally be shown here, too. 

Below that is an oval box. It has a number of handy links at the left -- try them to see where they go. It also has four rows of colored boxes, which show the pass/fail status of the last completed runs for four categories of slaves: Chromium, Webkit, Perf, and Official/Memory. If you click on one of those category names, you'll go to a partial waterfall view that shows only the related slaves. Hovering over a colored box shows you the name of the slave it's summarizing, and clicking on the box will go to a waterfall view with only that one slave. 

Sheriffs 

The last thing in the oval is a list of this week's build sheriffs. Although every developer is responsible for running tests before committing patches and watching the tree for problems afterward, the sheriffs have overall responsibility in case someone else is away or not paying attention. 

Sources

All the source for Chromium's buildbot setup is found in our Subversion repository. See Getting the Buildbot Source if you'd like to take a look or help develop it.

Builder setup

For some builders, there exist no trybots. In order to debug compile failures, you need to setup a similar compile environment locally:

The console view is the default one. Marc-Antoine is also looking for people willing to use the json interface to create cool live interface in javascript. Contact him for more details.