Chromium OS Commit Queue Overview

This page describes the Gerrit Commit Queue for Chromium OS (also known by the code name "Paladin")

The goal of the Commit Queue is to vet changes (build and test them on multiple platforms) and push them on behalf of developers.  It is responsible for not only committing changes, but also doing the Portage voodoo (uprevving ebuilds) necessary for other developers to see each others changes.

How do I submit my change to the Commit Queue?

If your change has been reviewed and marked as "Looks good to me, approved", you can submit your change by selecting the review button and marking your change as both "Verified" and "Ready".

How does the Commit Queue verify changes?

We first launch a trybot to sanity check your change. This trybot will compile your change and run unit tests on 3 major platforms. This check only includes your change and related changes, and takes about 20 minutes.

The commit queue runs approximately once an hour, and picks up all changes that have been fully verified by trybots. It builds a full image (on >15 platforms) and runs tests on both virtual machines and real hardware. Each commit queue run takes approximately an hour.

How does the commit queue decide when to pick up a change?

The Commit Queue only picks up changes that have been approved, verified and marked ready, and only if all their dependencies are similarly approved, verified, and marked ready (see the "How do I specify the dependencies of a change?" section below). The Commit Queue will start a run only if the tree is open (or, if the tree is throttled, the the CQ will wait until there are CQ+2 changes available, and if it times out will perform a run with no test patches).

Will the Commit Queue respect dependent changes in Gerrit?

Yes. This means that the Commit Queue will not try a change that depends on another until both it and the change it depends on are marked as ready for commit.

How do I specify the dependencies of a change?

If you commit a stack of CLs to a branch locally, and upload them all together, Gerrit will already know about your dependencies and the commit queue will honor these dependencies.

If you want to specify a dependency on other CLs, you can specify a dependency on any CL in Gerrit by using CQ-DEPEND. Here's how it works:

  • Specify dependencies in the last paragraph of your change, using CQ-DEPEND=CL:12345
  • You can specify multiple dependencies. Each dependency should be separated with a comma and a space. (E.g. CQ-DEPEND=CL:12345, CL:4321). You can also split dependencies into multiple CQ-DEPEND lines, if needed.
  • Internal dependencies should be prefixed with an asterisk. To depend on an internal change, use CL:*12345. To depend on an external change, use CL:12345.
  • Atomic transactions are supported. If a stack of changes need to go in atomically, the first change in the series should have a dependency on the last change in the series. This ensures that the Commit Queue will only push the whole group of CLs at once.

Here's an example:

Add file to install to 9999 ebuild file.

BUG=chromium:99999
TEST=Tested with dependent CL's in trybot.
CQ-DEPEND=CL:12345, CL:*4321

Can I launch a trybot before my change is approved?

If you have internal commit rights, yes you can. You can launch a trybot run using the following command:

cbuildbot -g '12345' '*4321' pre-cq-group

If you pre-verify your change using the "pre-cq-group" trybot, the commit queue will not launch a trybot run again.

For more details on remote trybots, see the remote trybot documentation.

What repositories are currently managed by the Commit Queue?

All projects under chromiumos/* and chromeos/* in gerrit and gerrit-int respectively are managed by the commit queue.

How do I watch what the Commit Queue is currently doing?

In order to see what the Commit Queue is doing, you can view its current work on the internal and external waterfalls.

  • The CommitQueueSync step shows what builds the current Commit Queue build is testing 
  • The CommitQueueCompletion step is responsible for submitting changes when they pass these tests

The Commit Queue didn't like my change, what can I do?

There are three places where the commit queue can fail to commit your change.

  1. When applying a change: If your change conflicts with another change, your change will be rejected and you will need to rebase your change locally and resolve the conflicts.
  2. When verifying a change: This is the standard failure where either we fail to build or test your change. Please read the error message and verify that the failure was not caused by your change. If your change was not at fault, you may mark it as Ready again.
  3. When submitting a change: This can happen if either you already submitted your change while the commit queue was working or someone else bypassed the commit queue and submitted a change that conflicts with your change. If the latter, follow (1).

If you need more information about a build failure, you can look at the commit-queue builders on the internal and external waterfalls and find the build run which has your CL in the CommitQueueSync stage.  These are the same logs other builders would have so delve accordingly.  The Commit Queue also sends a snippet of the build log and a link to help you debug the issue.

Will the Commit Queue commit my CL when the Tree is Closed?

No. However, it will commit any CLs it is processing if the tree is Throttled.

e.g. Tree is open, CQ picks up your CL, tree goes to throttled, CQ finishes testing & merges your CL.

How can I bypass the Commit Queue?

Please do not bypass the commit queue unless it is necessary. That said, to bypass the commit queue, hit the submit patch button and select "Yes, I'm a chump".

Can I bypass the hardware test step?

Each repository has a COMMIT-QUEUE.ini file. You can control what steps to skip using this file.

Does the Commit Queue ever pick up changes that haven't been verified by trybots?

If the Commit Queue finds that there are no changes that have been verified by trybots, the commit queue will start picking up unverified changes. This should only happen either at off-hours (when few developers are submitting changes) or when the trybot waterfall is having issues.

What should I do if the Commit Queue itself is down?

If the Commit Queue is down or consistently broken, please close the tree and contact a Build Sheriff or trooper immediately (chrome-troopers [at] google.com).

More Questions

Contact: chromium-os-dev@chromium.org