Troopers know more about maintaining the buildbot masters and slaves themselves. They're the people to look for when the bots need an OS update, a machine goes offline, checkouts are failing repeatedly, and so on.
What is a gardener?
Gardeners are watchers of particular component interactions. They generally watch a component's release or development and move the version included forward when it is compatible. Of particular interest to the Chromium projects are the gardeners who watch the interaction between WebKit, Skia and Chromium, and those who watch the interaction of Chromium and ChromiumOS.
See Sheriff Details: Chromium to revert changes, disable tests, etc.
The tree status can be closed, throttled, or open. These status levels control the activity of the commit queue. If the tree is open, the commit queue run as normal. If the tree is closed, the commit queue will not submit any changes. If the tree is throttled, a commit queue run that is already running will be allowed to submit changes, but further runs will not start.
Warning: before opening the tree for one failure, make sure that all the bots are green. if the tree's already closed then subsequent failures don't "reclose" so it can be easy to miss them.
Annotate the tree status with information about what is known about the status of build failures. For example, automatic closure messages such as...
... should be changed to:
... to indicate that committer 'johnd' has been notified of the problem and is looking into it. Once a fix has been checked in, sheriffs often use status:
... to indicate that a fix/revert has been checked in and the tree will likely be opened soon.
If the tree has been closed for an extended time, particularly if the breakage covered more than one working timezone (US Pacific, US Eastern, Europe, Asia), it is considered best practice to communicate what was needed to fix the breakage. That way the next sheriff knows what's been happening, and people in other timezones know what to do next time it breaks the same way.
If the fix was simple, it can be listed in the tree-open status message, such as...
If a more detailed fix was needed, send email to the chromium-dev mailing list explaining what happened.
Sometimes you just need to clobber some class of bots (win, mac+ninja, linux asan using make, etc.). You can do this by landing a landmine change. Docs are here: Chromium Clobber Landmines.
To retry the last build, you can force a build. From the waterfall (internal url - see note below), click the name of the builder in the top gray row, then enter your username and an optional reason and click "Force Build".
For Chromium only, if you check the "Clobber" checkbox, it will also delete the build output directory before redoing the compile.
Note: If this is not a builder (no compile step), then doing a clobber won't do anything. You need to clobber the "Builder" first.
Note 2: You need to use the Google-internal build bot to do a clobber: http://chromegw.corp.google.com/i/chromium/
If you are on the external page for a bot, you can replace build.chromium.org/p with chromegw.corp.google.com/i in the URL to find its internal page.
There is an option to stop a build, but do not use it! If you stop the build during the update step, the bot is going to be hosed for sure. Again, don't use this option, and if you feel like using it, talk to the troopers first.
Install the buildbot error extension to more quickly isolate errors on stdio pages. See Useful extensions for chromium developers for more information.
Documentation is here in the Pre Flight Queue documentation
The authoritative list is the on Google Calendar. Here's how to add the sheriff calendar to yours:
To see who the sheriff is, click an event and look at the guest list. (Yes, it would be nice if it showed the people in the event title, but then there's the issue of the event title and the guest list getting out of sync -- no easy answer.) To find when a specific person is going to be sheriff, use google calendar's advanced search box (click the down-triangle in the main search box), select the appropriate sheriff calendar, and type the person's username into the "Who" box.
The script/process that updates the calendars can be found in svn://svn.chromium.org/chrome-internal/trunk/tools/build/scripts/tools/sheriff.
If you need to swap shifts with someone, add them to the rotation so that the buildbot and other tools display the proper people as sheriffs. To do this: