This page has details to help Chromium perf sheriffs. For more information and procedures, see Tree Sheriffs and read the theses on Sheriffing.
Keep performance regressions from making it out into the wild! Also, triage all of the Chromium Perf alerts and keep the Chromium Perf waterfall green.
The Chromium Perf waterfall runs many different tests against the latest Chrome to measure Chrome’s performance. This data is automatically analyzed by GASP and will report anomalies via email to the current perf-sheriff and perfbot-gasper@ and also on the new perf dashboard. These anomalies likely correspond to a performance regression.
"There is usually a way to add functionality without hurting page load time, startup time or other key metrics. One just has to be a bit more clever at times." --Darin
What to Watch
What to Do
Triage anomalies from GASP
Visit the alerts dashboard page (sorted by default by most recent alerts) and make sure that the Perf Sheriff rotation is selected.
Click "sign in" in the top right corner and use your google.com account.
If the alert is a real regression, file a bug and track down the right owner (see detailed instructions below).
If the alert is invalid (i.e. GASP alerting on noise), mark alert as invalid or assign bug id -1.
Check the _ref trace as well (shown on graph by default) to see if it was a change in the machine. If both the trace of interest and the _ref trace move equally, something changed on the bot and it is not a regression in Chrome
By the end of your shift, the alerts dashboard page should have zero untriaged alerts.
As perf-sheriff, you are responsible for following through with the regression bugs filed during your shift. Assign yourself as the owner until you find a more appropriate owner.
If you aren't sure what to do, email chrome-perf-sheriffs@ and cc chrome-perf@ for backup.
For each regression, file a bug
You can file a bug directly from the alerts page or the graph page.
Label the bug with a milestone blocker for the next stable version of Chrome that could potentially include the regression if not fixed.
Assign yourself as the owner until you find a more appropriate owner.
For the fastest, best results it helps to identify the suspect CL(s) when filing the bug. Sometimes this is obvious from looking at the change list (click on a point on the graph to see the change list below). Most of the time it is not.
Check the other perf bots to narrow down the possible revision range, namely the Blink canary bots. Note that they always run ToT Blink, so if the regression is due to Blink, it might occur at a different Chromium CL.
If you've narrowed down a regression to a Blink roll, try to narrow down the Blink regression range by checking the Blink canary bots or using the bisection script, which can automatically drill down into Blink CLs. Don't assign it to the Blink gardener.
If you cannot narrow down the offending CL, you should speculatively revert suspicious CLs
By default, we operate on a no regressions policy. If you can identify a CL that causes performance regressions, revert the CL.
Known corner cases:
- If sizes fails, just update the expectations. Take a look at the graph and make sure there aren't any jumps of more than 100KB. If there are, file a bug first.
- If a test only fails on one bot by a few percent, ignore it and update expectations if necessary. This is especially true for older bots, like linux-lowmem and win-xp. You may be able to narrow down bot-specific flakiness by checking the WebKit canary bots too, since they run a similar config.
If you'd like to help keep the Chromium Perf waterfall green, send email to firstname.lastname@example.org. The rotation lasts 3 days every 3-ish weeks. The more that sign up, the less often the rotation will happen.
CalendarYou can view the current rotation by adding this calendar: google.com_2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com. Instructions on swapping shifts are here.