In order to reduce the size of the Chromium source checkout, Telemetry stores binaries in Cloud Storage. This includes:
- Recordings of webpages, known as "page set archives" or "WPR archives."
- Support binaries, such as device/host_forwarder, ipfw, minidump_stackwalk, crash_service.
- credentials.json, which is in Cloud Storage to provide access rights.
Many benchmarks require these files to run, and will fail without them.
Set Up Cloud Storage
Follow these instructions to install depot_tools.
Authenticate into Cloud Storage
Some files in Cloud Storage include data internal to Google or its partners. To run benchmarks that rely on this data, you need to authenticate. Run the command below and follow the instructions for authentication with your corporate account.
$ python depot_tools/third_party/gsutil/gsutil config
When prompted with “
What is your project-id?”, just enter
Telemetry has three Cloud Storage buckets you can put binary data in.
cloud_storage.PUBLIC_BUCKET == 'chromium-telemetry'
cloud_storage.PARTNER_BUCKET == 'chrome-partner-telemetry'
cloud_storage.INTERNAL_BUCKET == 'chrome-telemetry'
BSD-compatible support binaries should all go into
PUBLIC_BUCKET so that everyone can use them.
Google wants to avoid legal issues with distributing third-party content, so to be safe, most recordings of websites on the public web go in
PARTNER_BUCKET, which is accessible by Googlers and whitelisted Google partners. Recordings of Google-properties on the public web can go in
PUBLIC_BUCKET, and recordings of unreleased or internal Google websites go in
Upload to Cloud Storage
Upload your files into the bucket “chromium-telemetry”
Put the target file in the directory you want it to be when downloaded from Cloud Storage, say
path/to/target. Use this command to upload:
$ python depot_tools/upload_to_google_storage.py --bucket chromium-telemetry path/to/target
A SHA1 file
path/to/target.sha1 will be generated for each uploaded file.
Check the .sha1 files into the repository
$ git add path/to/target.sha1
Download from Cloud Storage
Download the file in Python
from telemetry.util import cloud_storage
Download the file manually
$ python depot_tools/download_from_google_storage.py -s [target.sha1] -b chromium-telemetry