Chromite guide

Introduction

The chromite shell is intended to be the central clearing house for all build-related commands in Chromium OS.  Before the chromite shell, developers would run a multitude of different shell scripts to perform build steps.  With chromite, we want to have a centralized and testable system for building.


Quick start

If you're in a hurry and just want to try some commands, you can do something like this:

cd ${HOME}/chromiumos
repo init -u http://git.chromium.org/chromiumos/manifest.git [-m minilayout.xml]
repo sync
chromite build x86-generic

That will build an x86-generic image.  Type chromite --help for more info.


Getting chromite

The chromite shell consists of two parts: the chromite wrapper and the main chromite source code.  

The chromite wrapper is installed with depot_tools, so you should already have it if you followed the Chromium OS Developer Guide.  If you don't, try running any gclient command once (like gclient --version), which will force your depot_tools to update itself.

gclient --version

The main chromite source code is part of the Chromium OS source code.  That means it gets updated each time you do a repo sync of your Chromium OS source code.  If this is your first time running chromite, you should repo sync now to make sure you have the newest version.  Assuming your Chromium OS source code is at ~/chromiumos:

cd ~/chromiumos
repo sync


SIDE NOTE: The chromite source code is split up into two parts for a good reason: you want the chromite wrapper in your path (and depot_tools should be in your path), but you want chromite to act based on your current working directory.  This is useful if you have two copies of Chromium OS on your computer: it lets you easily run a different version of chromite on each copy.


Running the chromite shell

Running the chromite shell is simple.  All you need to do is change into any directory in your source tree and run the chromite command.  If your source is at ~/chromiumos, you can do this:

cd ~/chromiumos
chromite


Chromite usage

Every chromite command is a little different.  However, they all have similar ways to specify parameters.

The chroot

All chromite commands have one thing in common: they act upon a given chroot.  If you’re already inside the chroot, that will always be the one acted on.  If you’re not already inside the chroot, the default chroot is the one located in the chroot folder at the root of your source tree.

A "chroot" is described in a "chroot.spec".  This describes how to create the chroot, how to enter it, and where it is located.  Chroot specs are currently located in: chromite/specs/chroot/.  NOTE that eventually the format of chroot specs will change as the underlying commands for building / entering the chroot change.

Interactive mode

In order to (hopefully) be more user friendly and more discoverable, the chromite shell works interactively.  That means that if you run chromite with no arguments, you will be given a menu of commands, like:

Which chromite command:

1. build - Build the chroot (if needed), the packages for a target, and the image.
2. clean - Clean out built packages for a target; if target=host, deletes chroot. 
3. ebuild - Run ebuild.                                                           
4. emerge - Run emerge.                                                           
5. equery - Run equery.                                                           
6. portageq - Run portageq.                                                       
7. shell - Start a shell in the chroot.                                           
8. workon - Run cros_workon.     

Choice (q to quit): 

For "full fledged" chromite commands, you can continue to use interactive mode to use chromite.  These are commands like build and clean.  If you choose one of those, you’ll see something like:

Choose a build spec:

1. HOST               4. x86-dogfood        7. x86-pineview   
2. tegra2_seaboard    5. x86-generic    
3. x86-agz            6. x86-mario      

Choice (q to quit): 

If you choose a less full fledged command (one that’s just a wrapper of another script), interactive mode is less useful since we don’t have menus for everything.  We’ll eventually fall back to asking for arguments.  This is a bit ugly, and I’d love suggestions for what to do there...  This will look like:

arg 1 (blank to exit): foo
arg 2 (blank to exit): bar
arg 3 (blank to exit): 

Running chromite with no command-line at all is not expected to be particularly common.  However, you will see later that interactive mode can even be useful when you specify most things on the command line.

Partly specifying command line arguments

The chromite shell tries to speed developer efficiency by allowing you to partly specify command-line arguments.  For commands, you can use any prefix of the command.  For build specs, you can use any substring.  So, for instance, you can do:

chromite bu mario  # Instead of chromite build x86-mario
chromite sh

Since the above are unique, they run without any interaction from the user.  If you specify things that are less than unique, you’ll get a menu with all possible completions.  At the moment, this might not be too useful for commands (I don’t think people will run chromite e to get a list of commands starting with e).  It’s more useful for boards.  Run chromite build x86 and see that all ARM specs (and the host) are filtered:

Choose a build spec:

1. x86-agz         3. x86-generic     5. x86-pineview
2. x86-dogfood     4. x86-mario   

Build specs

Build specs work a lot like chroot specs.  They can be found in the directory: chromite/specs/build/

Whenever they are read, chromite also reads the _default file in the directory, so build specs be thought of as subclasses of _default.  At the moment, all of our subclasses are effectively blank files.  That’s because the _default build spec can refer to the name of the subclass file like target: %(name)s.  The target is passed directly to various commands.  

That means you can setup a new target that just uses defaults for everything by a simple command like this:

touch chromite/specs/build/arm-generic.spec

Eventually the format of build specs will change as the underlying commands for building change.  The current format is hopefully described pretty well by the _defaults file.

The HOST pseudo-build target

If you looked closely at one of the above menus, you may have noticed a build target called HOST that doesn't seem to fit.  This isn’t really a build target, but was needed because some chromite commands act on the HOST, like emerge and equery.  

In general, commands should do something reasonable if the user chooses HOST as a target.  However, some of these things are little weird.  At the moment, the only two "weird" ones are:

build HOST ==> builds the chroot, but no board.
clean HOST ==> an alias for distclean (deletes the chroot).

Customizing your build

You should feel free to tweak spec files to adjust how your own personal build happens.  For instance:
  • You might want to tweak chromite/specs/build/_defaults to make --oldchromebinary the default for your builds.
  • You might want to tweak chromite/specs/build/x86-mario to pass --noenable_rootfs_verification if you want to turn off rootfs verification just for builds of mario.
  • You might want to tweak chromite/specs/chroot/_defaults to pass --verbose if you like lots of chatty output.
Remember that the build specs will eventually change (as we migrate from being a simple wrapper of existing scripts in src/scripts).

Using an alternate chroot

Whenever we read a chroot spec, we first start off reading the _defaults in the same directory and then treat the chroot.spec as a type of subclass.  The default spec contains enough information in it that a subclass doesn't really need to specify anything--a blank file is a valid chroot spec.  That means you can create an alternate chroot with something as simple as:

touch chromite/specs/chroot/alt.spec

If you’re running chromite from outside the chroot, you can specify to use this alternate chroot by simply passing --chroot=alt as the first argument to chromite.  If you want, you can easily have chromite always use this alternate chroot by doing:

alias chromite="chromite --chroot=alt"

Since the location of a chroot is (by default) the name of the spec file, your alternate chroot will be located in the directory alt in the root of your source tree.


Samples

Here are a few sample commands, and what they do:

# Run chromite completely in interactive mode
chromite

# Run chromite to build mario (mario uniquely matches the build spec x86-mario):
chromite build mario

# Enter the chroot
chromite shell  # (or just chromite sh)

# Build, prompting for which board you want to build w/ a menu
chromite build

# Build, prompting for a menu with just x86 targets (by substring match)
chromite build x86

# Start working on autotest on mario
chromite workon mario start autotest

# Start working on autotest on host
chromite workon host start autotest

# List all cros-workon packages for mario
chromite workon mario list --all

# Special case: chromite recognizes ‘start’ is not a valid board and prompts you.
chromite workon start autotest

# Runs equery on the host
chromite equery host belongs /usr/bin/sudo

# Shows that you can pass commands to the shell
chromite shell echo 'HI'

# More complex ways to pass shell commands, like cros_sdk.
chromite shell -- echo -n 'HI'
chromite shell ABC=def -- bash -c 'echo $ABC'

# Get overall help for chromite...
chromite --help

# Get help for a given command
chromite clean --help

# Get help for a wrapped commands (note: usage is wrong).
chromite workon mario --help


Why do we need the chromite shell?

Why not lots of separate commands?

It seems like you could just have a bunch of scripts, instead of having one main script that calls others.  Some of the reasons for the subcommand architecture are:
  • We can put chromite in the path.  We don’t have to worry about subcommands conflicting with other commands installed on a user’s system.
  • Having build-related tasks in one command makes build commands more "discoverable".  If you know something is related to building, you only have to look through chromite, not other scripts.
  • Syntax for different calling commands should be forced to be more similar to each other.
Perhaps it makes sense to think about other, similar instances.  The git and make commands are probably the most similar.  For instance, git doesn’t have a whole bunch of commands installed on your path.  Instead, you use git branch, git checkout, git push, etc.  

Chromite should be more testable

Since chromite is in python, we can write unit tests for it.  With the unit tests, people should be able to feel more comfortable modifying existing code (instead of adding new scripts that do similar things to other scripts).



Comments