Browser Components Design Document
Overview
Browser Components is the project of continuing to modularize the Chrome codebase. A large project to extract the content module out of the src/chrome/ directory is complete. Despite the efforts of this project, the src/chrome/ directory remains by far the largest inter-dependent block of code in the codebase, so continuing to tease the next largest modules out of that directory is the next step.
Objectives
The Chrome project has grown a lot over the last few years in both size and complexity. The number of people contributing to the project has increased a lot, the size of the codebase has increased, and we have added many new target platforms and configurations that were not envisioned in the start. It has also become harder for individuals to understand the whole of the architecture, let alone the whole of the codebase.
The Browser Components project is introducing and enforcing modularity, with these objectives in mind:
- Scale up the architecture to make it more adaptable to new requirements, e.g. new flavors of Chrome that use some features from src/chrome/browser but wish to turn off or reimplement others (e.g. ChromeOS and Chrome for Android);
- Make the intended architecture more apparent through further enforcement of the dependency graph via DEPS rules that specify which includes are allowed and disallowed;
- Significantly simplify the dependency graph of the features currently under src/chrome/, making it less cyclic;
- Speed up developer builds by increasing the number of independent components (production code as well as unit test executables) in the component build, and decreasing the size of some of the largest components with the highest link times such as ‘browser’ and ‘unit_tests’. Smaller unit test executables also make it easier to e.g. step through code in gdb, or iterate quickly on adding printf’s for debugging;
- All of the above should improve separation of concerns and allow our large engineering team to more easily collaborate, make it harder to misunderstand the architecture and write architecturally bad changes, and in general make us more productive.
Design
Design Overview
Q: What are you doing?
A: Extracting bits of src/chrome/ into components that are separate dynamic libraries in the component build, that have a separate unit test target, and that have no cyclic dependencies.
Q: What do you mean, components that have no cyclic dependencies?
A: They know nothing about their embedder (which initially will typically be src/chrome/). If they need information or services from their embedder they either receive them at initialization time, or retrieve them at runtime via an abstract delegate interface defined by the component and implemented by the embedder.
Q: What if one component wants to depend on another?
A: It can. In this case think of one component as the embedder of the other. This precludes the possibility of dependency cycles between two (or any N) components.
Q: Do we want to componentize UI?
A: Not in the sense described above. We do want to eliminate cyclic dependencies on UI; code under src/chrome/browser/ui/ may depend on src/chrome/browser/ and any component, but not vice-versa. Also, code in src/chrome/browser/ui/ should not depend on port-specific code, e.g. src/chrome/browser/ui/cocoa/. The UI code should be a separate dynamic library in the components build. However we do not anticipate ever breaking all of UI into separate, small, reusable components.
Q: What if I’m evaluating some code for componentization and I see that it has dependencies that are currently implemented by the rest of src/chrome/?
A: Usually, that means we first need to componentize those dependencies. There are two exceptions:
- If the dependency is small enough: The preferred approach is to fulfill it via methods on the component’s delegate interface.
- If it is a fairly large dependency on UI functionality which we don’t expect to ever make into a component:
- If the dependency will be used only by one component: Add the ability to access this functionality to the component’s delegate interface(s).
- If the dependency is or will be used by multiple components: You may be tempted to write one delegate interface for that dependency, and reuse it between components. The need for this should be rare or non-existent; such shared delegates tend to be larger than what each component specifically needs, which makes it harder to embed individual components. Our aim is to try very hard to avoid shared delegates, so we only plan to use them if we find a very compelling concrete scenario that needs them, where we agree after discussion they are needed.
Q: Where do components live?
A: If they depend on src/content/ and layers below that (which is typical) they live in a subdirectory of src/components/, e.g. a component named foo would be contained in the directory src/components/foo/. For components that depend only on lower layers than that, we evaluate each time. One example so far is src/base/prefs/, a component that depends only on src/base/.
- Note: Something that is more like a major abstraction layer (think src/content/) or a major product (think src/chrome/) should be its own top-level directory rather than a subdirectory of src/components/.
Q: Where do components’ delegate interfaces live?
A: Their declaration lives in the same directory as the component. Their embedder-specific implementation lives in each embedder, wherever the embedder chooses.
Q: Do components provide an API for the embedder to use?
A: Generally speaking, it is fine for the embedder to use the component as a concrete set of C++ classes, in which case the “API” is simply whatever the “entry point” classes are, where the component receives pointers to implementations of its delegate interfaces from its embedder. In certain cases, we may decide to introduce a fully-abstract API for the embedder to use, that hides implementation details of the component, as we did for src/content/. In certain other cases, we might categorize the component’s concrete C++ classes into an internal set and a publicly-usable set. In either of these cases, the API or publicly-usable set of classes should reside in a subdirectory of the component’s directory named public (e.g. src/components/mycomponent/public/). If a public directory is present, the component’s delegate interface definitions should live within that directory.
Q: Is there any further standardization of the structure within a component’s directory?
A: Some components that depend on the src/content/ layer may have bits of code intended to run in different processes of the embedder, e.g. browser or renderer. The convention for these is to place the code in sub-directories with names matching those already used in src/chrome/, so for a component xyz you might have src/components/xyz/browser/ and src/components/xyz/renderer/ directories.
Q: Is this like Microsoft COM or Mozilla XPCOM?
A: No. It is a non-goal to create binary reusable components similar to Microsoft’s COM or Mozilla’s XPCOM mechanisms. We expect the components we are introducing, and all users of the components, to live in a single codebase and be built together, so we do not need complicated registration and discovery mechanisms, or to limit ourselves exclusively to APIs that are pure vtables.
Component Examples
The content module (src/content/) is a component according to the definitions above; one that is quite large, and that provides a fully-abstract public API.
Smaller examples that illustrate how to componentize can be seen in the work done to implement the following issues:
- Componentize input_method (issue 164375).
- Componentize Prefs (issue 155525).
- Extract FaviconDownloadHelper; this ended up as a feature of the content module rather than as a separate component, but is still instructive (issue 146851).
DEPS Improvements
The DEPS tool has been very useful for maintaining the intended dependency graph in our codebase. We have made a couple of improvements to it to facilitate the Browser Components work:
- The tool previously had up to one set of rules per directory, of dependencies that are blacklisted and/or whitelisted for files residing in that directory and its subdirectories. While the dependents are specified as entire directories, the dependencies may be specified as folders or down to individual files. We added the ability to specify the dependents down to the file level, including down to a glob level (e.g. *.cc may depend on X, Y, Z, whereas *.h may not). This should let us enforce certain models in the dependency graph that otherwise could only be done by moving large numbers of files around, which although sometimes the right thing to do, is not something we want our tools to force us to do.
- The tool previously acted only on the entire state of the codebase. For our purposes, it is very useful to be able to express dependency rules that we aspire to, i.e. “don’t add dependencies to X, Y or Z, even though there are already some.” To achieve this, we added a new syntax to DEPS files that lets you document aspirational dependency rules, and we added a presubmit step that verifies changes as they are uploaded or committed against these aspirational rules.
We have also extended DEPS to allow generation of a visual graph of components and their dependencies. This is helpful when trying to find appropriate “leaves” in the dependency graph under src/chrome/, that make for good next things to componentize.
Automatic Refactoring
We have surveyed the available automatic refactoring tools for C++; the brief conclusions are:
- None of the commercially-available tools seem able to cope with the size of our code base;
- Clang-based refactoring tools have started becoming useful and can deal with our code, but they take a while to run;
- Clang-based tools can be useful for certain types of changes (e.g. renaming a method that has a very common name and exists in multiple classes) but more often than not, directly editing the code or using simpler tools that just parse our source files as text (rather than syntactically) is more efficient.
Simple Refactoring Tools
We have a tool for moving source files that updates include guards and updates references to files (#includes and #imports in other source files, references in .gyp(i) files), see src/tools/git/move_source_file.py.
We may create a tool to move classes into a different namespace, since this is another frequent and sometimes time-consuming operation. This would be a best-effort kind of tool, that could deal reasonably well with uniquely-named classes.
Code Review Conventions
Because the Chrome codebase changes so rapidly, we know from prior experience that all of the above needs to be done using rapid-fire incremental changes. The ability to get quick code reviews from multiple owners and, in many cases, the ability to TBR the trivial parts of changes with wide impact, will be key.
Project Plan
Team & Timeline
Full-time members of the Browser Components team are Jói Sigurðsson <joi@chromium.org>, Erik Wright <erikwright@chromium.org>, Cait Phillips <caitkp@chromium.org> and Kai Wang <kaiwang@chromium.org>.
The project started in mid-Q3 2012. Our expectation is that we can put at least a man-year or two of fast and furious work into this project before reaching diminishing returns. We will re-evaluate quarterly, involving the Chrome TLs in the discussion. We will also attempt to set quarterly goals that fulfill shorter-term needs while driving us forward on our longer-term goals (e.g. for Q4 2012 we have a goal to help the Android WebView folks reuse functionality from Chrome without depending on anything under src/chrome/).
Risks
While a full risk analysis has not yet been done, there are a couple of obvious risks:
- Making life harder for folks who live on a branch and upstream changes. Large refactorings that move files around can make it significantly harder to upstream changes from a branch. We believe paying this cost is reasonable, since we try to avoid branches in general (so it makes sense to make a trade-off that favors quality and maintainability of the trunk over ease of doing development on a branch), and because other factors in this refactoring work make upstreaming easier (e.g. the ability to turn off or replace individual components without dozens of #ifdefs littering the code). In conversation with folks responsible for upstreaming, no concerns were voiced, but rather support for the project.
- Spending too much time and effort on something with no direct user-visible improvements. The general consensus seems to be that the last project similar to this one (extracting the content module) was easily worth the effort and had substantial concrete benefits, although not user-visible. We believe the Browser Components project also has substantial concrete benefits as outlined in the Objectives section. We will also use metrics to keep track of progress and re-evaluate the need for continuing the project quarterly with key stakeholders, to avoid this risk from materializing.
|
