MemorySanitizer

MemorySanitizer (MSan) is a detector of uninitialized reads based on compiler instrumentation (LLVM).
It is EXPERIMENTAL. The only supported platform is x86_64 Linux.
Additional info on the tool is available at http://clang.llvm.org/docs/MemorySanitizer.html. Some information is available from MSan wiki.

MSan bots are running on chromium.memory.fyi and client.webrtcThere are also two LKGR builders for ClusterFuzz: no origins, chained origins (see below for explanation). Blink and V8 deployment is ongoing.

Using MSan

A note on V8: If you're trying to debug a V8-related issue, please keep in mind that MSan builds run V8 in ARM64 mode, as explained below.

If your build includes instrumented libraries (see below), you need to install build dependencies for those before building:

sudo third_party/instrumented_libraries/install-build-deps.sh

This is how we currently build:

GYP_DEFINES='use_goma=1 msan=1 use_instrumented_libraries=1 instrumented_libraries_jobs=20' gclient runhooks
ninja -C out/Release base_unittests

Flags breakdown:
  • msan=1: Enable MSan support in Chromium binaries.
  • use_instrumented_libraries=1: shared libraries which Chrome depends on will be rebuilt from source with MSan instrumentation. Without this flag MSan will produce lots of false reports, making it unusable. This flag is EXPERIMENTAL. Additionally, it's not currently guaranteed to work correctly on systems other than Ubuntu Precise and Ubuntu Trusty.
  • instrumented_libraries_jobs=20: Use multiple parallel jobs when building instrumented libraries. This specifies the number of jobs to be spawned per package, so the total number of jobs will be much higher. If you're not using goma, it's probably best to omit this flag.
Instrumented libraries are enabled with a separate flag because certain smaller targets do not need them. This applies to targets such as pdfium_test and d8, which do not depend on any external libraries. When building those targets you may set this flag, but doing so will not bring any additional benefit. Larger targets like Chrome and most Chromium test binaries will definitely require it.
Note that setting this flag will add a number (<100) of extra steps to the build. These steps will have names such as msan-<package_name> and each of them will build an entire instrumented package, thus they are expected to take much longer than other steps. The build should still take a reasonable time though (perhaps 20 minutes longer than a regular build).

The following flags are implied by msan=1 (i.e. you don't have to set them explicitly):
  • use_custom_libcxx=1Use a just-built, MSan-instrumented libc++ instead of the system-wide libstdc++. This is required to avoid false reports whenever the C++ standard library is used.
  • v8_target_arch=arm64: JavaScript code will be compiled for ARM64 and run on an ARM64 simulator. This allows MSan to instrument JS code. Without this flag there will be false reports.
Run the resulting binaries as usual. Pipe the output through tools/valgrind/asan/asan_symbolize.py to get symbolized reports.

Chrome must not use hardware OpenGL when running under MSan. This is because libgl.so is not instrumented and will crash the GPU process. OSMesa can be used as a software OpenGL implementation, although it is extremely slow. There are several ways to proceed:
  • --use-osmesa --disable-gpu-compositing: This is a reasonable combination of flags which enables Chrome's software compositor (much faster than OSMesa compositing) but still uses OSMesa for things like Flash and WebGL. Be aware that software compositing is not otherwise used by default, so you'll be testing a code path that is not enabled for most users. If you're not happy with that, you can drop the second flag.
  • --disable-gpu: This forces Chrome to use the software path for everything (not just compositing). WebGL will not be supported at all, but other GL-accelerated features may perform better compared to the combination of flags above.
  • --use-osmesa --disable-gl-drawing-for-tests: Use this if you don't care about the actual pixel output. This exercises the default code paths, however expensive OSMesa calls are replaced with stubs (i.e. nothing actually gets drawn to the screen).
If neither flag is specified, Chrome will fall back to the first option after the GPU process crashes with an MSan report.

Origin tracking

MSan allows the user to trade off execution speed for the amount of information provided in reports. This is controlled by the GYP flag msan_track_origins:
  • msan_track_origins=0: MSan will tell you where the uninitialized value was used, but not where it came from. This is the fastest mode.
  • msan_track_origins=1 (default): MSan will also tell you where the uninitialized value was originally allocated (e.g. which malloc() call, or which local variable).
  • msan_track_origins=2: MSan will also report the chain of stores that copied the uninitialized value to its final location. If there are more than 7 stores in the chain, only the first 7 will be reported. This mode is EXPERIMENTAL. Note also that compilation time may increase in this mode.

Suppressions

MSan does not support suppressions by design.

We have a blacklist file which is applied at compile time, and is used mainly to compensate for tool issues. Blacklist rules do not work the way suppression rules do - rather than suppressing reports with matching stack traces, they change the way MSan instrumentation is applied to the matched function. In addition, blacklist changes require a full clobber to take efffect. Please refrain from making changes to the blacklist file unless you know what you are doing.
Comments