AddressSanitizer (ASan) is a fast memory error detector based on compiler instrumentation (LLVM).
It is fully usable for Chrome on Linux and Mac. There's a mostly-functional Windows port in progress too, and you can using SyzyASan.
Additional info on the tool itself is available at http://clang.llvm.org/docs/AddressSanitizer.html.
For the memory leak detector built into ASan, see LeakSanitizer. If you want to debug memory leaks, please refer to the instructions on that page instead.
The Chromium Memory waterfall (not to be confused with the Memory FYI waterfall) contains buildbots running Chromium tests under ASan on Linux (Linux ASan/LSan bots for the regular Linux build, Linux Chromium OS ASan for the chromeos=1 build running on Linux), OS X (both 32 and 64 bits), Chromium OS (x86 and amd64 builds running inside VMs). Linux and Linux Chromium OS bots run with --no-sandbox, but there's an extra Linux bot that enables the sandbox (but disables LeakSanitizer).
The trybots running Chromium tests on Linux and OSX are: linux_asan (everything except browser_tests and content_browsertests), linux_browser_asan (browser_tests and content_browsertests), mac_asan (many tests including browser_tests and content_browsertests), linux_chromeos_asan (the chromeos=1 build running on a Linux machine, many tests including browser_tests and content_browsertests).
You can grab fresh Chrome binaries for Linux built with ASan here.
If you're on mac or linux64, building with ASan is easy:
Make sure you've run tools/clang/scripts/update.sh (see https://code.google.com/p/chromium/wiki/Clang for details).
Start with base_unittests and verify your build is working (see below). Then you can ninja ... chrome or ninja ... browser_tests.
GYP_DEFINES='asan=1 target_arch=x64 host_arch=x64' gclient runhooks
ninja -C out/Release base_unittests
On both Linux64 and Mac, building with ninja instead of make should just work. Make sure to do a release build.
Adding use_allocator=none to GYP_DEFINES also appears to be necessary as of 19/06/2014, but perhaps that's an unintended regression.
DUCT TAPE ALERT! EXPERIMENTAL!
ASan for Windows is not yet production quality. It is based on Clang for Windows and inherits the Clang/Win list of unsupported stuff. You can try it, but it might just be broken for you.
If you are assigned an ASan/Win bug, you might not be able to reproduce the report locally. In this case, you have to look at the stack on the bug, and you have to check in a speculative fix and rely on the FYI bots to tell you if your fix worked.
If you're feeling lucky, you can try using ASan/Win locally by running the steps below:
# You should have CMake installed and added to your PATH
# ASAn requires a 32-bit build on Windows for now; don't set target_arch=x64.
ninja -C out\Release <targets>
Or you may want to override the build directory like so:
ninja -C out_asan\Release <targets>
Please note that if you make a sandboxed [default] run, you have to redirect stderr to a pipe to see ASan output [issue 467021].
Alternatively, you can use --no-sandbox when you run chrome.exe.
ASan builds should work seamlessly with Goma (except for Windows); just add use_goma=1 to your GYP_DEFINES. Don't forget to use ninja -j <jobs> to take advantage of goma.
If you want your stack traces to be precise, you will have to disable inlining:
Note that this incurs a significant performance hit. Please do not do this on buildbots.
If you're working on reproducing ClusterFuzz reports, you might want to add
v8_enable_verify_heap=1 to GYP_DEFINES, e.g.
GYP_DEFINES='... v8_enable_verify_heap=1 ' gclient runhooks
in order to enable the --verify-heap command line flag for v8 in Release builds.
ATTENTION (Linux only): These instructions are for running ASan in a way that is compatible with the sandbox. However, this is not compatible with LeakSanitizer. If you want to debug memory leaks, please use the instructions on the LeakSanitizer page instead.
Now, check that the tool works. Run the following:
out/Release/base_unittests --gtest_filter=ToolsSanityTest.DISABLED_AddressSanitizerLocalOOBCrashTest --gtest_also_run_disabled_tests 2>&1 | tools/valgrind/asan/asan_symbolize.py
The test will crash with the following error report:
==26552== ERROR: AddressSanitizer stack-buffer-overflow on address 0x7fff338adb14 at pc 0xac20a7 bp 0x7fff338adad0 sp 0x7fff338adac8
WRITE of size 4 at 0x7fff338adb14 thread T0
#0 0xac20a7 in base::ToolsSanityTest_DISABLED_AddressSanitizerLocalOOBCrashTest_Test::TestBody() ???:0
#1 0xcddbd6 in testing::Test::Run() testing/gtest/src/gtest.cc:2161
#2 0xcdf63b in testing::TestInfo::Run() testing/gtest/src/gtest.cc:2338
Address 0x7fff338adb14 is located at offset 52 in frame <base::ToolsSanityTest_DISABLED_AddressSanitizerLocalOOBCrashTest_Test::TestBody()> of T0's stack:
This frame has 2 object(s):
[32, 52) 'array'
[96, 104) 'access'
Congrats, you have a working ASan build! :-)
Run chrome under ASan
And finally, have fun with the out/Release/chrome binary. The filter script tools/valgrind/asan/asan_symbolize.py should be used to symbolize the output.
(Note that asan_symbolize.py is absolutely necessary if you need the symbols - there is no built-in symbolizer for ASan in Chrome).
ASan should perfectly work with Chrome's sandbox. You should only need to run with --no-sandbox on Linux if you're debugging ASan.
Note: you have to disable the sandbox on Windows until it is supported.
You may need to run with --disable-gpu on Linux with NVIDIA driver older than 295.20.
You will likely need to define environment variable G_SLICE=always-malloc to avoid crashes inside gtk.
NSS_DISABLE_ARENA_FREE_LIST=1 and NSS_DISABLE_UNLOAD=1 are required as well.
When filing a bug found by AddressSanitizer, please add a label Stability-AddressSanitizer.
ASan's behavior can be changed by exporting the
Note that Chromium sets its own defaults for some options, so the default behavior may be different from that observed in other projects.
On Linux (and soon on Mac) you can build and run Chromium with NaCl under ASan. Untrusted code (nexe) itself is not instrumented with ASan in this mode, but everything else is.
To do this, remove
Pipe chromium output (stderr) through
If you're seeing crashes within nacl_helper_bootstrap, try deleting out/Release/nacl_helper.
It's possible to build and run Chrome tests for iOS simulator (which are x86 binaries essentially) under ASan. Note that you'll need a Chrome iOS checkout for that. It isn't currently possible to build iOS binaries targeting ARM.
Fix chromium.gyp_env to use ASan.
Now build the test binary and run it:Then update Clang and prepare the build env:
You'll see the same report as shown above (see the "Verify the ASan tool works" section), with a number of iOS-specific frames.
Follow AndroidBuildInstructions with minor changes:
Running ASan applications on Android requires additional device setup. Chromium testing scripts take care of this, so testing works as expected:
To run stuff without Chromium testing script (ex. ContentShell.apk, or any third party apk or binary), device setup is needed:
It only needs to be run once per device. It is safe to run it multiple times.
When this is done, the device will run ASan apks as well as normal apks without any further setup.
To run command-line tools (i.e. binaries), prefix them with
This is needed to detect addressability bugs in the ARM code emitted by V8 and running on an instrumented ARM emulator in a 32-bit x86 Linux Chromium. You probably don't want this.
See http://crbug.com/324207 for some context.
First, you need to install the 32-bit chroot environment using the
Second, install the build deps:
You'll need to make two symlinks to avoid linking errors:
Now configure and build your Chrome:
If for some reason you need to build a 32-bit Chrome binary, you'll need to do some magic. The same command is needed to use a custom compiler binary:
AsanCoverage is a minimalistic code coverage implementation built into ASan. For general information see https://code.google.com/p/address-sanitizer/wiki/AsanCoverage
To use AsanCoverage in Chromium, add
Note that renderers and the GPU process will not generate one coverage file per process per module, as would normally be the case. Instead, there will be two
Chrome must be terminated gracefully in order for coverage to work. Either close the browser, or SIGTERM the browser process. Do not do
Now, e.g., to list the offsets of covered functions in the
Please refer to the ASan wiki for more information.