IntroductionThe VMTest "smoke suite" is a suite of client-side autotests that performs basic sanity checks on a Chromium OS build.
Each of the tests in the smoke test suite is capable of running in a virtual machine.
This allows the smoke suite to be run as part of the automated build process on a 'builder'.
Smoke Suite testsThe Smoke suite currently consists of the following tests (this list is subject to change):
Running the Smoke Test Suite on a Virtual Machine (VM)Useful Links:The instructions on this page should be enough to get you going. They install a testable Chromium OS image on a VM and run the smoke suite. However, if you want more details, you can always go to one of the following pages:
Install qemu-kvmFirst of all, to run in a vm, you must have vm software installed.
On Ubuntu Lucid, use apt-get to install qemu-kvm:
$ sudo apt-get install qemu-kvmBuild a testable VM imageWARNING: After crbug/710629, 'betty' is the only board regularly run through pre-CQ and CQ VMTest and so is the most likely to work at ToT. 'betty' is based on 'amd64-generic', though, so 'amd64-generic' is likely to also work for most (non-ARC) tests. Next, build the testable VM image. The following steps assume that:
$ cros_sdk # To enter the chroot
# Setup the build environment for the target board. ${BOARD} could should usually be 'betty' or 'amd64-generic'.
(chroot)$ ./setup_board --board=${BOARD}
# Build all the source packages, including those required for running autotests
# Optional: "--oldchromebinary" may make your build faster, but will have old Chrome/libcros # NOTE: At the moment, "--withautotest" is the default, so don't worry if you built without it. (chroot)$ ./build_packages --board=${BOARD}
# Build a bootable image # Optional: Include "--noenable_rootfs_verification" if you think you might need to modify your rootfs. (chroot)$ ./build_image --board=${BOARD} test# Clone this image, modify it for test, and make image for use in qemu-kvm Virtual Machine
# Note: because we use "--test_image", an explicit "modify_image_for_test" is not required.
(chroot)$ ./image_to_vm.sh --board=${BOARD} --test_image# Exit chroot (chroot)$ exit The newly created VM image should be in the following path (relative to src/scripts):
../build/images/${BOARD}/latest/chromiumos_qemu_image.bin Run the autotest tests on the VM imageAutotests can be run against the VM via test_that.
# Run the entire smoke suite (chroot)$ test_that --board=${BOARD} localhost:9222 suite:smoke Don't forget to kill the head-less VM when you are done with it!
(chroot)$ cros_vm --stop Run the autotest tests on the VM image using cros_run_vm_testAnother script, cros_run_vm_test, can be used to start the VM, run autotest tests on it, and then shut down the VM, all from a single invocation:
# Run the entire smoke suite: (chroot)$ mkdir results (chroot)$ cros_run_vm_test --board=${BOARD} --results-dir=results --autotest suite:smoke
# Run an individual test from the suite (chroot)$ cros_run_vm_test --board=${BOARD} --results-dir=results --autotest platform_OSLimitsTroubleshootingIf you get an error that looks like this:
Unhandled OSError: [Errno 28] No space left on device: '/home/autotest/tmp/_autotmp_Woz_Qyharness-fifo' ...this may be because you have run out of space on your "stateful partition". This can happen if you leave the VM running for a long time and errors fill up the /var/log folder. In my case, this folder contained 250M of data! You should go into /var/log and cleanup stuff.
Use
df to determine how much space is available in the stateful partition:localhost ~ # df /mnt/stateful_partition Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 2064208 450612 1508740 23% /mnt/stateful_partition
You may be getting lots of errors if you're running a board-specific image with drivers for hardware that is not present on the VM. If this is the case, try stopping the responsible daemon with
stop <daemon_name>. Could not set up host forwarding rule 'tcp::9222-:22'If you get this error when running cros_vm --start, make sure that you don't have a VM already running. This should kill it for you.cros_vm --stop Could not initialize KVM, will disable KVM supportThis is probably because you don't have the kvm module loaded. The VM will run.....very.......slowly.open /dev/kvm: No such file or directory Could not initialize KVM, will disable KVM support $ modprobe kvm_intel # if you have an AMD machine, use 'kvm_amd' Note that this module interferes with VirtualBox so you may need to remove it ( rmmod kvm_intel ) if you want to use this.
Updating a client autotestAutotest tests are usually built and compiled on the autotest server machine.
A 'client' test is installed on the client by the autotest infrastructure when the test is run.
Individual tests and test suites are in the chromeos-base/autotest-tests ebuild.
The new way: Use cros_workon with all Python testsWorking on all Python autotests should be simple and fast.
First
cros_workon autotest for the board on which you plan to run tests:(chroot)$ cros_workon start autotest --board=${BOARD} Now, test_that will detect that you are working on autotests and automatically copy over Python in the autotest source tree to the sysroot. Run test_that as usual:
(chroot)$ test_that --board=${BOARD} localhost:9222 suite:smokeThe old way: emerge individual autotest testsIf you have a test with binary dependencies, you must rebuild them explicitly:
(chroot)$ emerge-${BUILD} autotest-tests This can take a very, very long time since it will rebuild ALL tests.
To speed this up, set the TESTS variable on the command line when runnning emerge to install an individual test:
(chroot)$ TESTS=<TestName> emerge-${BOARD} autotest-tests Note that doing this will deactivate all other tests! This is a problem for test suites.
To rebuild a whole test suite, specify the suite and all individual tests in the suite. For example, to rebuild smoke suite:
(chroot)$ TESTS='build_RootFilesystemSize desktopui_ChromeFirstRender \ desktopui_FlashSanityCheck desktopui_KillRestart logging_CrashSender \
login_Backdoor login_BadAuthentication login_CryptohomeIncognitoMounted \ login_CryptohomeMounted login_LoginSuccess platform_FilePerms platform_OSLimits' \
emerge-${BOARD} autotest-tests
Then go grab a coffee...
DebuggingYou can use chromium os debugging tips on VM as well. |
Chromium OS > Testing Home >