Native Client for ARM is a method for running programs — even malicious ones — safely, on computers that use ARM processors. It's an extension of our earlier work on Native Client for 32-bit x86.
On this page, we describe how Native Client works on ARM. We assume no prior knowledge about the internals of Native Client, on x86 or any other architecture, but we do assume some familiarity with assembly languages in general. If you're in a hurry, you can safely skip the "Aside" sections (highlighted in blue), which provide rationale or details, but aren't critical.
If you're looking for an intro to developing Native Client applications on ARM, this is unfortunately not the page for you. We don't have much ARM developer documentation yet.
In this section, we summarize the relevant parts of the ARM processor architecture. If you're familiar with the details of recent ARM processors, you can safely skip this part.
ARM is one of the older commercial "RISC" processor designs, dating back to the early 1980s. Today, it is used primarily in embedded systems: everything from toys, to home automation, to automobiles. However, its most visible use is in cellular phones. Essentially all modern "smartphones" use ARM processors. In 2009, ARM Ltd. (the company behind the architecture) began targeting the laptop market as well, creating the category of "smartbooks."
Through the years, there have been many revisions of the ARM architecture, written as ARMvX for some version X. Native Client specifically targets the ARMv7-A architecture commonly used in high-end phones and smartbooks. This revision, defined in the mid-2000s, adds a number of useful instructions, and specifies some portions of the system that used to be left to individual chip manufacturers. Critically, ARMv7-A specifies the "eXecute Never" bit, or XN. This pagetable attribute lets us mark memory as non-executable. Our security relies on the presence of this feature.
ARM is a 32-bit processor architecture. While modern ARM chips support several instruction encodings, Native Client focuses on the oldest: a fixed-width encoding where every instruction is 32 bits wide. This dramatically simplifies some of our analyses, as we'll see later. Nearly every instruction can be conditionally executed based on the contents of a (dedicated) condition code register.
ARM processors have 16 general-purpose registers used for integer and memory operations, written r0 through r15. Of these, two have special roles baked in to the hardware:
Other registers are given roles by convention. The only important registers to Native Client are r9 and r13, which are used as the Thread Pointer location and Stack Pointer. When playing this role, they're referred to as tp and sp.
Like other RISC-inspired designs, ARM programs use explicit "load" and "store" instructions to access memory. All other instructions operate only on registers, or on registers and small constants called immediates. Because both instructions and data words are 32 bits, we can't simply embed a 32-bit number into an instruction. ARM programs use three methods to work around this, all of which Native Client exploits:
We'll introduce more details of the ARM instruction set later, as we walk through the system.
Native Client runs an untrusted program, potentially from an unknown or malicious source, inside a sandbox created by a trusted runtime. The trusted runtime allows the untrusted program to "call-out" and perform certain actions, such as drawing graphics, but prevents it from accessing the operating system directly. This "call-out" facility, called a trampoline, looks like a standard function call to the untrusted program, but it allows control to escape from the sandbox in a controlled way.
The untrusted program and trusted runtime inhabit the same process, or virtual address space, maintained by the operating system. To keep the trusted runtime behaving the way we expect, we must prevent the untrusted program from accessing and modifying its internals. Since they share a virtual address space, we can't rely on the operating system for this. Instead, we isolate the untrusted program from the trusted runtime.
Unlike modern operating systems, we use a cooperative isolation method. Native Client can't run any off-the-shelf program compiled for an off-the-shelf operating system. The program must be compiled to comply with Native Client's rules. The details vary on each platform, but in general, the untrusted program:
We can't simply take the program's word that it complies with these rules — we call it "untrusted" for a reason! Nor do we require it to be produced by a special compiler; in practice, we don't trust our compilers either. Instead, we apply a load-time validator that disassembles the program. The validator either proves that the program complies with our rules, or rejects it as unsafe. By keeping the rules simple, we keep the validator simple, small, and fast. We like to put our trust in small, simple things, and the validator is key to the system's security.
In the original Native Client system for the x86, we used unusual hardware features of that processor (the segment registers) to isolate untrusted programs. This was simple and fast, but won't work on ARM, which has nothing equivalent. Instead, we use pure-software fault isolation.
We use a fixed address space layout: the untrusted program gets the lowest gigabyte, addresses 0 through 0x3FFFFFFF. The rest of the address space holds the trusted runtime and the operating system. We isolate the program by requiring every load, store, and indirect branch (to an address in a register) to use a pseudo-instruction. The pseudo-instructions ensure that the address stays within the sandbox. The indirect branch pseudo-instruction, in turn, ensures that such branches won't split up other pseudo-instructions.
At either side of the sandbox, we place small (8KiB) guard regions. These are simply areas in the process's address space that are mapped without read, write, or execute permissions, so any attempt to access them for any reason — load, store, or jump — will cause a fault.
Finally, we ban the use of certain instructions, notably direct system calls. This is to ensure that the untrusted program can be run on any operating system supported by Native Client, and to prevent access to certain system features that might be used to subvert the sandbox. As a side effect, it helps to prevent programs from exploiting buggy operating system APIs.
Let's walk through the details, starting with the simplest part: loads and stores.
All access to memory must be through load and store pseudo-instructions. These are simply a native load or store instruction, preceded by a guard instruction.
Each load or store pseudo-instruction is similar to the load shown below. We use abstract "placeholder" registers instead of specific numbered registers for the sake of discussion. rA is the register holding the address to load from. rD is the destination for the loaded data.
bic rA, #0xC0000000
The first instruction, bic, clears the top two bits of rA. In this case, that means that the value in rA is forced to an address inside our sandbox, between 0 and 0x3FFFFFFF, inclusive.
The second instruction, ldr, uses the previously-sandboxed address to load a value. This address might not be the address that the program intended, and might cause an access to an unmapped memory location within the sandbox: bic forces the address to be valid, by clearing the top two bits. This is a no-op in a correct program.
This illustrates a common property of all Native Client systems: we aim for safety, not correctness. A program using an invalid address in rA here is simply broken, so we are free to do whatever we want to preserve safety. In this case the program might load an invalid (but safe) value, or cause a segmentation fault limited to the untrusted code.
Now, if we allowed arbitrary branches within the program, a malicious program could set up carefully-crafted values in rA, and then jump straight to the ldr. This is why we validate that programs never split pseudo-instructions.
The first instruction, tst, performs a bitwise-AND of rA and the modified immediate literal, 0xC0000000. It sets the condition flags based on the result, but does not write the result to a register. In particular, it sets the Z condition flag if the result was zero — if the two values had no one-bits in common. In this case, that means that the value in rA was an address inside our sandbox, between 0 and 0x3FFFFFFF, inclusive.
The second instruction, ldreq, is a conditional load if equal. As we mentioned before, nearly all ARM instructions can be made conditional. In assembly language, we simply stick the desired condition on the end of the instruction's mnemonic name. Here, the condition is EQ, which causes the instruction to execute only if the Z flag is set.
Thus, when the pseudo-instruction executes, the tst sets Z if (and only if) the value in rA is an address within the bounds of the sandbox, and then the ldreq loads if (and only if) it was. If rA held an invalid address, the load does not execute, and rD is unchanged.
ARM has an unusually rich set of addressing modes. We allow all but one: register-indexed, where two registers are added to determine the address.
We permit simple loads and stores, as shown above. We also permit displacement, pre-index, and post-index stores:
In each case, we know rA points into the sandbox when the ldr executes. We allow adding an immediate displacement to rA to determine the final address (as in the first two examples here) because the largest immediate displacement is ±4095 bytes, while our guard pages are 8192 bytes wide.
We also allow ARM's more unusual load and store instructions, such as load- and store-multiple, coprocessor memory operations, etc.
There's one problem with the pseudo-instructions shown above: they are unconditional (assuming rA is valid). ARM compilers regularly use conditional loads and stores, so we should support this in Native Client. We do so by defining alternate, predicable pseudo-instructions. Here is a conditional store (store-if-greater-than) using this pseudo-instruction sequence:
In C-like languages, the stack is used to store return addresses during function calls, as well as any local variables that won't fit in registers. This makes stack operations very common.
Native Client does not require guard instructions on any load or store involving the stack pointer, sp. This improves performance and reduces code size. However, ARM's stack pointer isn't special: it's just another register, called "sp" only by convention. To make it safe to use this register as a load or store address without guards, we add a rule: sp must always contain a valid address.
We enforce this rule by restricting the sorts of operations that programs can use to alter sp. Programs can alter sp by adding or subtracting an immediate, as a side-effect of a load or store:
These are safe because, as we mentioned before, the largest immediate available in a load or store is ±4095. Even after adding or subtracting 4095, the stack pointer will still be within the sandbox or guard regions.
Any other operation that alters sp must be followed by a guard instruction. The most common alterations, in practice, are addition and subtraction of arbitrary integers:
The bic is similar to the one we used for conditional loads and stores, and serves exactly the same purpose: after it completes, sp is a valid address.
The thread pointer and IRT thread pointer are stored in the trusted address space. All uses and definitions of r9 from untrusted code are forbidden except as follows:
ldr Rn, [r9] ; load use thread pointer.
ldr Rn, [r9, #4] ; load IRT thread pointer.
By extension, we also allow loads through the pc without a mask. The explanation is quite similar:
We do not allow pc-relative stores, because they look suspiciously like self-modifying code, or any addressing mode that would alter the pc as a side effect of the load.
There are two types of control flow on ARM: direct and indirect. Direct control flow instructions have an embedded target address or offset. Indirect control flow instructions take their destination address from a register. The b (branch) and bl (branch-with-link) instructions are direct branches and calls, respectively. The bx (branch-exchange) and blx (branch-with-link-exchange) are the indirect equivalents.
Because the program counter (pc) is simply another register, ARM also has many implicit indirect control flow instructions. Programs can operate on the pc using add or load, or even outlandish (and often specified as having unpredictable-behavior) things like multiply! In Native Client we ban all such instructions. Indirect control flow is exclusively through bx and blx. Because all of ARM's control flow instructions are called "branch" instructions, we'll use the term "indirect branch" from here on, even though this includes things like virtual calls, returns, and the like.
Indirect branches present two problems for Native Client:
Checking both of these for direct branches is easy: the validator just pulls the (fixed) target address out of the instruction and checks what it points to.
For indirect branches, we can address the first problem by simply masking some high-order bits off the address, like we did for loads and stores. The second problem is more subtle. Detecting every possible route that every indirect branch might take is difficult. Instead, we take the approach pioneered by the original Native Client: we restrict the possible places that any indirect branch can land. On Native Client for ARM, indirect branches can target any address that has its bottom four bits clear — any address that's 0, mod 16. We call these 16-byte chunks of code bundles. The validator makes sure that no pseudo-instruction straddles a bundle boundary. Compilers must pad with nops to ensure that every pseudo-instruction fits entirely inside one bundle.
Here is the indirect branch pseudo-instruction. As you can see, it clears the top two and bottom four bits of the address:
This particular pseudo-instruction (a bic followed by a bx) is used for computed jumps in switch tables and returning from functions, among other uses. Recall that, under ARM's modified immediate rules, we can fit the constant 0xC000000F into the bic instruction's immediate field: 0xC000000F is the 8-bit constant 0xFC, rotated right by 4 bits.
The other useful variant is the indirect branch-with-link, which is the ARM equivalent to "call:"
This is used for indirect function calls — commonly seen in C++ programs as virtual calls, but also for calling function pointers in C.
Note that both indirect branch pseudo-instructions use bic, rather than the tst instruction we allow for loads and stores. There are two reasons for this:
On ARM, there is no "call" or "return" instruction. Calls are simply branches that just happen to load a return address into lr, the link register. If the called function is a leaf (that is, if it calls no other functions before returning), it simply branches to the address stored in lr:
If the function called other functions, however, it had to spill lr onto the stack. On x86, this is done implicitly, but it is explicit on ARM:
There are two things to note about this code. First, as we mentioned before, we don't allow arbitrary instructions to write to the Program Counter, pc. Thus, while a traditional ARM program might have popped directly into pc to end the function, we require a pop into a register, followed by a pseudo-instruction.
Second: function returns really are just indirect branches, with the same restrictions. This means that functions can only return to addresses that are bundle-aligned: 0 mod 16.
The implication here is that calls — the branches that enter functions — must be placed at the end of the bundle, so that the return address they generate is 0 mod 16. Otherwise, when we clear the bottom four bits, the program would enter an infinite loop! (Native Client doesn't try to prevent infinite loops, but the validator actually does check the alignment of calls. This is because, when we were writing the compiler, it was annoying to find out our calls were in the wrong place by having the program run forever!)
In the section where we described the ARM architecture, we mentioned ARM's unusual immediate forms. To restate,
Here's a typical example of the use of a literal pool. ARM assemblers typically hide the details — this is the sort of code you'd see produced by a disassembler, but with more comments.
Because table is a static array, the compiler knew its address at compile-time — but the address didn't fit in a modified immediate. (Most don't.) So, instead of loading an immediate into r0 with a mov, we stashed the address in the code, generated its address using pc, and loaded the constant. ARM compilers will typically group all the embedded data together into a literal pool. These typically live just past the end of functions, where they won't be executed.
This is an important trick in ARM code, so it's important to support it in Native Client...but there's a potential flaw. If we let programs contain arbitrary data, mingled in with the code, couldn't they hide malicious instructions this way?
The answer is no, because the validator disassembles the entire executable region of the program, without regard to whether the programmer said a certain chunk was code or data. But this brings the opposite problem: what if the program needs to contain a certain constant that just happens to encode a malicious instruction? We want to allow this, but we have to be certain it will never be executed as code!
As we discussed in the last section, ARM code in Native Client is structured in 16-byte bundles. We allow literal pools by putting them in special bundles, called data bundles. Each data bundle can contain 12 bytes of arbitrary data, and the program can have as many data bundles as it likes.
Each data bundle starts with a breakpoint instruction, bkpt. This way, if an indirect branch tries to enter the data bundle, the process will take a fault and the trusted runtime will intervene (by terminating the program). For example,
So, we have a way for programs to create an arbitrary, even dangerous, chunk of data within their code. We can prevent indirect branches from entering it. We can also prevent fall-through from the code just before it, by the bkpt. But what about direct branches straight into the middle?
The validator detects all data bundles (because this bkpt has a special encoding) and marks them as off-limits for direct branches. If it finds a direct branch into a data bundle, the entire program is rejected as unsafe. Because direct branches cannot be modified at runtime, the data bundles cannot be executed.
So far, the rules we've described make for boring programs: they can't communicate with the outside world!
We fix this by allowing the untrusted program to call into the trusted runtime using a trampoline. A trampoline is simply a short stretch of code, placed by the trusted runtime at a known location within the sandbox, that is permitted to do things the untrusted program can't.
Even though trampolines are inside the sandbox, the untrusted program can't modify them: the trusted runtime marks them read-only. It also can't do anything clever with the special instructions inside the trampoline — for example, call it at a slightly offset address to bypass some checks — because the validator only allows trampolines to be reached by indirect branch (or branch-with-link). We structure the trampolines carefully so that they're safe to enter at any 0 mod 16 address.
The validator can detect attempts to use the trampolines because they're loaded at a fixed location in memory. Let's look at the memory map of the Native Client sandbox.
The sandbox is always at virtual address 0, and is exactly 1GiB in size. This includes the untrusted program's code and data, the trampolines, and a small guard region to detect null pointer dereferences. In practice, the untrusted program takes up a bit more room than this, because of the need for additional guard regions at either end of the sandbox.
Within the trampolines, the untrusted program can call any address that's 0 mod 16. However, only even slots are used, so useful trampolines are always 0 mod 32. If the program calls an odd slot, it will fault, and the trusted runtime will shut it down.
When we introduced trampolines, we mentioned that they can do things that untrusted programs can't. To be more specific, trampolines can jump to locations outside the sandbox. On ARM, this is all they do. Here's a typical trampoline fragment on ARM:
The only odd thing here is that we push the incoming value of lr, and then use blx — not bx — to escape the sandbox. This is because, in practice, all trampolines jump to the same routine in the trusted runtime, called the syscall hook. It uses the return address produced by the final blx instruction to determine which trampoline was called.
To complete the sandbox, the validator ensures that the program does not try to use certain forbidden instructions.
If an instruction can't be decoded at all within the ARMv7-A instruction set specification, it is forbidden.
ARM has traditionally added new instruction set features through coprocessors. Coprocessors are accessed through a small set of instructions, and often have their own register files. Floating point and the NEON vector extensions are both implemented as coprocessors, as is the MMU.
We're confident that the side-effects of coprocessors in slots 10 and 11 (that is, floating point, NEON, etc.) are well-understood. These are in the coprocessor space reserved by ARM Ltd. for their own extensions (CP8-CP15), and are unlikely to change significantly. So, we allow untrusted code to use coprocessors 10 and 11, and we mandate the presence of at least VFPv3 and NEON/AdvancedSIMD. Multiprocessor Extension, VFPv4, FP16 and other extensions are allowed but not required, and may fail on processors that do not support them, it is therefore the program's responsibility to validate their availability before executing them.
We don't allow access to any other ARM-reserved coprocessor (CP8-9 or CP12-15). It's possible that read access to CP15 might be useful, and we might allow it in the future — but again, it's easier to loosen the restrictions than tighten them, so we ban it for now.
We do not, and probably never will, allow access to the vendor-specific coprocessor space, CP0-CP7. We're simply not confident in our ability to model the operations on these coprocessors, given that vendors often leave them poorly-specified. Unfortunately this eliminates some legacy floating point and vector implementations, but these are superceded on ARMv7-A parts anyway.