Executive Summary

This document discusses the pros and cons of verious approaches to debugging.  It also provides a summary of various design decisions that have guided the development process up to this point as well as explaining some new conclusions that will influence it in the future.  In particular, an in-process implementation and an out-of-process implementation are compared and the limitations of both approaches considered.  We make the case for the implementation of an out-of-process, client-server architecture for debugging, in the long term, and to discuss how the SDK team in Boulder will aim to accomplish such an architecture without dropping support for any existing debugging implementations.  The goal of this document is to show how our targeted primary architecture solves the problems under discussion more completely and more simply than other approaches that have been tried, especially from the users’ pespective.

Conventions

Some matters discussed in this document can be confusing.  In particular, we are providing a product not for end-users but for developers.  As a result the terminology throughout this document regarding different levels of users need to be clarified.
Developer:  Refers to anyone working on or contributing to the Native Client or Native Client SDK projects.
User:  Refers to a programmer who is a consumer of Native Client and the SDK.  A user may be referred to as a Developer in a situation where he contributes back to the SDK.
End User:  Refers to the person executing the code developed by our us and our users who may or may not have knowledge of any implementation specifics.
Also, since a client-server architecture is introduced early in this document, the term debugger is rarely used.  Instead we use “debug client” for the symbol-parsing, user-facing component of the debugger, and “debug server” for the code-facing, state-machine implementing part of the debugger.

Scope

• In Scope of this document
• A debugging architecture and its variations, that collectively can allow us to debug all supported platforms, presented in overview form.
• A reference architecture, presented in overview form, which our debugging architecture should loosely resemble.
• Design considerations that lead to the development of each form of our debugging architecture.
• Any information necessary to determine which interfaces should be required and reusable.
• Any platform specific details that can derail development of a particular variation of the debugging architecture on a particular platform (force factors).
• A high-level description of the project’s priorities and of a time-line for achieving the project’s goals.
• Out of Scope
• Any platform specific details that don’t contradict or influence our choice debugging architectures for a given platform.
• Any information specific to the implementation of a debugger on a given platform except what is being influenced by force factors.

Requirements

High Level Requirements

The following requirements should apply to the primary architecture chosen for our debugger.  Secondary or fail-over architectures may exist temporarily and may be allowed to fail on one or more of the requirements.

Functional Requirements (F)

1. The primary architecture should allow debugging of a program that is as similar as possible to the program the end-user will run.
2. Symbolic debugging should be possible for the user of a given debugger.
3. The debugging mechanism should be able to attach to code that is already running in case a developer has encountered a bug that is hard to reproduce.
4. Most programmers have a favorite IDE, or other set of development tools, not to mention a favorite operating system.  As a result, our users should be able to debug from that environment.
5. To increase our chances of adoption, developers should not be forced to set up multiple development environments to debug their products on multiple supported platforms so a developer does not need to set up a development environment for each platform they are debugging.  This requirement strongly suggests a remote debugging architecture.
6. Debugging should not influence band-width requirements for nexe providers.  This means that it should be possible to debug a binary that is served without symbol information.
7. The implementation of the debugger should not compromise the security of sel_ldr.  If the debugger is a part of sel_ldr it should be ensured that the user can be prevented from bypassing the sandbox, especially if it allows communication via a socket.

Implementation Requirements (I)

1. To meet requirement F5, the primary architecture should allow for debugging clients to be independent of the architecture being debugged so a nexe can be deployed and debugged on all target architectures from one development environment.
2. In order to enable users to contribute debuggers for their favorite development environments, we should stick to common, well-documented protocols, preferably with options to extend for nacl-specific demands.
3. To keep debuggers as lightweight as possible.  We should focus on implementing a minimal subset of debugging functionality.  We can achieve this by requiring only the basic set of debugging commands.  These are described in more detail under the following document on debugging via a debug stub: https://sites.google.com/a/chromium.org/dev/nativeclient/design-documents/remote-debugging-with-native-client-debug-stub
4. To meet requirements F2 and F3, the server should not be required to parse symbol information as the server has to debug the code the way it is going to be distributed (F1)
5. For the same reason, we should be able to provide a standardized test suite for debug clients and servers.
6. Tests written against a given architecture ought to apply to that architecture on all operating systems so we can test the required interoperability.
7. The architectures on all platforms we support should provide a common interface so we can implement a single plug-in to debug from chrome, though the implementation of such a plug-in may not be very high priority (See High Level Design Decisions)
8. It should be possible to provide a debugger without requiring code to be specifically recompiled to enabled it.
9. The debugger binary should be provided through the SDK.

Deferred Requirements

• Debugging the entire web page and its nexe modules in one application is beyond the scope of this design.  However, a few questions about how it might be done are presented in the “Open Questions” section.  Note that the question of whether to implement debugging in chrome and how soon to do so, could have an impact on the plan to develop debugging extensions for different development environments.

Non-Requirements

• The debugger itself does not have to be completely secure, provided that it is not enabled by default, and there is no way to enable it remotely.

High Level Design Decisions

• What to use as a reference architecture?
• Consideration: Our nexes are generated by a modified linux gcc compiler, they are ELF binaries with optional DWARF symbol information for debugging.
• Decision: It makes sense for us to model our debugging architectures on the most commonly used debugger on UNIX-like environments: gdb.
• Remote Debugging or Local Debugging?
• GDB can be run as a unified local debugger or as a debugging client that parses symbol information and connects to a gdb-server to retrieve state information about the process being debugged.
• Local Debugging Advantages
• Already exists to some extent for debugging trusted modules.
• Development has already been done, though never stabilized, for debugging untrusted modules on linux (though remote exists on windows).
• Easy to set up a development process for an end-to-end local debugger since only one platform is required.
• Manual testing is relatively easy as well
• Local Debugging Drawbacks
• It fails to meet our high level requirement, that users be able to establish a single development environment but still debug on each platform that they intend to support.
• Difficult to test, since the debuggers have no common automated communication interface.
• Remote Debugging Advantages
• Allows users to support all platforms for which a server is provided, while sticking to their favorite development environment.
• Allows us to develop debug servers and debug clients independently of each other.
• Allows manual testing of any server or client we’re working on with a preexisting server or client.
• Allows automated testing of debug servers using a single RSP playback and record mechanism and apply the same test suite to all debugged modules.
• Already exists for windows 64.
• Remote Debugging Drawbacks
• Setting up a development process that allows a server or client module to be written for a new platform requires more work, including, in some cases using 2 machines with separate development environments.
• Decision: Use Remote Debugging
• The biggest decision making factor is the end-user experience.  With a remote debugger, we can provide debug servers for all platforms and the user can use a client on his favorite platform.  This means we can enable users to develop and debug applications even on platforms where they are unlikely to have a development environment (i.e. ARM)
• What interface to use for remote debugging?
• Option 1: Home Brew one
• Advantages
• We could distill an interface that only supports our minimum requirements for debugging
• Drawbacks
• More work to come up with a new spec.
• Specs take a long time to write
• Good specs take even longer
• We have to provide our own public documentation for our spec
• We’ve already implemented RSP and so far have not been limited in providing debugging essentials.
• Option 2: Use RSP
• Advantages
• We get at least some of our debug clients for free, provided we test against an unmodified gdb client.
• The protocol tried and proven.
• Already implemented on windows so we have proven that the protocol can describe the platform equally well.
• Users can find the spec and implement their own clients more easily.
• Allows incremental development of debugging functionality
• We already have a working client (gdb) and can develop Debug Servers while testing them.
• We can develop clients for new platforms by testing them against existing servers and against gdb_server.
• Drawbacks
• The protocol requires a bit of familiarization because it uses a dense format for communication.
• Option 3: Use a different one, such as VS’s Debug Line Protocol
• Generally RSP is the debug protocol that has the best documentation.  Proprietary remote debugging protocols like the one used by Visual Studio are really difficult to track down.  Our users would have a much harder time implementing their own clients.
• Decision:
• Use RSP
• Should we focus on implementing debugger clients or on providing a debugger in chrome?
• Advantages
• It could be used to provide stack-trace information to the user, though that could also be done without a full-featured debugger in chrome.
• Implementing a portable debug client in chrome could save us the work of implementing platform-specific clients.
• Drawbacks
• Nexes with symbol information are unlikely to be provided by NaCl users due to their large size and concerns about the size of site-embedded modules, so full-featured debugging is going to be less frequent for chrome users.
• There are some limitations to the functionality that a chrome build-in debugger would provide to the developer.  Chrome could be extended to act as a debugger but it would take much more work to turn chrome into an effective editor for untrusted code.  Developers who are used to IDEs will want to be able to edit and debug from the same tool.
• Current Decision:  Focus on debug servers, and standardizing a remote debugging interface.  If we also provide one or two examples of debugging clients on commonly used platforms, users may start to contribute their own debugging clients for their favorite development environments.  Debugging through chrome can be provided as an add-on later.
• In-Process Debugger or Out-of-Process Debugger?
• This is a complex issue which is discussed in detail in the following sections. Each architecture will be held up to the set of requirements defined above, along with a brief consideration of its other advantages and disadvantages where appropriate.  We already have some implementations for in-process debug servers, but out-of-process debug servers have a few functional advantages.  The long-term goal will be to develop out-of-process debug servers for all supported architectures in order to provide the best debugging experience for our users without compromising the security or simplicity of sel_ldr.  In order to prevent regression, both architectures will be supported for the near future, though in-process will probably be removed at some point.  An explanation of this conclusion and a discussion of the advantages and disadvantages of each approach will be provided as they are described in more detail.

Architecture Variations

Debug Trusted Modules

• Description:
  There is no particular architecture involved here.  This method involves building a native version of your plug-in on the operating system and debugging it using normal debugging tools.
• Advantages
• This may appeal to developers using highly customized development environments that we can’t reproduce.
• Allows developers to get started with debugging before we can provide untrusted debugging tools.
• We will never able to support all IDEs with NaCl specific debugging tools.  Some are not extensible enough or not used frequently enough to merit support from our engineers.
• We can’t support all native development tools, such as profilers, memory error detectors, cache analysis tools, etc...
• Drawbacks
• Can’t be done on windows because our ppapi interfaces are built on platform specific interfaces which we normally provide to untrusted code.
• May prove equally difficult on ARM devices though they are built on a linux kernel.  Besides, who wants to set up a native build environment and debug on a phone or tablet?
• There are differences between trusted and untrusted code, these have varied over time but the experience has at no point been quite the same for both layers.  This approach can allow a user to use a broader range of tools on their code but, in the end, the user must be allowed to debug the code as it will be run: Untrusted
• Because the code cannot be debugged as it is being run, this approach fails in the primary requirement for an actually debugging solution.  However, because of the flexibility it provides, building an analyzing of trusted plug-ins should be documented.

Reference Architecture for remote debugging: gdb_server with gdb as client

• Viewed from a high level, gdb implements the model described below.  The gdb_server is an out of process debugger, meaning that it launches the application being debugged.  Note that the debugger does not descend into child-processes by default but can always be asked to when it attaches.  Using GDB as a model architecture allows us to take advantage of preexisting specifications and to develop incrementally, always using a preexisting client or server to test the one that we’re developing.
• 

Secondary Architecture: In Process (IP) Debug Server

• In this approach, the debug server is implemented in sel_ldr.  This means that, either via recompilation or a command line flag, sel_ldr can be made to open a debugging port.
• 
• Advantages
• We already have it working on two platforms
• It meets most of the requirements
• Drawbacks
• Fails at least one functional requirement of being able to attach to debug unexpected problems that are hard to reproduce
• Opens a security concern in sel_ldr by making Native Client, an integral part of chrome eventually, contain code that could be exploited to allow debug port access to the host machine
• There are options for making the debugger itself secure but that is hard to do for a tool that allows direct writing of memory, where we have to make assumptions about addresses that are provided to the debug server by the client - references are absolute but pointers are relative.
• Enabling the debugger in sel_ldr in chrome depends on either an environment variable or an about::flag.  Either is easy for the user to forget about, leaving him with an open security hole that could exist for a long time.
• The Debug Server can be made to use a local pipe for communication, rather than a port, but then the ability to debug across platforms would be lost.
• Requirements Met
• Functional:
• 1: The user is able to debug code that is as close to shippable code as native application code.
• 2: Symbolic debugging can be provided by the debug client
• 4: Clients and plugins can be written for various development environments, so this requirement can be met.
• 5: The debug server uses a common protocol so clients can connect from a variety of platforms (But note that this requirement cannot be satisfied by this architecture in concert with the requirement to have a secure sel_ldr).
• 6: Since the user can upload a stripped binary and use a binary with symbol info in the debug client, there is very little impact on binary size and thus bandwidth required by the server.
• Implementation
• 1: Debug clients can be implemented independently
• 2: The debug server/stub implements RSP
• 3: We can provide a minimal debugging protocol to make client implementation more efficient.
• 4: Symbol parsing can be performed in client.
• 5: A standardized test suite can be written against debug servers to ensure that code executes the same way across operating systems (i.e: x86 == x86).
• 6: It may be possible to design a O/S independent test suite for any given architecture, but it’s hard to be sure.
• 7: Since the interface is RSP, there is a single debugging interface.
• 8: It may be possible to provide a debugger that can be enabled via a runtime flag, but there are security concerns that need to be addressed very carefully.
• 9: It should be no problem to ship the binary in the SDK, even if a separate build of sel_ldr is required.
• Requirements Missed
• Functional
• 3: If a problem occurs unexpectedly that is hard to reproduce, it is unlikely that a user would be able to attach to debug it, having to first restart sel_ldr to activate a debugging environment.
• 7: This approach cannot be made secure without breaking F3.  An implementation could be provided that uses internal pipes rather than sockets, but platform interoperability would be sacrificed.
• Implementation
• 7: Implementation of an O/S independent test suite may not succeed.
• 8: It may remain necessary to provide a separate build of sel_ldr so the end-users’ security is not compromised when Native Client is enabled.

Secondary Architecture, Form 2: Chrome as Debug Client

• An alternative form of the architecture is to provide Chrome as the debug client.  This would allow us to focus on using a local handle rather than an open debug port.  The drawbacks to this approach have, however, been outlined already.
• 

Primary Architecture: Out of Process (OOP) Debug Server

• In this approach, the Debug Server is provided as a separate module from sel_ldr.  It either starts or attaches to the instance of sel whose run-time is being debugged.  It may also start chrome and let chrome, in turn, launch sel.  This is the flow indicated below.  To initialize debugging at start-up, debug server is able to set a break-point at the very first instruction of the nexe based on a customized signal, received from sel.  Alternately, the sel process may reserve a specific region in memory for debugging-related information which would allow for run-time attach.
• 
• Advantages
• The only architecture that meets all requirements.
• It allows remote debugging from any given platform
• It doesn’t require security compromises from sel_ldr
• Drawbacks
• Is it worth it?  This involves rewriting debug servers on linux and windows.
• There could be stability concerns. Careful versioning will have to be used, or the debugger implemented in the Native Client repository, to ensure that it consistently interacts as desired with Native Client
• Requirements Met
• Functional:
• 1: The user is able to debug code that is as close to shippable code as native application code.
• 2: Symbolic debugging can be provided by the debug client
• 3: Depending on the exact implementation of communication between the debug server and sel_ldr itself, it should be possible to allow the user to attach and debug an unexpected problem.
• 4: Clients and plug-ins can be written for various development environments, so this requirement can be met.
• 5: The debug server uses a common protocol so clients can connect from a variety of platforms (But note that this requirement cannot be satisfied by this architecture in concert with the requirement to have a secure sel_ldr).
• 6: Since the user can upload a stripped binary and use a binary with symbol info in the debug client, there is very little impact on binary size and thus bandwidth required by the server.
• 7: sel_ldr needs to provide some information needed by the debugger, but none of the mechanisms under discussion involve external communication by sel_ldr itself.
• Implementation
• 1: Debug clients can be implemented independently
• 2: The debug server implements RSP
• 3: We can provide a minimal debugging command subset to make client implementation more efficient.
• 4: Symbol parsing can be performed in client.
• 5: A standardized test suite can be written against debug servers to ensure that code executes the same way across operating systems (i.e: x86 == x86).
• 6: It may be possible to design a O/S independent test suite for any given architecture, but it’s hard to be sure.
• 7: Since the interface is RSP, there is a single debugging interface.
• 8: Since the debugger is provided separately from sel_ldr there should never be a need to recompile sel_ldr.
• 9: The binaries for the debug_server implementations and for the debug_clients can be easily included in the SDK.

Primary Architecture, Form 2: Chrome as Debug Client

• Without modification to the RSP Server, it should be possible to allow chrome to implement a debug client.  Chrome already contains a debugging framework.  Significant effort will be required to integrate an RSP client and symbolic ELF/DWARF debugger into chrome but they can be developed outside of Chrome first.  Perhaps it would be possible to add a version of gdb to chrome as a plugin.  There are open questions about this option below.
• 

Development

Design and Security Considerations

• Would it be possible to skip the RSP Server if a native debugger can be implemented more easily on a given platform?
• Advantages
• It may be possible to save the work of integrating a debugger with the RSP interface.
• Drawbacks
• It will be impossible to apply the same test-suite to all debug cores  and ensure that they implement the same functionality.
• It will be impossible to support chrome as a debug client in the long term since chrome cannot be expected to work with different native binary debuggers as that would require a significant amount of work.
• Our users will never be able to debug any platform from any other platform.  RSP is our primary mechanism for guaranteeing that contract.
• The work of implementing a native symbol parser for that particular debugger would be wasted for the purpose of debugging other platforms.
• Decision
• No.  All debuggers, whether in-process our out-of-process should definitely implement RSP to be interoperable with all debug clients.  This is the only way we can apply the same test suite to all implementations of the debug server.
• There is a security feature on Ubuntu that prohibits a debugger from attaching to a running process.  Should we supply and In-Process debugger on Ubuntu to sidestep this problem?
• Option 1: We could side-step the security feature by developing an in-process debugger on linux that does not attach.
• Advantages
• In-Process debugger is easy to implement
• Drawbacks
• In-Process debugger has the potential to open a serious security hole into sel_ldr.
• The In-Process debugger is only capable of attaching to a problematic nexe after the fact chrome started sel_ldr with a specific flag.
• Decision
• We can do this temporarily but have to think about security and attach in the long term.  May not be releasable.
• Option 2: Develop an OOP Debugger on linux.
• Advantages
• It would be more consistent with our other platforms and would allow us to boil our RSP implementation to one single, portable, implementation.
• Drawbacks
• The implementation of an OOP p-trace debugger may be more complicated.
• Decision
• It is common practice, when setting up a development environment in Ubuntu, to disable this security feature (see: https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#ptrace%20Protection).  Goobuntu is an example of such a development environment.  We can provide documentation to that effect and proceed with our OOP architecture.  However, we may also find a way to solve the security hole without scrapping the in-process debugger.
• The In-Process debugger seems to constitute a security concern.  Is there any feasible way to work around this so we can provide it in a normal build of sel_ldr?
• Can we ignore the security concerns?
• Advantages
• Ignoring the security concern is the only way to provide any-to-any platform debugging, by allowing a way to connect to the debugger externally.
• Drawbacks
• This can never ship in an official release of chrome.
• Can we change the communication to be local, rather than opening a socket?
• Advantages
• This solution could potentially be shippable from a security standpoint.
• Drawbacks
• This solution would not provide any-to-any platform debugging, forcing our users to support multiple development environments.
• Are we concerned about the plugin exploiting the debugger?
• Will either architecture hinder us in developing a chrome debugging plugin?
• Whether the debugger is in-process our out of process, chrome has to launch a custom binary and implement a gdb-like client in order to provide symbolic debugging for the user.
• Chrome always runs sel_ldr on the same machine, so it could use a pipe based or a socket-based debug server equally easily.
• Is it worth it to implement an OOP debug servers on all platforms, even if we can bring up IP debug server more quickly?
• As long as we have no debug_client in chrome, we have to support remote debugging.  Remote debugging with an IP debug server is a security concern because the sel_ldr opens a port that could allow abuse.
• Once we have a debug_client in chrome, debugging from chrome could become our primary means of debugging support.  At that point client and server could always be running on the same platform and the debug_stub could use a PIPE or IPC socket rather than an external socket, but note that such an implementation change would result in the loss of cross-platform interoperability.

Open Questions

• Can a symbolic nexe  debugger be added to chrome?
• Could chrome use gdb as a plugin?  How can this be made portable?
• Would it be more feasible to implement a DWARF debugger and RSP on chrome’s existing debugging interfaces?
• Are they extensible?
• Erik Kay: The extensibility model for the devtools allows you to add panels to the devtools UI and have some access to the underlying web page. The debugger itself is not extensible, so without more work from the devtools team, you'd essentially just have a blank pane to implement your own debugger UI. In any event, you'd want to send mail to chrome-devtools-team when you want to get more info.
• The devtools exist partly in webkit, partly in chromium
• Could a modified gdb serve as chrome’s debug plugin to save us from implementing an entire symbolic debugger?
• A previous implementation of this plugin was build on nacl-gdb.  Definitely not a portable approach.
• What repository should the debug server code live in?
• In process debug servers should definitely stay in nacl repo since it’s a lot of infrastructure work to rip it out.
• Could OOP debug servers be as appropriately implemented in another repo, like the SDK, since they are not trusted code?
• Advantages
• For SDK developers, the build system of the SDK repo is already familiar, making it easier to add build-targets, update installer, and add automated tests.
• Drawbacks
• This would imply putting rsp-playback and rsp-recorder in the SDK which would mean that integration tests for the debug_stub in nacl would be implemented in the SDK. That could make red bots harder to trace.
• Decision?
• Should mac debugger be in-process or out-of-process?
• Since mac has dtrace, which is similar to p-trace an IP debugger may make sense here as well, for the short term.
• Decision
• In the long-term all of our debuggers should be out of process so we can meet an ideal set of requirements, but we may implement a mac debugger in-process as as hold over.
• Can gdb be built to understand ELF/DWARF binaries on mac?
• This will require some experimentation with the gdb build configuration on mac.  If it can be done, we don’t have to implement a symbolic debugger on mac.\
• Will this get hosed by breakpad

Implementation Plan

Generally, implementation should follow the shortest path to a working prototype. Since the in-process debugger is generally easier to implement, we may implement it for various architectures if it can be done quickly.  We can then use the RSP playback and recorder as a test scaffold and ensure the debug_core is implemented in the same way on all platforms.  We will support whatever debugging support has already been implemented and improve the testing framework for our code as we discover flaws.

• Remaining Work
• Testing
• Integration testing via RSP Playback and Recording Mechanism to exercise debug server via RSP
• Implementation
• This still needs work, the code is in an experimental state and work needs to be done to improve quality and ready it for use in the release code base.
• Golden Sessions
• One set per architecture (x86, x86-64, arm)
• Should be interoperable between OSs
• Should be built from the same source code
• Tests can be written using python as RSP clients where appropriate as well.
• Unit Testing
• Any new code written as part of a debug server or debug_client implementation or as part of the integration testing framework should have tests.
• Any time a bug is discovered in inherited code, a test should be provided to reproduce the problem before it is fixed.
• Test Plan for Manual Testing
• A test plan to exercise the debugger by hand in preparation for release cycles.
• Platform support
• Minimal change to support IP Windows 64 Debugger while OOP Windows 64/32 debugger comes on line.
• Fix Bugs
• Add some minimal build framework to easily produce a sel_ldr version with debugger and to prevent regression
• Implement a linux IP debugger
• Implementation of core debugger for linux
• Test scaffolding for core debugger
• Implement a debugger for mac
• Decide on IP vs. OP question
• Implement the appropriate debug server
• Debugging Clients and Development Support
• Microsoft Visual Studio Extension (VSX) plug-in
• Initially support VS 2008
• Move VSX plugin code into the SDK repository
• Integrate VSX plugin code with TCB code in NaCl repo
• In a previous attempt to get the two to inter-operate, changes were made to the debug_stub to make it work with MSVC.  MSVC should actually be made to behave more like gdb and changes to the debug_stub should be rendered unnecessary.
• Add tests to the VSX plugin code so it can be moved from experimental.
• Add the VSX plugin to the SDK installed content on windows
• Port to MSVS 2010
• Linux
• gdb works out of the box as a debug client.
• Eclipse debugging support via gdb on linux
• Automate symbol loading for debugger
• Mac
• XCode debugging support
• Possibly use gdb as debug client
• Chrome
• Implement a portable debug client in chrome debugger or dust off the experimental implementation and see what’s involved in getting it to work.
• Priorities
• Baltic Brill
• Write design documentation (garianov, mlinck)
• End-to-end symbolic debugging client and debug server on windows
• Catch up on Code Management to bring in-process debugger, and VSX code base into a consistent state (mmortensen)
• Find a way to run a VSX debugging plugin via build-bot (mmortensen, mlinck)
• RSP Playback and Recording Mechanism (garianov)
• Add build targets for in-process debugger(s) so it can be distributed in the SDK (mlinck)
• Maintain IP implementation of debug server (mlinck, garianov)
• Baltic Brill and Beyond
• Implement an OOP implementation of debug server to support win32 (garianov)
• Implement a Linux debug server (partially done by mseaborn, not assigned)
• In process debug server and a lightweight tracer implementation
• A testing mechanism for said debug server
• debug server for mac
• debug client for mac
• Support symbolic debugging in chrome (possibly move this up)
• debug server for ARM
• Update Visual Studio support to work with MSVS 2010