Better Software Development with Replay Debugging

Wednesday, September 14, 2011

Goodbye, Replay Debugging…

Note: I'm speaking for myself on this blog, not my employer.

You may have noticed that there hasn't been much action at replaydebugging.com recently. Well, there's a reason. As of VMware Workstation 8 (released September 14, 2011), replay debugging no longer exists. Both the Windows and Linux flavors of replay debugging are no more. But I should point out that live debugging, assisted via the VMware Visual Studio add-on, continues to work as it always has.

The Workstation management team has decided to focus on features that reach a wider audience, including developers. After the Workstation 7.1 release, we identified the next set of improvements that needed to be made to Replay Debugging to evolve it into the system we wanted it to be. We wanted to include multi-processor VM support, support additional high-level languages, support debugging of system code, and continue to improve performance and usability. Our effort estimate turned out to be rather significant (not surprising given the epic scope of the problem that we tackled in the first place). The Workstation management team concluded that not enough people had demonstrated the need or invested the time necessary to configure and use the feature, so they decided to dedicate our engineers to features that would be used by more developers and other customers. I'm disappointed, but I understand and respect their decision.

I'm grateful for the opportunity given to our team to build our replay debugging system, and I'm proud of what we've produced. We have some truly heroic developers on our team that helped us achieve what at first blush appeared to be impossible. It has been extremely gratifying working with customers who have derived benefit from our little project. And it's been great hearing your stories and feedback and successes.

To those of you who have come to rely on replay debugging, I offer my apologies. But I think we've proven the practicality and utility of replay debugging systems, and I suspect others will build similar great products to help you solve your most challenging development puzzles.

Happy hacking.

Wednesday, November 25, 2009

Preparing to Replay Debug. . .

This blog post provides some basic info about how to set up your host/guest/VS for replay debugging. Please note that things work a bit differently in WS 7 versus 6.5. Failure to follow these steps will likely result in an unproductive replay debugging experience. But it'll be worth it! Here we summarize each section.

  • Preparing the Host and Guest. This section describes how you must prepare your host and guest systems for replay debugging. It is critical that you carefully read and obey this section.
  • Configuring Visual Studio. This section describes how Visual Studio must be configured to use replay debugging. Note that this section contains only enough information to get started. Please see the Manual for a more complete treatment of the many ways Visual Studio can be configured for replay debugging.
  • Creating a Recording. Before you can replay and debug a recording, you must create a recording. This section describes several ways that this can be achieved.
  • Starting Replay Debugging. Once you have a recording and Visual Studio has been configured to be aware of it, you can debug programs running in this recording. This section describes how to kick off a replay debugging session.
  • Why All the Preparing? This section helps you understand why the host and guest must be prepared for replay debugging.

Preparing the Host and Guest

You cannot simply use replay debugging out of the box. You must prepare your host and guest systems first. Fortunately, none of the preparation steps are difficult. Failure to carefully complete these steps will likely result in an unacceptable and unproductive replay debugging experience. I'm serious here!

Preparing the Host

You can't use just any old host for replay debugging. It must have the appropriate hardware and software.

Requirements.

  1. The host must have a replay-capable CPU. Use VMware Workstation to create and replay a recording to confirm that you have a replay-capable processor.
  2. Windows XP, Vista, or 7 must be installed on the host.
  3. Visual Studio 2005 or 2008 must be installed on the host.
  4. VMware Workstation 7.0 must be installed on the host. Please note that Workstation must be installed after Visual Studio so that the VMware Integrated Virtual Debugger Plugin for Visual Studio is properly installed.
Ensure guest DLLs are available on the host. All DLLs used by the programs you debug in the guest must be accessible on the host (please see "Why All the Preparing?" below for an explanation of why this is so). The DLLs that are built as part of your Visual Studio project will automatically be available on both the host and guest, but all other DLLs (including system DLLs) must be copied from the guest to the host. In particular, you will at least want to copy c:\windows\system32 on the guest to a folder on the host (e.g., c:\guestdlls\system32).

Preparing the Guest

You also need to get the guest in order.

Requirements.

  1. The virtual machine must have exactly one virtual CPU. We inherit this limitation from the record/replay technology on which replay debugging is built.
  2. 32-bit Windows XP, Server 2003, Vista, or 7 must be installed on the guest.
  3. Guest tools corresponding to the version of VMware Workstation on the host must be installed in the guest.
Disable paging of kernel-mode stacks. Paging of kernel-mode stacks is disabled by changing a registry setting. This is most simply performed with the Global Flags utility distributed with the Debugging Tools for Windows package (Microsoft Debugging Tools for Windows Page (http://www.microsoft.com/whdc/devtools/debugging/default.mspx)). You need not install this package in the guest; instead, install it in the host and copy the Global Flags utility (you'll find it at C:\Program Files\Debugging Tools for Windows\gflags.exe) to the guest. Run the Global Flags utility in the guest, select Disable paging of kernel stacks under the System Registry tab, and click "OK". You must reboot the guest for this change to take effect.

Copy necessary DLLs from the host to the guest. You must ensure that any DLLs used by the program you intend to debug are available in the guest (that's where your program will be running after all!). System DLLs are already in the guest and DLLs built by your Visual Studio project will automatically be available in the guest, but any other DLLs must be manually copied into the guest; drag and drop from the host to the guest works well for this. For simple programs you won't need to do this.

Take a snapshot. This will serve as the starting point for recordings you create and will obviate the need to power on the virtual machine every time we want to create a recording. Let's call this snapshot "BaseSnapShot".

Configuring Visual Studio

Now it's time to get Visual Studio ready for replay debugging. I'm assuming you've already created a project, built it, and run it locally.

Use Microsoft symbol server. Visual Studio interacts with the replay debugging infrastructure much better when it has access to Microsoft symbols. In Visual Studio go to Tools > Options... An options window will appear. In the left pane, select Debugging > Symbols. Add a new symbol file location by clicking on the "folder" icon. Enter http://msdl.microsoft.com/download/symbols (type it carefully!). For Cache symbols from the symbol server to this directory: enter some directory, like C:\mysyms.

Don't use debugging DLLs. By default Visual Studio generates programs that use debugging DLLs that are not available on most machines. In Visual Studio go to Project > Properties... The Property Pages window appears. In the left pane, select Configuration Properties > C/C++ > Code Generation. In the right pane, change Runtime Library to Multi-threaded Debug (/MTd). Make sure you rebuild your project and run it locally to confirm that all is well.

Configuring the Integrated Virtual Debugger (IVD)

We're almost there. Now we need to configure the VMware Visual Studio plugin (the Integrated Virtual Debugger or IVD). Note that we'll only consider the essential configuration options here. A more complete treatment can be found in the Manual. Here's the recipe...

  1. Open the IVD options (VMware > Options...).
  2. In the left pane, click on Replay Debugging in VM > General.
  3. Ensure that Local or Remote (in the right pane) is set to Local (we'll talk about remote debugging later).
  4. In the right pane, enter the path to your virtual machine's .vmx file in the Virtual Machine field.
  5. In the left pane, click on Replay debugging in VM > Pre-Record Event.
  6. In the right pane, enter the name of your base snapshot ("BaseSnapShot") in the Base Snapshot for Recording field.
  7. Click "OK".
If you were paying attention you probably saw configuration options for recording name, guest login credentials, etc. These are important, but you'll be prompted for these as you go, and they'll automatically be filled in for us.

Creating a Recording

Let's create a recording. Sure, you could always manually power on your virtual machine, copy your program from the host to the guest, start a recording, run your program, wait for it to finish, stop the recording, and update the Integrated Virtual Debugger options to refer to this new recording. Sure, you could do that! But the IVD automates the process for us.

Creating a Recording via Visual Studio. Ensure that your project is built and go to VMware > Create Recording for Replay. That's it. You'll be prompted for the guest login credentials and then you just wait for your program to run as the recording is created.

Trouble shooting. If your program never appears to start in the guest, it may be because the program can't run in the guest (e.g., due to a missing DLL). To diagnose this you may want to copy your program from the host to the guest and try to run it.

Starting Replay Debugging

The time has arrived! Let's start debugging. Set a breakpoint in the first line of main() and go to VMware > Start Replay Debugging in VM. Replay will start. You will be prompted to enter the folder containing various DLLs (e.g., ntdll.dll). Enter the name of the host folder in which you placed guest DLLs (in this tutorial it is c:\guestdlls\system32). Don't worry, the configuration options will be updated with this folder so you won't be asked again (unless a different DLL can't be found in this folder). Eventually your breakpoint will be hit. That's it, you're debugging. You can do almost anything you could do in traditional debugging. Play around. Have some fun. Don't forget to check out VMware > Reverse Continue to simulate executing in reverse.

Why All the Preparing?

Above we asked you (among other things) to ensure that Visual Studio is configured to use the Microsoft symbol server, DLLs used by your program are available in both the guest and the host, and paging of kernel-mode stacks is disabled. Why? It mostly comes down to one thing... accessing memory in the guest. From the virtualization layer we only have access to guest physical memory. When you or Visual Studio wants access to memory that is not present in guest physical memory (e.g., because it hase been paged), we can't get that memory directly.

What do about this problem? Well, in some cases, we simply try to avoid the problem. If paging of kernel-mode stacks is disabled, we are guaranteed it will be in guest physical memory. Similarly, we ask you to configure Visual Studio to use the Microsoft symbol server because when Visual Studio has access to symbols it is much more accurate in the memory it chooses to read (i.e., it reads memory that is more likely to be in guest physical memory).

Otherwise, we try to get the desired data in some other way. One technique is to read code memory out of the file from which it was mapped. This is why we need access to the same DLLs on the host and the guest. Finally, as a last resort, we take a snapshot, go live (i.e., diverge from the replay process), ask the guest tools to read the memory, and restore the snapshot. This technique is effective but slow, so it is critical that all the preparatory steps are taken to avoid these costs!

Note that the content of this post is derived from a document entitled "Guerrilla Replay Debugging Manual for Workstation" (published on the VMware Communities web site by ecl100).

Monday, November 16, 2009

Workstation 7.0 Released!

Workstation 7.0 has been released, and it includes improvements to replay debugging. You still get all the benefits of replay debugging, plus we've made some improvements and added some new features.

  • General stability improvements.
  • Reverse execution performance improvements (reducing number of replay operations required to achieve reverse execution).
  • Can start debugging at snapshot embedded in recording (in addition to starting at beginning of recording).
  • Remote replay debugging (i.e., Visual Studio and VMware Workstation are running on different hosts) in addition to local debugging.
  • Can disambiguate between multiple processes with the same name.
  • Enhancement to read non-present (i.e., swapped out) memory.
  • Improved exe/dll version checking between host and guest.
  • Linux/gdb support (see this technote).
Please give it a try!

Resources

Monday, December 1, 2008

Replay Debugging Talk at Google Now Online

Last month the replay debugging team stopped by Google to talk about replay debugging. It was great fun. The talk is now online at YouTube. Enjoy.

Monday, August 18, 2008

Getting Started with Replay Debugging

The article will get you started using the replay debugging features of VMware Workstation 6.5. What is replay debugging? Replay debugging allows C/C++ developers to use Visual Studio to debug recordings of programs running inside virtual machines. This is really useful for difficult-to-reproduce bugs. Check out this article for elaboration. You might also find the Workstation manual (see the side bar) a useful reference.

Note: If you find errors/ambiguities in this article, please leave comments here (use the link at the end of this article). If you encounter any problems with VMware Workstation, please post your concerns or comments on the VMware replay debugging forum. Thanks!

Requirements

Please ensure that your hardware and software are supported.
  • Processor: P4, Core 2, Penryn (stepping B0+), Barcelona (stepping B3+)
  • Host OS: 32/64-bit Windows XP SP2, Windows Server 2003, Vista (Recommended: 32-bit Windows XP SP2)
  • Guest OS: 32-bit Windows XP, Windows Server 2003, Vista (Recommended: Windows XP SP2)
  • Debugger: Visual Studio 2005 SP1 or Visual Studio 2008 (Recommended: Visual Studio 2005)
  • VMware Workstation: Recent version of Workstation 6.5 (release candidate 2 or newer)

Installation

Please ensure that the necessary software is installed.
  • Visual Studio: Ensure that you have installed Visual Studio before installing the latest version of VMware Workstation. This is necessary to ensure that the Workstation installation process installs the Visual Studio plugin.
  • VMware Workstation 6.5: Ensure that you install Workstation after Visual Studio. Install Workstation 6.5 (release candidate 1 is available here for beta program members (free signup)). When it is released, Workstation 6.5 will be a free upgrade for all Workstation 6.0 license holders.
  • Guest operating system: Ensure that a supported version of Windows (above) is installed in the virtual machine you intend to use for debugging. This virtual machine can only have one virtual processor. You find that performance is better if the virtual machine has a relatively small amount of memory (say 256MB) and if background snapshots are disabled (Edit > Preferences > Priority tab).
  • Guest tools: Ensure that the latest version of guest tools are installed in the Windows virtual machine. This is essential. Replay debugging will not work without the latest tools.

Power On Your Virtual Machine

Ensure that the virtual machine is working, and don't forget to install the latest guest tools! This is a good time to create a snapshot (let's call it "InitialState"). We'll use this as the starting point for the recordings we create.

Create a Visual Studio Project

Start Visual Studio. You should see a VMware menu. If you do not, this indicates that VMware Workstation was not properly installed (did you ensure that Workstation was installed after Visual Studio was installed?).

Create a Visual Studio project as follows.

  • File > New > Projects...


  • Project type: Visual C++ (Win32)
  • Visual Studio installed templates: Win32 Project
  • Name: HiMom
  • In Win32 Application Wizard, set application type to Console application
It is essential that the program we create be able to run in our virtual machine, so we must not use any DLLs that are not present in the VM. You'll probably need to change the project properties as follows.
  • Project > HiMom Properties
    • Configuration Properties > C/C++ > Code Generation: change runtime library to "Multi-threaded Debug (/MTd)"

Now edit your program in HiMom.cpp (make your mother proud!), save it, compile it (Build > Build Solution), and run it (Debug > Start Debugging). Okay, that's nice, but it doesn't look like replay debugging. It isn't. Be patient! We need to configure the VMware Options first.

Configure VMware Options in Visual Studio

You need to tell Visual Studio what virtual machine you would like to use for debugging.
  • Visual Studio: VMware > Options...
    • Configuration Properties > Replay Debugging in VM > General: Make sure you are changing the replay debugging properties (the remote debugging properties will have no effect for us). Set the "Virtual Machine" field to the full path of the .vmx file of the virtual machine you would like to use to debug (note that you can browse the filesystem to find this file). The "Recording to Replay" field specifies the recording you would like to use for debugging, but as you haven't created a recording yet, you can ignore this.

    • Configuration Properties > Replay Debugging in VM > Advanced: Set the Base Snapshot for Recording field to the name of the snapshot you created, above ("InitialState"). Just before creating a recording, this snapshot will be restored. If no snapshot is specified, the virtual machine has to be powered on (and we don't want to wait for that!).

Create a Recording

Before we can debug a recording, we need to create a recording. Makes sense. We can do this via the VMware menu (VMware > Create Recording for Replay). This will start up Workstation, ask for your guest login credentials, restore the "InitialState" snapshot, share the program to debug (HiMom.exe) with the virtual machine, start a recording, run the program, and stop the recording. Note that this step automatically populates the "Recording to Replay" field in the General options with the name of the recording you just created (it'll have a boring name like "Recording 1").

Troubleshooting: If your program never starts, this may be because the program can't run in the VM (e.g., because it's missing a needed DLL). The best way to diagnose these problems is to copy the program to debug into the VM (drag and drop works well) and run it manually. This usually provides sufficient diagnostic information to figure out what the problem is.

Start Debugging

Now that you've created a recording in which the program is running, let's debug it. Set a breakpoint on the first line of main and select VMware > Start Replay Debugging. The recording will start replaying and you'll hit the breakpoint. You should be able to examine local variables, single step, set other breakpoints, and do (almost) all the usual debugging activities.

The important difference with regular debugging is that here you are debugging a recording of the program, so you cannot change the behavior of the program as you debug it. This means you can't change the value of variables/registers nor can you execute code that was not executed in the recording. In addition, you will notice that input to the program (network, keyboard, etc.) need not be provided during replay debugging, because the input provided while the recording was created is reused during replay.

Reverse Execution

Please try out the reverse execution feature (VMware > Reverse Continue). This feature simulates reverse execution by replaying from an earlier point in time. This is particularly useful for tracking down memory corruption problems. When you encountered a corrupted data value, place a data breakpoint on this data value and execute backward to find out where it was corrupted! In a subsequent article I'll discuss methods for improving the performance of reverse execution.

An Alternate Method for Creating a Recording

Recordings can be created via Visual Studio (see above), but they can also be created using the Workstation UI. You simply copy the program you would like to debug to the VM (or place it in a shared file system), start recording, start the program, and stop recording. That's it. Naturally, you'll have to configure Visual Studio (via the VMware > Options... menu) to use the recording you have created.

Labels:

VMware Workstation 6.5: Reverse and Replay Debugging is Here!

I'm proud to announce that VMware Workstation 6.5 includes new experimental features that provide replay debugging for C/C++ developers using Microsoft Visual Studio. Replay debugging allows developers to debug recordings of programs running in virtual machines, and it is valuable for finding, diagnosing, and fixing bugs that are not easily reproduced, a particularly challenging class of bugs. Once the manifestation of a bug has been recorded, it can be replayed (and debugged) over and over again, and it is guaranteed to have instruction-by-instruction identical behavior each time. In addition, Workstation includes a feature that simulates reverse execution of the program, making it easier to pin point the origin of a bug.

Question: How can replay debugging help me?

Answer: Replay debugging helps find, diagnose, and fix bugs that are not easily reproduced. Consider the following examples:

Non-deterministic bugs. Some bugs only rear their ugly heads when a particular interleaving of concurrent entities (e.g., threads, the operating system, network activity, and user input) is present. Given that the programmer cannot normally control (or even reason about) this interleaving, these bugs can be very difficult to reproduce. But if such a bug is recorded, we can examine it in the debugger as often as we like. Once a bug is recorded, it is no longer non-deterministic.

Pseudo non-deterministic bugs. Some bugs are deterministic, but it's not always clear exactly what is necessary to reproduce the bug. Bug reports indicating that a program crashed "...after clicking on a bunch of different UI elements," are common but usually useless. If the bug is recorded, it is unnecessary to describe the steps for reproduction, because they are an implicit part of the recording itself.

Bugs that can only be reproduced with a complex environment. If a bug appears in an application that communicates with dozens of other local and remote processes that need to be setup and configured, reproducing the bug may be possible but prohibitively expensive. If the the bug is recorded, the developer need not setup the other components of the system (e.g., other processes or remote clients). The effect of the other components is recorded, allowing the program of interest to be debugged in isolation.

Heisenbugs. A Heisenbug (a play on the Heisenberg uncertainly principle) is a bug that does not manifest itself when it is examined in the debugger. Such bugs are common because debugging is an invasive process that dramatically changes the timing characteristics of the program. If a bug is recorded, it can be debugged without impacting the behavior of the program at all.

Memory corruption bugs. Memory corruption bugs are difficult to diagnose because almost any pointer dereference in the program can be responsible for the corruption. The replay debugging facility in VMware Workstation 6.5 includes a reverse execution feature that (among other things) greatly helps track down memory corruption bugs. When a corrupted data structure is identified, the developer simply sets a data breakpoint (i.e., a watchpoint) on the corrupted data and executes backward. This will take the developer to the last point in time that the corrupted data structure was modified.

Question: Do I have to learn a new debugging environment?

Answer: No, our replay debugging features are tightly integrated with the Visual Studio development environment. Aside from the replay-specific features, your debugging experience will be the same as a traditional debugging experience.

Question: How do I create recordings?

Answer: Our extension to Visual Studio provides a facility for creating recordings from within Visual Studio. Alternatively, recordings can be created from the VMware Workstation user interface. The later is useful when someone other than the developer (e.g., a tester) is creating recordings.

Question: How can I get it?

Answer: A release candidate for VMware Workstation 6.5 is currently available here. The release candidate is available to members of the beta program (free signup). The release version of the product will be available soon. Workstation 6.5 is a free upgrade for all Workstation 6.0 license holders.

Question: Where can I learn more?

Answer: We have created a number of resources to get you up and running with replay debugging.

Introductory video. This video introduces the replay debugging feature.

Tutorial screen cast. [Coming Soon!] This screencast shows the configuration and use of replay debugging.

Replay debugging forum. This form is a place for users and VMware engineers to get together to discuss replay debugging experiences, problems, and requests.

This blog. This blog will be updated with topics related to replay debugging. I expect to add articles on use cases, best practices, pitfalls, and so on.

VMware Workstation 6.5 Manual. The Workstation manual includes an appendix describing the Integrate Virtual Debugger for Visual Studio, which includes the replay debugging features. This is a useful reference but it does not include much more than the basic facts.

Labels: