ProcDump Post Mortem Switches: Order Matters

This week I’m working behind the “Great Firewall” in China and Twitter is blocked by the Chinese government, but my blog is not. I would have tweeted this but instead dear reader, you get the complete back story along with the debugging challenge on how I figured out the problem.

While working with a customer we wanted to set up a test machine to get a minidump when anything crashed. That sounds like a job for the awesome ProcDump from Sysinternals; especially now that it can act as the post mortem debugger. In my elevated PowerShell window I typed the following command for ProcDump to set itself up as the debugger.

I started running tests and had some crashes, but no dumps were created. The ProcDump window appeared, but it opened and closed so fast I couldn’t see what the problem was, which made it hard to diagnose. Before you read on, stop and think how you’d solve this problem because it’s an interesting exercise.

One way I came up with was using screen recording software like Camtasia. That way could could slow down the playback and see if there was anything in the flashing ProcDump window. If the recording software is on the machine, that might work.

As a debugger kind of guy, my second thought was if I could get a debugger on the process, half the battle is won. The challenge is that the process we need to attach to is not under our control for starting and is ending far faster than I can move the mouse to attach using Visual Studio.

The Windows team thought about this problem many years ago and it’s Image File Execution Options (IFEO) to the rescue. This special registry key lets you tell the operating system when a specific EXE starts, run it under the debugger, among many other options. While you can manipulate IFEO with the registry editor, it’s best if you use GFLAGS.EXE that comes with WinDBG (the Debugging Tools for Windows). For more on WinDBG see my WintellectNOW video: http://www.wintellectnow.com/Videos/Watch/windows-debugger-windbg-for-native-debugging.

I need to note that GFLAGS.EXE is a very dangerous program. You can make your computer unbootable with it so be careful.

Below is my GFLAGS.EXE screen shot showing that I set up WinDBG as the debugger. Note that WinDBG is permanently in my path so I don’t need to add the full path when specifying the debugger value.

After triggering a crash I got into WinDBG and let ProcDump run to completion because that does not close the program’s console window.  That made it extremely easy to see what the issue happens to be.

That’s a pretty strange filename ProcDump is trying to write and it’s certainly in the C:JUNK directory I wanted. That got me to look closely at the ProcDump command line I used. It looks like ProcDump requires the –i <Directory> to be the last command line argument when setting itself as the post mortem debugger. Sure enough when I reran my ProcDump command like the following, all the crashes were saved as minidumps.

I hope this helps someone out for figuring out why ProcDump wasn’t working and gave you a trick for getting a debugger on the process even if that process is not under your control.