In this tutorial, you learn how to debug a kext. You'll set up a two-machine debugging environment and use GDB to perform remote debugging. If you have not yet created a kext, complete “Creating a Generic Kernel Extension with Xcode” or “Creating a Device Driver With Xcode” before completing this tutorial. If you are unfamiliar with GDB, see Debugging with GDB.
Although this tutorial is written with a device driver as the example, the steps for debugging are similar for debugging any type of kext.
You need two machines for remote debugging: a target machine and a development machine. You load and run the kext on the target machine and debug the kext on the development machine. It is important to keep the two machines clear in your mind as you work through this tutorial, because you will be moving back and forth between them many times. It may help if you take a piece of paper, tear it in half, and write “Development” on one piece and “Target” on the other. Then place the pieces of paper next to the two machines.
These are the major steps you will follow:
“On Target Machine, Enable Kernel Debugging”
After you have completed the first three steps, you do not need to repeat them, even if you need to start over part way through the tutorial. The remaining steps prepare the kext for debugging, set up GDB, and attach the two machines to begin debugging your kext. If you need to start over part way through the tutorial, you should repeat all of the remaining steps.
“On Target Machine, Get the Target Machine’s IP Address”
“On Development Machine, Get the Load Address of the Kernel Extension”
“On Development Machine, Create and Load the Symbol File”
Prepare the two machines by doing the following:
Ensure that the machines are running the same version of Mac OS X.
Ensure that the machines are connected to the same network with their built-in Ethernet ports.
Ensure that you are logged in as an administrator on both machines, which is necessary for using the sudo
command.
Mount the Kernel Debug Kit disk image on the development machine. Download the Kernel Debug Kit from the Apple Developer website under the Mac OS X download category. Make sure the Kernel Debug Kit you download matches the version of Mac OS X installed on your target machine.
For more information on the Kernel Debug Kit, see the Read Me file included in the disk image.
If your target machine is running Mac OS X Server, disable the Mac OS X Server watchdog timer.
$ sudo killall -TERM watchdogtimerd
For more information, see the manual page for watchdogtimerd
.
To better simulate a real-world kext debugging scenario, you need your kext to produce a kernel panic. The easiest way to do this is to dereference a null pointer.
In Xcode, add the following code to your driver’s start
method (if you are debugging a generic kext, add it to the kext’s MyKext_start
function):
char *kernel_panic = NULL; |
char message = *kernel_panic; |
Rebuild your kext. In the Terminal application, create a copy of the kext as root:
$ sudo cp -R MyDriver.kext /tmp |
Copy your kext’s dSYM
file (in the Xcode project’s build folder with your kext) to the same location you copy your kext.
Transfer the copy of your kext from the development machine to the target machine.
If the transferred copy of your kext has an incorrect owner or group, correct it with the following command:
$ sudo chown -R root:wheel MyDriver.kext |
Before you can debug your kext, you must first enable kernel debugging. On the target machine, do the following:
Start the Terminal application.
Set the kernel debug flags.
To enable kernel debugging, you must set an NVRAM (nonvolatile random access memory) variable:
$ sudo nvram boot-args="debug=0x144 -v" |
Password: |
For more information on debugging flags, see “Building and Debugging Kernels” in Kernel Programming Guide.
Restart the computer for the debugging flags to take effect.
To connect to the target machine from the development machine, you need the target machine’s IP address. If you don’t already know it, you can find it in the Network pane of the System Preferences application.
Start GDB with the following command, indicating the target machine’s architecture and the location of your debug kernel:
$ gdb -arch i386 /Volumes/KernelDebugKit/mach_kernel |
Add kernel-specific macros from the Kernel Debug Kit to your GDB session.
(gdb) source /Volumes/KernelDebugKit/kgmacros |
Inform GDB that you are going to be debugging a kext remotely.
(gdb) target remote-kdp |
You are ready to load your kext and cause a kernel panic. Do so with the following command:
$ sudo kextutil MyDriver.kext |
The kernel panic should occur immediately. Interactivity ceases and debugging text appears on the screen, including the text Awaiting debugger connection
.
Now you can tell GDB to attach to the target machine. On the development machine, do the following:
Attach to the target machine. At the GDB prompt, use the kdp-reattach
macro with the target machine’s name or IP address:
(gdb) kdp-reattach target.apple.com |
The target machine prints:
Connected to remote debugger. |
You need your kext’s load address in order to generate a symbol file for it. Enter the following at the GDB prompt:
(gdb) showallkmods |
A list appears displaying information about every kext running on the target machine. Find your kext in the list and write down the value in the address column. Note that the values in the kmod and size columns look similar to the address value, so make sure you have the correct value.
You can create a symbol file for your kext on the development machine with the kextutil
command. Once again, make sure that the version of the Kernel Debug Kit you provide matches the version of Mac OS X on the target machine.
Open a second Terminal window.
Create the symbol file.
Adjust the path to the Kernel Debug Kit, the path to your kext, and the architecture of the kernel as appropriate. The path after the -s
option specifies the output directory where the symbol file is written. This should be the directory you copied your kext and dSYM
file to. The -n
option prevents the command from loading the kext into the kernel.
sudo kextutil -s /tmp -n -arch i386 -k /Volumes/KernelDebugKit/mach_kernel -e -r /Volumes/KernelDebugKit /tmp/MyDriver.kext |
The kextutil
tool prompts you for the load address of your kext. Provide the load address you obtained in the previous step.
When you finish, the symbol file is in the output directory you specified. The filename is the bundle identifier of the kext with the .sym
extension.
At the GDB prompt, specify the location of the symbol file.
Again, make sure the symbol file is in the same folder as the copy of your kext and dSYM
file.
(gdb) set kext-symbol-file-path /tmp |
At the GDB prompt, add your kext to the debug environment with the following macro:
(gdb) add-kext /tmp/MyDriver.kext |
GDB asks you if you want to add the kext’s symbol file. When you confirm, it loads the symbols.
Now you are ready to begin debugging! Request a backtrace from GDB to locate the source of the panic:
(gdb) bt |
A list of stack frames appears. Your kext’s stack frame that caused the panic should be easily identifiable as about the fifth frame from the top. When you are debugging your own kernel panics and don’t know the cause, you can now enter the offending stack frame and figure out what exactly caused the panic.
Note: Because driver debugging happens at such a low level, you can’t use some GDB features, including the following:
You can't call a function or method in your driver.
You can't debug interrupt routines.
Kernel debug sessions don't last indefinitely. Because you must halt the target machine's kernel to use GDB, internal inconsistencies may appear that cause the target kernel to panic or hang, forcing you to reboot the target machine.
When you've finished debugging, stop the debugger by quitting GDB.
(gdb) quit |
The debugging session ends. Because the target machine is still panicked, you need to reboot it. When you log back into the target machine, it displays the following message:
Click Ignore.
Congratulations! You've learned how to set up a two-machine debugging environment to debug a kext with GDB. To learn how to package your kext for installation by your customers, read “Packaging a Kernel Extension for Distribution and Installation.”
Last updated: 2010-03-19