Writing Debug Visualizers for GDB / QtCreator 2.8

Whether you call them Debug Helpers, Debug Dumpers, Debug Visualizers, or whatever else, sometimes you want to customize the way your classes are displayed by the debugger. One of the most common reasons for needing a custom debug visualizer is when you've written a class with a dynamically allocated array. Typically, the debugger will simply see this as a pointer to a single object and will either show you the value of the pointer (an address), or the value of the object being pointed to, but you won't get to see all the elements in the array.

We ran into this exact scenario when we started using QtCreater 2.8.1 for a recent project. The available documentation (http://qt-project.org/doc/qtcreator-2.8/creator-debugging-helpers.html) provides a lot of good information, but it seems to only be part of the puzzle, and you're left to dig under the couch to find the missing pieces.

I'm going to assume that you've read through the linked documentation above so you have a general idea of what is going on, but don't worry if you don't understand it all right away (I certainly didn't). In the step-by-step guide below, I'll highlight the missing pieces and explain what I was able to figure out.

Here's an example of the default visualization:

 ... and an example of what we'd like to see...
 

...collapsed:

...and expanded:

 

...

 

QtCreator has support for debug helpers written in both C++ and Python, with some caveats to each.

C++ based helpers:

• must be recompiled for each version of Qt, and the resulting library will be dynamically loaded into the debugged application, which can put additional stress on the application.
• can be used with QtCreator on all platforms.

Python based helpers:

• do not need to be recompile.
• can be used by any python-enabled gdb, even outside of QtCreator.

We decided to use the Python based helpers because we have some engineers that love using gdb directly while others are using QtCreator, and also because we wanted to avoid having an extra library loaded into our application while debugging.

It is useful to know that QtCreator installs it's Dumper class source code to: /<home dir>/qtcreator-2.8.1/share/qtcreator/dumper/* Here you will find the C++ header and implementation files, several python files including qttypes.py which does formatting of the Qt data types, and a set of tests. All of these files are very useful references.

Step 1: Create a python file to contain the debug visualizer.

For this example, put the file in your home directory, and call it debugVisualizer.py

Step 2: Tell gdb about the new python file.

Create a ~/.gdbinit file if you don't already have one, and add the following line, replacing <home dir> with the full path to your home directory (note that it will not work if you use '~/':
python execfile('/<home dir>/debugVisualizer.py')

Step 3: Identify the members of the class you want to visualize.

I'll demonstrate here using a dynamic array of templated objects, which has a pointer to the templated type and a size of the array.

template<typename T> class DynamicArray {
   public:
      DynamicArray()
      : m_pArray(NULL),
        m_size(0)
      {}
     
      T* m_pArray;
      unsigned int m_size;
};

Step 4: Decide on how you want it to be displayed.

There are several ways we could view this data, see the earlier screenshot to see what we're going to accomplish. To help ease the introduction, I'll start off by just exposing the array size and the array pointer, then we'll add all the array elements in the final step.

Step 5: Formatting the data.

Part 1: Specify the exact string 

(this part can actually be skipped, but it is worth explaining as it can be used to debug and test visualizations later on)

As the documentation explains, the debugger plugin needs a string that provides the formatting, but it doesn't give a very good description of each of the tags.

iname: This is an internal name that just needs to be unique, it is optional and if not specified will be automatically generated by the debugger plugin. It appears to be best to simply let this be automatically generated, if I specify an iname as part of a child, the child will not appear in the debugger.
name: The variable's name, or whatever you want to show up in the 'name' column. It is optional, but should be specified for clarity.
addr: The address of the variable. This is also optional, but very useful when debugging and wanting to look at memory locations.
value: This will be the content of the value column. The debugger knows how to display basic data types, so they can be used when generating a more complex value for our DynamicArray class.
type: This can be a customized string to display in the type column. Although the documentation does not specify this as being optional, I've found that the correct type tends to be displayed automatically.
numchild: This identifies the number of children, primarily for the purposes of whether or not the node should be expandable (hence the documentation says that "zero/nonzero is sufficient"). It's probably best to make sure this number is accurate.
childnumchild: The default number of grandchildren. I'm assuming this is used to improve memory allocation, but it's marked as optional and I haven't confirmed its importance yet.
children: This node specifies the children, all of which can have any of the tags above. Getting this part of the string to be formatted correctly is one of the most difficult parts of writing a custom debug visualizer. Also, this node can be added (and should ONLY be added) if the parent item is expanded in view. This will get more discussion later on.

The following python code will display a node with two children - one for the size and one for the array pointer.

#!/usr/bin/python

def qdump__DynamicArray(d, value):
    array = value["m_pArray"]
    size = value["m_size"]
    innerType = d.templateArgument(value.type, 0)
    p = gdb.Value(array.cast(innerType.pointer()))
    d.put('value="[%d] @0x%x",' % (size, p.dereference().address) +
          'addr="%x", ' % value.address +
          'numchild="2",' +
          'children=[' +
          '{name="m_size", value="%d", numchild="0",},' % size +
          '{name="m_pArray", value="@0x%x", numchild="0",},' % p.dereference().address +
          ']')

NOTE: In the documentation where it shows what the string should look like, the quotation marks are not correct. If you use that format (with double quotes on the outside and single quotes to surround the values) the variables will not appear in the debugger at all! You must use the quotations as I have shown them in the example above. Also the starting and ending bracket '{' and '}' will be automatically added to by the debug plugin, so they do not need to be included in the string.

The first two lines define 'array' and 'size' variables equal to the array and size members of our DynamicArray class. Remember that python stores classes as a hash, so member variables are obtained using the dictionary syntax.

Using the put(..) member of the Dumper class allows us to append a string directly to the dumper. In this case we're not specifying an 'iname', 'name', or 'addr', as the debug plugin already knows that information. We are providing a value to display, the number of children, and a child node for each of the member variables.

A subtle detail that you may have noticed that we are printing the value of the 'm_pArray' variable (an address) as part of the value for the overall object. The reason for this is so that when the node is collapsed in the tree, the important information is easily visible. In order to get the correct pointer (and to use it later on for referencing the actual values), we obtain the inner type of the template argument and cast the array member to that type. We are however using the actual address of the object ('value.address') when we set the 'addr' field so that the debugger can easily navigate to the appropriate memory location.

If you make a mistake here, you may experience errors in several ways:

• The variable may not appear at all in the visualizer - likely due to incorrect quotation marks.
• The debug plugin may revert back to using the default visualization - likely due to python syntax errors, in which case running gdb from the command line might display useful information.
• The visualizer may show a value of "<not accessible>" meaning that gdb could not access something that you are trying to do in the python code - most likely due to incorrectly trying to access or use the members of class being visualized.

Part 2: Put the Dumper class to work.

Now that this initial data is displaying the way I'd like, I'm going to use the Dumper class to recreate the same contents

#!/usr/bin/python

def qdump__DynamicArray(d, value):
    array = value["m_pArray"]
    size = value["m_size"]
    innerType = d.templateArgument(value.type, 0)
    p = gdb.Value(array.cast(innerType.pointer()))
    d.putValue('[%d] @0x%x' % (size, p.dereference().address))
    d.putAddress(value.address)
    d.putNumChild(2)
    with Children(d):
        d.putSubItem("m_size", size)
        with SubItem(d, "m_pArray"):
            d.putValue("@0x%x" % p.dereference().address)

You can see how the hardcoded text is now being replaced by functions of the dumper object. You might notice that I am using d.putSubItem(..) for the size, but "with SubItem(..)" for the array. The dumper object does not have an overloaded putSubItem(..) which formats the value as an address, and instead will display the object that is being pointed to (ie, the first element in the array). Since that is not what we want just yet, we have to use "with SubItem(..)" and do our own formatting of the address.

As it turns out, we will have to use "with SubItem(..)" anyway since we will want to add more children for each element in the array.

Part 3: Dynamically add array elements as children

#!/usr/bin/python

def qdump__DynamicArray(d, value):
    array = value["m_pArray"]
    size = value["m_size"]
    maxDisplayItems = 100
    innerType = d.templateArgument(value.type, 0)
    p = gdb.Value(array.cast(innerType.pointer()))
    d.putValue('[%d] @0x%x' % (size, p.dereference().address))
    d.putAddress(value.address)
    d.putNumChild(2)
    with Children(d):
        d.putSubItem("m_size", size)
        with SubItem(d, "m_pArray"):
            d.putItemCount(size)
            d.putNumChild(size)
            if d.isExpanded():
                numDisplayItems = min(maxDisplayItems, size)
                with Children(d, numChild=size,
                       maxNumChild=numDisplayItems,
                       childType=innerType,
                       addrBase=p,
                       addrStep=p.dereference().__sizeof__):
                    for i in range(0,numDisplayItems):
                        d.putSubItem(i, p.dereference())
                        p += 1

In this version, we've replaced the 'm_pArray' SubItem with a more complex sub item which has 'size' number of children. The "putItemCount(size)" line tells the debug plugin to set the value to "<size items>" (ie "<1000 items>"). Since we are displaying an array, showing the number of items in the array is appropriate (even though that same number will be visible as the value of 'm_size').

You will also notice the use of "d.isExpanded()" - this tells the debugger to only evaluate the inner text once the parent item is expanded. Since our array may hold many items, adding all the children (whether visible or not) by default could really slow down the debugger when a breakpoint is hit and all the variables in scope need to be evaluated. By moving the addition of children into the isExpanded condition, we delay that processing until the parent node is expanded. Note that once the node has been expanded and all the children are added, they will not be re-evaluated if the user continually collapses and expands the node. It will however be reevaluated if the user steps or runs the debugger.

Another trick to save processing time (and perhaps screen realestate) is to limit the number of elements shown in the list. This is achieved using the "maxDisplayItems = 100" variable and comparing that value to the size of the array to determine the number of items to display ("numDisplayItems = min(maxDisplayItems, size)"). I've added "maxDisplayItems" near the top of the function to make it easy to find and customize that value if desired. The "numDisplayItems" is then passed into "with Children(...)" to help customize how the debugger handles the long list of children. See below for more details on these parameters and the role they play:

numChild: The number of children that will be added.
maxNumChild: This is the number of children that will be added and displayed. NOTE: This does not actually control how many children will get displayed by the debugger, this number is compared to numChild, and if the maxNumChild is less than numChild, a final node will be automatically added to the children with the name "<incomplete>". This will happen even if you add all numChild SubItems to the parent.
childType: The default data type to use for each child.
addrBase: The base address to use for the array object. In our case, it is the value of the array pointer.
addrStep: The number of bytes needed to step from one element to the next. Since we are adding children for an array of elements, each element is a fixed size and so the debugger can automatically calculate the address for each child using this value.

Finally, the children are added one at a time using a for-loop. We are specifying the name of the SubItem as 'i' which will be automatically formatted with brackets around it like so: "[0]". Also, because putSubItem(...) is being used to add the item, the object being dereferenced will automatically be formatted based on its debug helper. In this manner, if our array contains elements of another custom class which has a custom debug helper, it will automatically be used by the debug plugin and formatted correctly!

All done! Have fun debugging!