Tracing Your OpenGL Application with Vogl Just Got Easier!

VoglEditor now has support for launching and tracing your application, directly from the UI. Previously, (and still available) you'd have to launch your application from the command line, which could add confusion on where to supply vogl or application command line arguments.

> cd vogl_build/bin  
> VOGL_CMD_LINE="--vogl_tracefile vogltrace.glxspheres64.bin" 
  LD_PRELOAD=$(readlink -f libvogltrace64.so) ./glxspheres64 

Also, it is not clear what options vogl has to control its tracing behavior; what would you run to find out those options?!? (Answer: there actually is no executable to run which will provide that help information, but the information is at the bottom of this post)

Generate Trace

The new addition to VoglEditor is available via a "Generate Trace" button on the main toolbar, and provides a clean interface for launching your application as well as setting some common command line options.

Application to trace: Path to the application that you'd like to trace.

Application arguments: Command line arguments to your application.

Output trace file: Path to the resulting trace file.

vogl options:

• Use SteamLauncher
•  When launching a Steam game, this is the preferred approach to enable tracing, although you'll need to sync the vogl_chroot repo to get the script. If the script is not available, the option will be disabled.
• You should already have Steam running prior to using this option, otherwise the client may appear to hang after the application has closed.
• The dialog asking to load the new trace file may pop-up before the application has been launched; this is due to the way that the script works. Please wait until AFTER you exit your application to load the trace file.
• Application arguments are not currently supported by the script.
• The "Application to trace" should be the game ID, or one of the following pre-configured names:
• 214910 - AirConflicts
• 400       - Portal1
• 218060 - BitTripRunner
• 570       - Dota2
• 35720   - Trine2
• 440       - TF2
• 41070   - Sam3
• 1500     - Darwinia
• 550       - L4D2
• 1500     - Darwinia2
• 570       - Dota2Beta
• 221810 - TheCave
• 220200 - KerbalSpaceProgram
• 44200   - GalconFusion
• 201040 - GalconLegends
• 25000   - Overgrowth
• 211820 - Starbound
• 64-bit
• Enable if your application is 64-bit
• Force debug context
• Force the OpenGL Debug Context so that additional driver performance information can be output
• Gather call stacks
• If your application has been compiled with symbols, this will tell the tracer to gather a call stack at every OpenGL API call
• Disable glProgramBinary
• Applications which use binary programs will be able to replay on the same driver and hardware as they were traced, but may not replay correctly after updating the graphics driver, or replaying on a different platform. To solve this, the tracer can disable the GL_ARB_get_program_binary extension and will force all calls to glProgramBinary() to fail, which should cause your application to take a fallback path. These calls should replay correctly and the replayed stream will reflect the fallback path of your application.

The trace will complete when your application exits, and VoglEditor will ask you if you'd like to load the resulting trace file, allowing you to quickly dive in and identify any rendering artifacts.

Command Line Vogl Tracing Options

Although the UI only supports 3 of the vogl tracing options, a lot more are available at the command line and we plan to integrate UI support for these functionalities in the future. My apologees for not adding detailed help for these options, but I believe the names themselves should be sufficient, and the code is available if you wish to understand them more thoroughly ;-)

• "vogl_dump_gl_full"
• "vogl_dump_gl_calls"
• "vogl_dump_gl_buffers"
• "vogl_dump_gl_shaders"
• "vogl_sleep_at_startup <duration>"
• "vogl_pause"
• "vogl_long_pause"
• "vogl_quiet"
• "vogl_debug"
• "vogl_verbose"
• "vogl_flush_files_after_each_call"
• "vogl_flush_files_after_each_swap"
• "vogl_disable_signal_interception"
• "vogl_logfile <filename>"
• "vogl_logfile_append"
• "vogl_tracefile <output trace filename>"
• "vogl_tracepath <path for generated files>"
• "vogl_dump_png_screenshots"
• "vogl_dump_jpeg_screenshots"
• "vogl_jpeg_quality"
• "vogl_screenshot_prefix <prefix for files>"
• "vogl_hash_backbuffer"
• "vogl_dump_backbuffer_hashes <filename>"
• "vogl_sum_hashing"
• "vogl_null_mode"
• "vogl_force_debug_context"
• "vogl_disable_client_side_array_tracing"
• "vogl_disable_gl_program_binary"
• "vogl_func_tracing"
• "vogl_backtrace_all_calls"
• "vogl_backtrace_no_calls"
• "vogl_exit_after_x_frames <frame count>"

Shared Contexts, Uniforms, ARB Programs, and Buffers get added to VoglEditor

After an extended holiday, a flurry of work has been completed on vogleditor, adding improved support for shared contexts, program uniforms, ARB programs, and visualizing buffer contents. In the meantime, John McDonald has been working on the Windows port, and Rich Geldreich has been adding the final bits of the GL3.3 feature set and making progress on GL4!

Shared Contexts

It turns out that a significant number of games use shared contexts so that resources can be used by multiple contexts and on different threads. Very quickly it became clear that simply supporting the 'current context' was not going to be sufficient.

A drop-down has been added above the state tabs which lists all the available contexts. As expected, the current context is the default selection, and the set of contexts which are shared by each context are indicated.

In the case above, we can see that both the current context 0x84bd954 and context 0x8a9cbd4 share context 0x8533d24. Most OpenGL objects which are created can be accessible by all three contexts; Vogl will represent this by storing these shareable objects in the shared context (0x8533d24), and this can be seen via the objects listed in the state tree (below, left). Other objects (primarily Framebuffer and Vertex Array Objects) are not shareable, and therefore are only listed the corresponding context's state tree (below, right). 

Note that although the shared objects are only listed in the state tree of the owning context, they will appear in the object explorers for all objects that can access those objects.

 

Program Uniforms

... are now visible below the shader source code. The location, name, value, and type can be seen. Currently these values are NOT editable, but it could be an easy addition if someone wants to add it in soon.
 

 

ARB Programs

A new explorer has been added to display ARB Program Objects. These programs only contain source for a single shader type, so there is no drop-down control to select a shader type. This is in contrast to a Program Object which has several shaders attached to it. Instead of uniforms, ARB Programs rely on environment and local parameters, which are always 4 component floating point vectors. The parameters which are used by the program are visible in the bottom half of the explorer. The program source is editable and can be saved and will be used when replaying the trace. Environment and local parameters are not yet editable.

Buffer Objects

Buffer objects can be used to hold arbitrary data - vertex indices, vertex attributes, or even texture data. A single buffer can hold a variety of data types, so the explorer provides a set of common types that you can use to interpret the buffer contents - 8, 16, 32, 64-bit hex, float, double, byte, unsigned byte, short, unsigned short, int, and unsigned int are among the options available.
 

VoglEditor Adds Trimming Support and Persistent Settings

After receiving a few bug reports about users not being able to load large trace files, it became very clear that VoglEditor needed support for trimming large trace files.

VoglReplay Trim Support

The functionality was already available to users via command line options to voglreplay, but getting there after making a trace file is not an obvious second step.

VoglEditor Trim Support

The command line above can now be executed interactively from within VoglEditor when a trace file is loaded. VoglEditor detects whether the trace contains "too many" frames, and will display the following prompt. Selecting 'No' will continue to load the selected trace; 'Yes' will bring up the Trim Trace dialog.

The dialog will ensure that valid frames are entered for 'trim_frame' and 'trim_len' - validating that the user selects a frame that actually exists in the loaded trace file, and that "too many" frames are not included in the trim file (as this would cause the same dialog to appear when loading the trimmed trace). A new trim filename and path can be directly typed into the dialog, or click the '...' button to open a typical FileSaveDialog.

Clicking 'OK' will confirm the file does not exist and prompt you to overwrite if it does (or cancel to return to the dialog and select a new file name). The trace file will be trimmed in another process, saved to disk, and then the user is prompted to open the newly trimmed trace.

As expected, 'Yes' will load the new trimmed file into VoglEditor; 'No' will cause the originally selected trace file to be loaded.

This same trimming functionality is also now made available on-demand using the new 'Trim Trace' toolbar button.

NOTE: Currently, the 'Trim Trace' behavior is slightly different than the 'Play Trace', in that its input is the original trace file, whereas 'Play Trace' operates on the trace data that is in memory in the editor. This means that any edits made in VoglEditor will be reflected by 'Play Trace', but NOT reflected in 'Trim Trace'.

So... what constitutes as "too many" frames? You decide! (continue below)

VoglEditor Settings File

When implementing the automatic detection of "large traces" to prompt for trimming, it became clear that different users will have different definitions of "large" - in house we often look at multi-gigabyte trace files with hundreds of frames, and other systems may only be able to support a few hundred frames, or perhaps for debugging a particular issue only a handful of frames will be needed. In order to configure the definition of "large traces", a settings file was added.

To keep it easily readable, editable, and consistent with other vogl features, the settings file is stored in JSON. To stick with Linux standards, it is stored in one of the following paths:

$XDG_CONFIG_HOME/vogleditor_settings.json

or

$HOME/.config/vogleditor/vogleditor_settings.json

In addition to the number of frames at which to consider a trace "large", the initial implementation of the settings file also supports storing the window position and size.

Here are the contents of the default settings file, which will be automatically created when you first open vogleditor, and automatically resaved when the editor is closed.

{
  "metadata" : {
      "vogleditor_settings_file_format_version" : "0x1"
   },
   "settings" : {
      "trim_large_trace_prompt_size" : 200,
      "window_position_left" : 0,
      "window_position_top" : 0,
      "window_size_width" : 1024,
      "window_size_height" : 768
   }
}

Manually editing this file should be done while VoglEditor is closed, as any manual changes will be overwritten when the editor is closed.

VoglEditor Feature List

The past few months have been spent plugging away with Valve on a user interface for a new OpenGL debugging tool called VOGL, which we've recently open sourced on github - https://github.com/ValveSoftware/vogl

We are developing entirely in Linux, but are also assisting several members of the community who are working on ports to other platforms.

Since I've primarily been working on the user interface, called "VoglEditor", here's an initial feature set as of the time VOGL went public. New features are being added almost daily, so keep an eye out for updates, and feel free to get involved in the project!

Current VoglEditor features:

• Loading multi-frame trace files
• Obtaining a GL state snapshot at any API call that is not within a glBegin/glEnd block
• Save / Load debug sessions
• Saves a JSON file which links together the base trace file and all collected state snapshots
• Saves all collected state snapshots to disk
  
• Traces can be replayed from within vogleditor
• CPU-based timeline 
• Shows API call execution and cost
• Most expensive call is shown in red and all other calls are scaled between Red -> Green based on relative execution time
• API call hierarchy
• Shows frames and API calls hierarchically
• Clicking on an API call will launch the replayer and collect a state snapshot after that call executes
• Icons indicate which API calls have a snapshot
• Supports searching entrypoints and parameters for a supplied string, and navigating to prev / next search match
• Supports jumping to prev / next draw call
• Supports jumping to prev / next snapshot   
• State snapshot panel 
• Viewing all GL state within a snapshot
• Automatic diff'ing of state between two snapshots
• GL object explorers will default to displaying currently bound / active objects
• Visualization of all existing framebuffers, renderbuffers, and textures
• All visualizations can be scrolled and zoomed
• Ability to view RGBA (with customizable alpha blend color), RGB, individual color components, 1-component, and 1/component
• Support for viewing individual samples
• Support for Y-flipping the images
• Viewing of all created shader objects 
• Viewing of all created program objects and their linked / attached shaders
• Linked shaders can be edited and saved back into the snapshot so that the changes affect the trace replay