{"id":124,"date":"2011-02-22T17:29:15","date_gmt":"2011-02-23T01:29:15","guid":{"rendered":"http:\/\/blog.mozilla.org\/sfink\/?p=124"},"modified":"2021-06-13T14:08:58","modified_gmt":"2021-06-13T21:08:58","slug":"fun-with-gdb","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/sfink\/2011\/02\/22\/fun-with-gdb\/","title":{"rendered":"Fun and Games With gdb"},"content":{"rendered":"<p>As with most developers, I have a love\/hate relationship with gdb, and have built up a clunky set of tips, tricks, and workarounds that I commonly use. I thought I&#8217;d share some of mine here, in a typically disorganized fashion:<\/p>\n<p><strong>Breakpoint Command Lists<\/strong><\/p>\n<p>This may be well known to everyone but me, but gdb lets you run arbitrary canned expressions every time a breakpoint is hit. This is incredibly useful when you need to know the last time something changed before a crash or some other detectable event. The usual problem is that if you set a breakpoint or watchpoint or whatever, you&#8217;ll hit it over and over again before the &#8220;interesting&#8221; one happens, and it&#8217;ll take way too long and you&#8217;ll give up in disgust. So instead, try something like this:<\/p>\n<pre>  (gdb) b somefilename.cpp:1234\r\n  Breakpoint 2 at ...\r\n  (gdb) command 2\r\n  &gt; bt\r\n  &gt; cont\r\n  &gt; end\r\n  (gdb) cont\r\n<\/pre>\n<p>Now, gdb will run normally, except whenever that breakpoint is hit it&#8217;ll dump out the full stack and then continue. When you crash or hit your other breakpoint or whatever, you can look backwards to the previous time you hit breakpoint 2 (hardware watchpoints are especially good with this) and see what the stack was then. If the breakpoint is hitting a lot, this may not be fast, but you can leave it running unattended all night if you have to.<\/p>\n<p>I&#8217;ve tried getting clever with this, with mixed results: once, I wanted to know when a particular value changed. I could set a hardware watchpoint, but it was constantly getting changed by a function f() that I didn&#8217;t care about. It was some other mysterious source of mutation that was tripping me up. So I set the hardware watchpoint, and then set breakpoints on the beginning and end of f(). I then created a command list for the breakpoint on the start of f() that disabled the watchpoint and continued, and a command list for the breakpoint on the end of f() to re-enable the watchpoint.<\/p>\n<p>Unfortunately, it didn&#8217;t work. I still don&#8217;t know why. Maybe there&#8217;s something weird about watchpoints. I didn&#8217;t have time to dig into it then. But you get the idea. (Hmm&#8230; if it was having trouble with enable\/disable, perhaps I should have set and cleared a convenience variable $active and made the watchpoint conditional on that&#8230;)<\/p>\n<p><strong>Logging From gdb<\/strong><\/p>\n<p>Occasionally, I have some big wad of data that needs to be interpreted by a different program. Let&#8217;s say it&#8217;s in a char* variable with its length stored in another variable. My usual approach is to print out the contents using the &#8216;x&#8217; or &#8216;p *foo @ 100&#8217; or whatever, then use emacs or Perl to unescape. But for large wads of data, this is unworkable. (For example, I&#8217;ve done this with entire images.)<\/p>\n<p>So write it out to a file instead:<\/p>\n<pre>  (gdb) p creat(\"\/tmp\/somefile.dat\", 0777)\r\n  $26 = 37\r\n  (gdb) p write(37, s, len)\r\n  $27 = 168\r\n  (gdb) p close(37)\r\n  $28 = 0\r\n<\/pre>\n<p>&#8230;and now you have a handy little file containing just your data and nothing else.<\/p>\n<p>In conjunction with the previous trick, you can append wads of binary data to a trace file every time you hit a breakpoint. Fun!<\/p>\n<p><strong>Watchpoints<\/strong><\/p>\n<p>This may be outdated. gdb has a nice watchpoint facility, but it has a tendency to take you too literally &#8212; when you ask it to watch a particular expression, it often sets a <em>software<\/em> watchpoint on that expression, meaning it constantly re-evaluates the expression over and over and slows execution down to a crawl. If you know that the various pointers involved aren&#8217;t going to change (or if they change, you want the current one anyway), it&#8217;s often handy to take the address of the expression you want to watch and then watch the dereference of that:<\/p>\n<pre>\u00a0 (gdb) p &amp;fun-&gt;atom\r\n  $27 = (JSAtom **) 0x7fffe3b6b778\r\n  (gdb) watch *$27\r\n  Hardware watchpoint 5: *$27<\/pre>\n<p>This doesn&#8217;t seem to be as necessary as it used to; it seems to do hardware watchpoints automatically in many more cases. But it still complains when the expression is no longer valid.<\/p>\n<p>I notice (while writing this) that there is now a -l option for doing something like this automatically. I&#8217;ve never tried it, but it sounds like it&#8217;d do what I want.<\/p>\n<p><strong>Automagical debugging<\/strong><\/p>\n<p>I nearly always run gdb within emacs so it automatically loads and highlights the current line of code for me (and gives history, etc.) But say I want to run<\/p>\n<pre>  myprog -s -t lamppost -- monkey walrus -teeth-<\/pre>\n<p>I have to:<\/p>\n<ul>\n<li>run emacs<\/li>\n<li>M-x gdb<\/li>\n<li>type &#8220;myprog&#8221;&lt;enter&gt;<\/li>\n<li>type &#8220;run -s -t lamppost &#8212; monkey walrus -teeth-&#8220;<\/li>\n<\/ul>\n<p>&#8230;and that means retyping it all or cutting &amp; pasting, even though I probably just ran that command. For running Firefox under the debugger, it&#8217;s even worse, because on Linux &#8216;firefox&#8217; is actually a shell script that sets up the environment and then invokes &#8216;firefox-bin&#8217;. There, you have to use the &#8216;-g&#8217; option on the shell script to do the right thing. I also want to start up the Perl debugger under emacs for Perl scripts, and (not that you care) I used to work on a program at a previous company that had an embedded scripting engine, and I sometimes wanted to run gdb on the C++ generated binary, and sometimes the script debugger on the embedded scripts.<\/p>\n<p>Phew. So I wrapped it all up in a &#8216;debug&#8217; script that guesses what you want and twiddles the options to emacs, gdb, Perl, and\/or the Firefox wrapper script to do the right thing. Usage:<\/p>\n<pre>  debug firefox -no-remote -P test about:blank\r\n<\/pre>\n<p>That will bring up an emacs window running gdb on firefox-bin with the environment all set up automagically. (It makes some extreme assumptions, such as guessing that if you give it a shell script then it&#8217;s the firefox wrapper shell script so it can use the -g and -d options.) It is surprisingly robust; if you use &#8216;debug&#8217; in place of &#8216;gdb&#8217; in commands that accept a debugger, it&#8217;ll often just work. For example, here&#8217;s a way to run a single directory&#8217;s worth of mochitests:<\/p>\n<pre>  cd obj\/_tests\/testing\/mochitest\r\n  python runtests.py --test-path=js\/jsd\/test\r\n<\/pre>\n<p>But it turns out runtests.py has a &#8211;debugger argument, so you can do:<\/p>\n<pre>\r\n<pre>  python runtests.py --test-path=js\/jsd\/test --debugger=gdb\r\n<\/pre>\n<p>and it&#8217;ll run the browser under gdb and give you a command-line prompt. So, to make it nicer:<\/p>\n<pre>\r\n<pre>  python runtests.py --test-path=js\/jsd\/test --debugger=debug\r\n<\/pre>\n<p>Voil\u00e0! You get emacs running gdb running firefox running your mochitests.<\/p>\n<p>I just tossed a snapshot of the script onto <a href=\"http:\/\/people.mozilla.org\/~sfink\/uploads\/debug\">http:\/\/people.mozilla.org\/~sfink\/uploads\/debug<\/a>. It has some of my environment hardcoded in it, but it should probably work for you unmodified anyway. (The main thing I have to keep tweaking is the &#8211;annotate option to gdb. Either gdb or emacs changed somewhat recently.)<\/p>\n<p><strong>Conditionals with strings<\/strong><\/p>\n<p>Say you want to set a breakpoint on a certain line when script-&gt;filename is your script, perhaps &#8220;http:\/\/lifewithducks.com\/cheesetasting\/reformat.js&#8221;. You could do<\/p>\n<pre>  cond 7 strcmp(script-&gt;filename, \"http:\/\/lifewithducks.com\/cheesetasting\/reformat.js\") == 0<\/pre>\n<p>but that&#8217;s (1) a lot of typing, and (2) slow because that breakpoint may get hit a lot and gdb has to do some ptrace-y context dance to invoke expressions every time. For the typing problem, strstr() is your friend:<\/p>\n<pre>  cond 7 strstr(script-&gt;filename, \"reformat\")<\/pre>\n<p>Specifically, strstr() returns the location of a substring within a larger string, or NULL if it doesn&#8217;t find it. So you just need to call it with a unique snippet that you&#8217;re looking for, and it&#8217;ll Do The Right Thing.<\/p>\n<p>For the &#8220;slow&#8221; problem, er&#8230; well, that example is going to be really messy. Let&#8217;s simplify and pretend you&#8217;re interested in breaking when strcmp(s, &#8220;spatula&#8221;) == 0. Here&#8217;s my evil shortcut:<\/p>\n<pre>  cond 7 *(int*)s == *(int*)\"spatula\"<\/pre>\n<p>gdb seems to be able to handle that expression much, much faster. Ok, that sort of assumes that your ints are 4 bytes. But again assuming 4-byte ints, it&#8217;s equivalent to<\/p>\n<pre>  cond 7 *(int*)s == *(int*)\"spat\"<\/pre>\n<p>because it&#8217;s treating the literal &#8220;spatula&#8221; as an address of a character array, and *(int*)&#8221;spatula&#8221; is thus giving the 1st 4 bytes interpreted as an integer. Here, that&#8217;s 1952542835 aka 0x74617073 ({ &#8216;t&#8217;, &#8216;a&#8217;, &#8216;p&#8217;, &#8216;s&#8217; }), because I&#8217;m little-endian. If you want more characters and you&#8217;re on 64-bit, cast to (long*) instead. Or use multiple pieces:<\/p>\n<pre>  cond 7 ((int*)s)[0] == ((int*)\"spatulamurders\")[0] &amp;&amp; ((int*)s)[1] == ((int*)\"spatulamurders\")[1] &amp;&amp; ...<\/pre>\n<p>Obviously, this is really not helping the typing problem. I didn&#8217;t say I&#8217;d solve both at once, ok?<\/p>\n<p>Back to the original example, you can use the latter trick to pick out the unique piece of the string you&#8217;re interested in. Let&#8217;s say that &#8220;reformat.js&#8221; is what we really care about, we&#8217;re on 64-bit, and we&#8217;ll ignore the string-length problem:<\/p>\n<pre>  cond 7 ((long*)script-&gt;filename)[5] == *(long*)\"eformat.\"<\/pre>\n<p>I don&#8217;t use this often, but it&#8217;s been useful when I&#8217;ve needed it.<\/p>\n<p><strong>Mozilla Customizations<\/strong><\/p>\n<p>The Mozilla source uses a bunch of datatypes that gdb doesn&#8217;t do very well with, from 16-bit characters in strings to big nasty structures that are mostly not interesting but clutter up the output. I use a modified version of <a href=\"https:\/\/developer.mozilla.org\/en\/Debugging_Mozilla_with_gdb#This_is_hard._Give_me_a_.gdbinit_that_works.\">this .gdbinit<\/a>. I also highly, highly recommend <a href=\"http:\/\/itcouldbesomuchbetter.wordpress.com\/2010\/12\/20\/debugging-spidermonkey-with-archer-2\/\">Jim Blandy&#8217;s archer-mozilla gdb extensions<\/a> written in Python. I&#8217;d actually like to stop using that .gdbinit, since it&#8217;s written directly in gdb&#8217;s &#8220;scripting&#8221; language, and is painfully slow for printing strings. It should be rewritten with the Python stuff.<\/p>\n<p>When I was working on compartment-related problems, I took a stab at modifying jimb&#8217;s stuff to automatically display the compartment for strings and to pretty-print compartments with their principals. It was pretty handy, though I doubt you&#8217;d want it to be active all the time. I should look into controlling it with an option or something, I guess. If you&#8217;re interested, you can find my changes at <a href=\"https:\/\/bitbucket.org\/sfink\/archer-mozilla\">bitbucket<\/a> (but note that <a href=\"http:\/\/hg.mozilla.org\/users\/jblandy_mozilla.com\/archer-mozilla\">jimb&#8217;s repo<\/a> is the real repo; I just pushed to bitbucket so I could point to it from this blog entry.)<\/p>\n<p>You may need to recompile gdb from the archer gdb sources for this stuff to work; I&#8217;m not sure. See jimb&#8217;s blog post for details.<\/p>\n<p><strong>How About You?<\/strong><\/p>\n<p>Those are all the ones I can think of for now. I&#8217;d be interested in hearing about other people&#8217;s tricks. It seems like everyone uses gdb a bit differently, so there&#8217;s a lot of opportunity for cross-pollination.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As with most developers, I have a love\/hate relationship with gdb, and have built up a clunky set of tips, tricks, and workarounds that I commonly use. I thought I&#8217;d share some of mine here, in a typically disorganized fashion: Breakpoint Command Lists This may be well known to everyone but me, but gdb lets [&hellip;]<\/p>\n","protected":false},"author":206,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[4141,666,137,610],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/posts\/124"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/users\/206"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/comments?post=124"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/posts\/124\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/media?parent=124"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/categories?post=124"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/sfink\/wp-json\/wp\/v2\/tags?post=124"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}