Creating a Perl Debugger

Although most people are probably already familiar with Perl's strengths for short scripts and rapid development, they might not know that they can do "serious" development in Perl. Perl has a lot of features and tools to aid the professional programmer, including a debugger. I will explore the Perl debugger and show how it does its magic. Once you know its basics, you can move on to writing your own debugger or using many of the other development tools found in the Comprehensive Perl Archive Network (CPAN).

Perl's debugger is built-in. This is not to say that Perl just comes with a debugger tool — the debugger is actually part of the Perl interpreter. During the compilation phase, Perl can remember special information about the source, such as line numbers, for use at run time. At run time, Perl can then load a special library containing the debugger which uses that information.

I wrote a simple script to demonstrate this and saved it as hello.pl.

	
	#!/usr/bin/perl

	print "Hello Dr. Dobbs readers!\n";

	__END__
	

To run hello.pl in the debugger, I simply invoke Perl with the -d switch and the name of the script.

	
	perl -d hello.pl
	

I see something like the following output (which may vary from version to version a little bit. This is from Perl 5.6.0). A debugging prompt waits for me to give debugging commands:

	
	 prompt$ perl -d

	hello.pl Loading DB routines from perl5db.pl version
	1.0401 Emacs support

	available. Enter h or `h h' for

	help.     main::(hello.pl:3): print "Hello Dr. Dobbs
	  readers!\n";DB<1> 
	

This debugger allows me to do the usual operations that I expect, such as stepping through code, setting breakpoints, and examining variables. Notice that the debugger knows the name of the script, the line number of the print statement, and the actual text for that line of code. If you want to learn how to use this debugger now that I have shown you how to invoke it you should read the perldebug man page.

However, the debugger is really just another library that Perl pulled in at run time when I used the -d switch. You can see how this debugger works by looking at the perl5db.pl source in the directory where your standard Perl libraries are stored. On my machine, it is /usr/local/lib/perl5/5.6.0/perl5db.pl since I am using the latest release of Perl. You might find it installed in a different directory depending on your local configuration.

Since the debugger is just a Perl library, if I do not like how it works or want it to work differently, I can change the source in perl5db.pl to do what I like. If you have looked at the source for perl5db.pl then you have probably already decided that you do not want any part of that self-labelled "unreadable code." Neither do I. That is not a problem — the Perl debugger is just a library and I can write my own and use it instead of perl5db.pl.

CPAN has a graphical debugger, called ptkdb, which uses the Tk library that does just that. It works on Windows and the usual flavors of Unix, and operates in much the same way that the default debugger does, but includes a different library to do the work. I can tell perl which debugger to use by specifying the alternate debugger with the -d switch. I separate the -d and the library name with a colon.

	
	perl -d:ptkdb hello.pl
	

The compilation goes through the script as before, but when the run time phase starts, the ptkdb debugger library is pulled in. If I am running this in a graphical environment (it will fail otherwise) the debugger creates a window that looks something like Figure 1.

You might have noticed that the actual module name is Devel::ptkdb. Perl expects the debugging modules to be under the Devel::* namespace and looks for them there even though you only specified ptkdb. When you create your own debugger, start off in the Devel::* namespace.

Now that I have shown you how to use a different debugger with perl you can start to create your own debuggers. To illustrate this process, I created a debugger that simply counts of the number of times that Perl executes each line in a script. This sort of debugger is typically called a "line profiler". If I wanted to do this during actual development, perhaps to find out where loops are doing more work than they should, I would use the Devel::SmallProf module. However, for my simple example I pretend that it does not exist.

The Perl interpreter, when run in debugging mode, looks in the special DB namespace for a subroutine named DB, which put all together is &DB::DB(). In my debugger, I need to create the &DB::DB() routine to do the line profiling work.

	
	package Devel::LineProfiler;

	package DB;

	sub DB
		{
		my ( $package, $file, $line ) = ( caller(0) )[0..2];

		next unless $file eq $0;

		$profile[$line]++;
		}

	END
		{
		print "\nLine usage summary for $0\n\n";

		open FILE, $0 or die "Could not open $0\n$!";

		while( <FILE> )
			{
			$count++;
			printf "%6d %s", $profile[$count], $_;
			}
		}

	1;
	

I first created the Devel::LineProfiler namespace with the package declaration, but then I immediately switched to the DB namespace so that my DB subroutine would be in the right package. That way I can easily add things to either package if I later decide to add features to this bare bones example.

I then defined a routine DB in the DB namespace. The Perl interpreter will call this routine before each statement that it will execute. In this simple example, I used the caller() function to look at the stack information for the &DB::DB() invocation to find out from where it was called - in this case, the point in the script right before the statement that is about to execute. In this case I keep the name of the package, file name, and line number of that statement, although I only care about the line number. If I wanted to debug more complex scripts that might span multiple files, I would keep track of line counts on a per file basis.

The END block will be executed after the rest of the script has finished and perl is about to stop running so it is a good place to output the profile of the line usage since everything interesting should have already happened. I open the script, whose name is stored in the Perl special variable $0, and read it in line by line. I prepend the count of the number of times that line was executed before the line itself and output that result. This way I can output the entire script line by line rather than just the lines that actually were executed.

To demonstrate this I created a sample program, loop.pl, in which I expect some lines of code will be executed more often than others so I see if my new debugger actually works.

	
	#!/usr/bin/perl

	print "Hello readers!\n";

	for( my $i =	
	

	  

	  0;  $i <  10; $i++ )
		{
		my $noop = 0;         #don't do anything
		$noop = 0; $noop = 0; #don't do anything, twice
		}

	__END__
	

To use my line profiler on loop.pl I do the same thing I did before. I invoke perl with the -d switch and the name of my debugger, LineProfiler.

	
	perl -d:LineProfiler loop.pl
	

The perl interpreter runs the program with Devel::LineProfiler and I get the output which shows how many times each line executed.

	
	Hello readers!

	Line usage summary for loop.pl

	     0 #!/usr/bin/perl
	     0 
	     1 print "Hello readers!\n";
	     0 
	    11 for( my $i =	
	 

	    

	      
	      
	        
	      
	       0;  $i <  10; $i++ )
	     0 	{
	    10 	my $noop = 0;         #don't do anything
	    20 	$noop = 0; $noop = 0; #don't do anything, twice
	     0 	}
	     0 
	     0 __END__
	

You should realize a couple of things about how Perl does this. Since Perl has a C-like syntax, multiple statements, or even the whole program, can show up on the same line. My line counter does not really count the number of times a line is seen but rather the number of times a single statement on that line is executed. If there are two statements on a line, as there is in

	
	$noop = 0; $noop = 0; #don't do anything, twice
	

the interpreter invokes the &DB::DB() subroutine for the first statement, which it is about to execute, and &DB::DB() increments the number of times that line has been seen. Then Perl moves onto the next statement which happens to be on the same line. The &DB::DB() routine gets control again and the same line number is incremented again. The debugger saw that line twice because two statements are on that line. Thus, that line has a count twice as high on the line that has only one statement even though the entire line was seen the same number of times.

You should also realize that lines of code may come from different files, and in Perl they may come from eval statements as well. Although I included a variable for $file in the example code, since I used a simple script that did not reference any external modules or files I left those details out. Suppose, though, that I wanted to count the number of statements executed per file. Instead of the line counter I can increment a file counter every time Perl executes a statement and then print out a summary of the participation of each file as I do in the Devel::FileCounter example.

	
	package Devel::FileCounter;

	package DB;

	sub DB
		{
		my ( $package, $file, $line ) = ( caller(0) )[0..2];

		$counter{$file}++;
		}

	END
		{
		print "File usage summary for $0\n\n";

		foreach my $file ( sort keys %counter )
			{
			printf "%6d %s\n", $counter{$file}, $file;
			}
		}

	1;
	

I created what appears to be a simple script, called get.pl, which fetches a web page.

	
	#!/usr/bin/perl
	
	use LWP::Simple qw(getstore);
	
	getstore( "http://www.perl.org", "www.perl.org" );
	

I invoke the debugger as before, but I specify the new FileCounter module.

	
	perl -d:FileCounter get.pl
	

The output shows the number of lines executed in each file. Even though the script has only two lines, the powerful LWP library did a lot of work behind the scenes.

	
	File usage summary for get.pl

	     4 (eval 1)[/usr/local/lib/perl5/site_perl/5.6.0/HTTP/Status.pm:161]
	     1 (eval 2)[/usr/local/lib/perl5/site_perl/5.6.0/HTTP/Message.pm:38]
	     1 (eval 4)[/usr/local/lib/perl5/site_perl/5.6.0/LWP/Protocol.pm:119]
	     2 (eval 5)[/usr/local/lib/perl5/5.6.0/i386-freebsd/Socket.pm:446]
	     2 (eval 6)[/usr/local/lib/perl5/5.6.0/i386-freebsd/Socket.pm:446]
	     1 (eval 7)[/usr/local/lib/perl5/5.6.0/i386-freebsd/Socket.pm:446]
	     1 (eval 8)[/usr/local/lib/perl5/5.6.0/i386-freebsd/Socket.pm:446]
	     1 (eval 9)[/usr/local/lib/perl5/5.6.0/i386-freebsd/Socket.pm:446]
	    34 /usr/local/lib/perl5/5.6.0/AutoLoader.pm
	  2967 /usr/local/lib/perl5/5.6.0/Exporter.pm
	   722 /usr/local/lib/perl5/5.6.0/Exporter/Heavy.pm
	   101 /usr/local/lib/perl5/5.6.0/IO/Socket/INET.pm
	    17 /usr/local/lib/perl5/5.6.0/IO/Socket/UNIX.pm
	    15 /usr/local/lib/perl5/5.6.0/SelectSaver.pm
	    29 /usr/local/lib/perl5/5.6.0/Symbol.pm
	    18 /usr/local/lib/perl5/5.6.0/Time/Local.pm
	    52 /usr/local/lib/perl5/5.6.0/i386-freebsd/Config.pm
	    66 /usr/local/lib/perl5/5.6.0/i386-freebsd/DynaLoader.pm
	    42 /usr/local/lib/perl5/5.6.0/i386-freebsd/Errno.pm
	     9 /usr/local/lib/perl5/5.6.0/i386-freebsd/IO.pm
	    57 /usr/local/lib/perl5/5.6.0/i386-freebsd/IO/Handle.pm
	   355 /usr/local/lib/perl5/5.6.0/i386-freebsd/IO/Select.pm
	    75 /usr/local/lib/perl5/5.6.0/i386-freebsd/IO/Socket.pm
	    60 /usr/local/lib/perl5/5.6.0/i386-freebsd/Socket.pm
	    54 /usr/local/lib/perl5/5.6.0/i386-freebsd/XSLoader.pm
	   146 /usr/local/lib/perl5/5.6.0/overload.pm
	   310 /usr/local/lib/perl5/5.6.0/strict.pm
	   333 /usr/local/lib/perl5/5.6.0/vars.pm
	   228 /usr/local/lib/perl5/5.6.0/warnings/register.pm
	   110 /usr/local/lib/perl5/site_perl/5.005/URI.pm
	   540 /usr/local/lib/perl5/site_perl/5.005/URI/Escape.pm
	    21 /usr/local/lib/perl5/site_perl/5.005/URI/URL.pm
	    56 /usr/local/lib/perl5/site_perl/5.005/URI/WithBase.pm
	    65 /usr/local/lib/perl5/site_perl/5.005/URI/_generic.pm
	    15 /usr/local/lib/perl5/site_perl/5.005/URI/_query.pm
	    41 /usr/local/lib/perl5/site_perl/5.005/URI/_server.pm
	   162 /usr/local/lib/perl5/site_perl/5.005/URI/http.pm
	    22 /usr/local/lib/perl5/site_perl/5.6.0/HTTP/Date.pm
	   606 /usr/local/lib/perl5/site_perl/5.6.0/HTTP/Headers.pm
	   139 /usr/local/lib/perl5/site_perl/5.6.0/HTTP/Message.pm
	    32 /usr/local/lib/perl5/site_perl/5.6.0/HTTP/Request.pm
	    22 /usr/local/lib/perl5/site_perl/5.6.0/HTTP/Response.pm
	     2 /usr/local/lib/perl5/site_perl/5.6.0/HTTP/Status.pm
	    28 /usr/local/lib/perl5/site_perl/5.6.0/LWP/Debug.pm
	    13 /usr/local/lib/perl5/site_perl/5.6.0/LWP/MemberMixin.pm
	   114 /usr/local/lib/perl5/site_perl/5.6.0/LWP/Protocol.pm
	   219 /usr/local/lib/perl5/site_perl/5.6.0/LWP/Protocol/http.pm
	    17 /usr/local/lib/perl5/site_perl/5.6.0/LWP/Simple.pm
	   149 /usr/local/lib/perl5/site_perl/5.6.0/LWP/UserAgent.pm
	  1095 /usr/local/lib/perl5/site_perl/5.6.0/i386-freebsd/HTML/Entities.pm
	    84 /usr/local/lib/perl5/site_perl/5.6.0/i386-freebsd/HTML/HeadParser.pm
	    35 /usr/local/lib/perl5/site_perl/5.6.0/i386-freebsd/HTML/Parser.pm
	     1 get.pl
	

You can do quite a bit more and use a lot of other features to make your debugger give you all sorts of useful information which you can find in the perldebguts man page, but that is just a Simple Matter of Programming. Although I have only shown you the beginning of Perl's debugging capabilities, I will cover some of the many other sorts of debuggers available in the Devel::* namespace in upcoming columns.

brian d foy has been a Perl user since 1994. He is founder of the first Perl users group, NY.pm, and Perl Mongers, the Perl advocacy organization. He has been teaching Perl through Stonehenge Consulting for the past three years, and has been a featured speaker at The Perl Conference, Perl University, YAPC, COMDEX, and Builder.com. Some of brian's other articles have appeared in The Perl Journal.