How does a debugger work?

Fri 31 July 2015 by Saruta

In this post, the main goal is to understand how a debugger (like gdb) works. In that purpose, the exemples will be based on my personnal toy debugger Edb (Easy DeBugger). Because of few bugs in the x86_64 version, the article is based on the i386 (32 bits) version.

How to handle i386 and x86_64

I wanted my debugger to be able to debug 32 and 64 bits x86 programs. Thus, I needed to make a configure file which initializes some Makefile variables and also make some defines about the target architecture.

edb-configure

Ptrace

This project was mainly based on a syscall called ptrace. This one allows the tracer (edb) to follow a tracee (program to debug). Thus, the tracer can change the tracee's memory, change its registers, trace their syscalls or put some breakpoints.

Here is the basic architecture of a program using ptrace:

int main () {
  pid_t child;
  child = folk ();
  if (child == 0) {
    ptrace (PTRACE_TRACEME, 0,NULL,NULL);
    execv ("my_prog", NULL);
  } else {
    wait (NULL);
    printf ("I am the tracer");
    ptrace (PTRACE_CONT, child, NULL,NULL);
  }
  return 0;
}

Control Flow

Breakpoint

To put a breakpoint in your program, you have to put the opcode 0xcc where you need to stop and that's it. But the problem is, if you want to continue, you will be blocked by the breakpoint. So, you also need to keep a table, called the breakpoint table.

In this table, there will be three fields for each breakpoint:

  • The breakpoint's address
  • An ELF symbol (optional, can be NULL).
  • The content of the memory before the breakpoint opcode (0xcc) was placed.

Kill

To kill the process traced, it is very easy:

ptrace (PTRACE_KILL, pid, NULL, NULL);

Continue

It is more complicated because, if the program is on a breakpoint, then to continue, it needs to pass over the breakpoint. So, let's check if there is a breakpoint:

  • Extract current IP
  • Check if this one is in the breakpoint table, if it is, that's a breakpoint.

Now, there are two cases:

  • If it is a breakpoint, let's pass over:

    • decrement current IP
    • put the opcode saved in the breakpoint table.
    • do a single step (see below)
    • continue
  • Otherwise:

    • just continue

Step

This feature is already implemented in ptrace.

ptrace (PTRACE_SINGLESTEP, pid, NULL, NULL);

Execution Monitoring

GetRegs

This Ptrace call gives you all the registers in a register structure. This structure must be already declared because Ptrace needs a pointer.

struct user_regs_struct regs;
ptrace (PTRACE_GETREGS, pid, NULL, &regs);

Memdump

There are three commands:

  • 'x' for hexadecimal memory dump
  • 'd' for signed decimal
  • 'u' for unsigned decimal

The first argument is the number of bytes to dump and the second is the address which can be in decimal or hexadecimal.

Let's see how it works, p is the address and count is the number of bytes to dump:

for (int i = 0; i < count_; i++) 
  val = ptrace (PTRACE_PEEKTEXT, pid_, p, NULL);
  cout << val << endl;
  p++;

Backtrace

A backtrace is a monitoring of the functions called before the current function thanks to the callstack. So, to get a backtrace, you have to walk through the stack until ptrace returns -1. The current frame begins at the value pointed by EBP (Base pointer), which points to the old EBP.

int
backtrace (pid_t pid_)
  {
    unsigned long pc, fp, nextfp;
    int ret;
    struct user_regs_struct regs;

    ret = ptrace (PTRACE_GETREGS, pid_, NULL, &regs);

    pc = PROGRAM_COUNTER(regs);
    fp = FRAME_POINTER(regs);

    cout << pc << endl;

    while (fp) 
      {
        nextfp = ptrace (PTRACE_PEEKDATA, pid_, 
          NEXT_FRAME_POINTER_ADDR(fp),0);

        if (nextfp == (unsigned long) -1) break;
        if (!nextfp) break;

        pc = ptrace (PTRACE_PEEKDATA, pid_, 
          NEXT_PROGRAM_COUNTER_ADDR(fp),0);

        if (pc == (unsigned long) -1) break;

        fp = nextfp;
        cout << pc << endl;
      }
    return 0;
 }

Links:

  • http://t-a-w.blogspot.fr/2007/03/how-to-code-debuggers.html
  • https://www.lse.epita.fr/data/sujets/sujet-my_dbs.pdf

github


Comments