Friday, July 10, 2015

Starting with Strace

What is strace?

Strace is a “system call trace” program. It attaches to a process and tracks system calls and signals made to and from it (and possibly it’s children). There are limitations to strace, some of which are detailed below. However, strace can be a very valuable tool for determining the root cause of many issues. This post only covers very basic usage of strace. The example used here can be done using jailshell.

What is a process?

For the purposes of this tutorial, a process is a unique instance of program being run, identified on the system by a unique process identifier (pid). For more information see and

What is a system call?

A system call is how a program requests that the operating system kernel perform a task for it. There are quite a few “normal” tasks that the operating system kernel does for programs regularly, including extremely common file operations like reading and writing. More information regarding system calls can be found at:

What information can strace capture? Not capture?

Strace can tell you
  • what files and network connections were opened, closed, or attempted
  • what was read from and written to from said files and connections
Strace cannot tell you what is happening inside the process. It is not a debugger and is not aware of the variables used inside the process.

What do I have to know before I can use strace?

Strace is a power tool in your troubleshooting bag. But it’s not the most basic tool. General troubleshooting (checking log files, researching error messages, etc) should be done before attempting to solve a problem with strace. Before using strace, you’ll need to be able to reliably reproduce the issue so that you can catch it with strace. You’ll also need to determine what process to strace (covered later).

When am I not able to use strace?

In addition to there being times when strace will not be helpful, there are a few scenarios when you can not use strace. While most normal processes can be attached to by strace, there are some exceptions.
You cannot attach to a process in these scenarios:
  • when a debugger or other tracing program is already attached
  • when you do not own the process, unless you are running the strace as root
  • when the system has special protections preventing strace from being used to gather information

Resource considerations

Output files created by strace files do get very big very quickly, but more importantly are the numerous writes that strace output produces. This i/o hit is not mitigated (and is frequently made worse) by outputting to the terminal instead of a file.

Before you start to strace

Ensuring strace is installed

You can determine if strace is installed by typing:
strace -V
If strace is installed, you’ll see something like: strace — version 4.5.19. If it’s not installed, you’ll see something like bash: strace: command not found. Install strace with:
yum install strace
This command does require root access, so if you do not have root, you will need to request that your web hosting provider install strace for you.