Skip navigation links
22.5.1.5 Using a Stack Trace »
Section Navigation      [Toggle]

22.5.1.4. Debugging mysqld under gdb

On most systems you can also start mysqld from gdb to get more information if mysqld crashes.

With some older gdb versions on Linux you must use run --one-thread if you want to be able to debug mysqld threads. In this case, you can only have one thread active at a time. It is best to upgrade to gdb 5.1 because thread debugging works much better with this version!

NPTL threads (the new thread library on Linux) may cause problems while running mysqld under gdb. Some symptoms are:

  • mysqld hangs during startup (before it writes ready for connections).

  • mysqld crashes during a pthread_mutex_lock() or pthread_mutex_unlock() call.

In this case, you should set the following environment variable in the shell before starting gdb: 

LD_ASSUME_KERNEL=2.4.1
export LD_ASSUME_KERNEL

When running mysqld under gdb, you should disable the stack trace with --skip-stack-trace to be able to catch segfaults within gdb.

In MySQL 4.0.14 and above you should use the --gdb option to mysqld. This installs an interrupt handler for SIGINT (needed to stop mysqld with ^C to set breakpoints) and disable stack tracing and core file handling.

It is very hard to debug MySQL under gdb if you do a lot of new connections the whole time as gdb doesn't free the memory for old threads. You can avoid this problem by starting mysqld with thread_cache_size set to a value equal to max_connections + 1. In most cases just using --thread_cache_size=5' helps a lot!

If you want to get a core dump on Linux if mysqld dies with a SIGSEGV signal, you can start mysqld with the --core-file option. This core file can be used to make a backtrace that may help you find out why mysqld died: 

shell> gdb mysqld core
gdb>   backtrace full
gdb>   quit

See Section B.5.4.2, “What to Do If MySQL Keeps Crashing”.

If you are using gdb 4.17.x or above on Linux, you should install a .gdb file, with the following information, in your current directory: 

set print sevenbit off
handle SIGUSR1 nostop noprint
handle SIGUSR2 nostop noprint
handle SIGWAITING nostop noprint
handle SIGLWP nostop noprint
handle SIGPIPE nostop
handle SIGALRM nostop
handle SIGHUP nostop
handle SIGTERM nostop noprint

If you have problems debugging threads with gdb, you should download gdb 5.x and try this instead. The new gdb version has very improved thread handling!

Here is an example how to debug mysqld: 

shell> gdb /usr/local/libexec/mysqld
gdb> run
...
backtrace full # Do this when mysqld crashes

Include the above output in a bug report, which you can file using the instructions in Section 1.7, “How to Report Bugs or Problems”.

If mysqld hangs you can try to use some system tools like strace or /usr/proc/bin/pstack to examine where mysqld has hung. 

strace /tmp/log libexec/mysqld

If you are using the Perl DBI interface, you can turn on debugging information by using the trace method or by setting the DBI_TRACE environment variable.


User Comments

Posted by Martin Mokrejs on November 4 2002 4:30am[Delete] [Edit]

On Solaris you can use instead of strace system
utility called truss. For example, "truss
-t\!ioctl,\!lseek,\!stat,\!fstat,\!fstat64 -e
-vall -xall -a -f ./mysqld"

Posted by Martin Mokrejs on November 4 2002 4:47am[Delete] [Edit]

To debug under a newly started process, chdir() to
the source tree, compile mysql using -g3 CFLAGS
option at the best and run:

# gdb ./sql/mysqld

(gdb) set args --datadir=/data/mysql2
--basedir=/usr/local/mysql-BK20021104
--pid-file=/usr/local/mysql/var/kulan.pidd
--port=3307 --socket=/tmp/mysql2.sock

(gdb) break mysqld.cc:2039

(gdb) break open() <- stop on every open() call

(gdb) run <- run until we reach our breakpoint

(gdb) where <- where are we now in the sources?

(gdb) l <- list source code

(gdb) s <- step

(gdb) n <- next

(gdb) p varname <- print variable contents

(gdb) quit

Posted by Martin Mokrejs on November 20 2002 6:31am[Delete] [Edit]

To debug an already running mysqld process and see
what happens to it if you issue a specific SQL
command from the interactive mysql session, you
have to figure out first to which PID you have to
connect. Issue from you client commands line
"SELECT 3+3" and figure out the PID. (Sinisa says
that you can in principle connect to any mysqld
process and switch context to access another
thread, but NOT on Linux these days yet. Use `info
threads' command at the gdb prompt in such case)

chdir() to unpacked source tree. At the best,
compile next time mysql using CFLAGS="-g3".

# gdb
(gdb) attach 1234
(gdb) detach
(gdb) break sql_parse.cc:981
(gdb) b pthread_mutex_unlock
(gdb) info break
(gdb) attach 12345
(gdb) c
(gdb) s
(gdb) n
(gdb) where
(gdb) l
(gdb) detach
(gdb) quit

Posted by Martin Mokrejs on January 16 2003 8:19am[Delete] [Edit]

Where to set breakpoints as a general start place?
In client in real_connect(). On the server in handle_connections(), but in version above 4.0.8 it got renamed, so use handle_one_connection() for example.

Add your own comment.