Skip navigation links
18.4.1.5 Using Server Logs to Find Causes of Errors in mysqld »
Section Navigation      [Toggle]

18.4.1.4. Using a Stack Trace

On some operating systems, the error log contains a stack trace if mysqld dies unexpectedly. You can use this to find out where (and maybe why) mysqld died. See Section 5.3.1, “The Error Log”. To get a stack trace, you must not compile mysqld with the -fomit-frame-pointer option to gcc. See Section 18.4.1.1, “Compiling MySQL for Debugging”.

A stack trace in the error log looks something like this: 

mysqld got signal 11;
Attempting backtrace. You can use the following information
to find out where mysqld died. If you see no messages after
this, something went terribly wrong...

stack range sanity check, ok, backtrace follows
0x40077552
0x81281a0
0x8128f47
0x8127be0
0x8127995
0x8104947
0x80ff28f
0x810131b
0x80ee4bc
0x80c3c91
0x80c6b43
0x80c1fd9
0x80c1686

You can use the resolve_stack_dump utility to determine where mysqld died by using the following procedure:

  1. Copy the preceding numbers to a file, for example mysqld.stack: 

    0x9da402
0x6648e9
0x7f1a5af000f0
0x7f1a5a10f0f2
0x7412cb
0x688354
0x688494
0x67a170
0x67f0ad
0x67fdf8
0x6811b6
0x66e05e

  2. Make a symbol file for the mysqld server: 

    shell> nm -n libexec/mysqld > /tmp/mysqld.sym

    If mysqld is not linked statically, use the following command instead: 

    shell> nm -D -n libexec/mysqld > /tmp/mysqld.sym

    If you want to decode C++ symbols, use the --demangle, if available, to nm. If your version of nm does not have this option, you will need to use the c++filt command after the stack dump has been produced to demangle the C++ names.

    Note that most MySQL binary distributions (except for the "debug" packages, where this information is included inside of the binaries themselves) ship with the above file, named mysqld.sym.gz. In this case, you can simply unpack it like this: 

    shell> gunzip < bin/mysqld.sym.gz > /tmp/mysqld.sym

  3. Execute the following command: 

    shell> resolve_stack_dump -s /tmp/mysqld.sym -n mysqld.stack

    If you were not able to include demangled C++ names in your symbol file, process the resolve_stack_dump output using c++filt: 

    shell> resolve_stack_dump -s /tmp/mysqld.sym -n mysqld.stack | c++filt

    This prints out where mysqld died. If that does not help you find out why mysqld died, you should create a bug report and include the output from the preceding command with the bug report.

    However, in most cases it does not help us to have just a stack trace to find the reason for the problem. To be able to locate the bug or provide a workaround, in most cases we need to know the statement that killed mysqld and preferably a test case so that we can repeat the problem! See Section 1.8, “How to Report Bugs or Problems”.


User Comments

Posted by Martin Mokrejs on January 12 2004 1:21am[Delete] [Edit]

This is a nice way to resolve the stack trace:

echo "This scripts expects stack trace from your mysql error log file"
echo "placed in the current directory as ./stack"

if [ ! -r ./stack ]; then
echo "./stack file not present or readable"
exit 255
fi

if [ -r /usr/local/mysql/bin/mysqld.sym.gz ]; then
gzip -dc /usr/local/mysql/bin/mysqld.sym.gz > ./symbols
else
echo "nm -n /usr/local/mysql/bin/mysqld > ./symbols"
nm -n /usr/local/mysql/bin/mysqld > ./symbols
fi

echo "/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack"
/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack

# if you have c++ filter, use rather this output, it's even better
echo "if you don't have c++filter, the next command will fail"
echo "/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack | c++filter"
/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack | c++filter

Posted by Martin Mokrejs on January 12 2004 1:53am[Delete] [Edit]

In my previous comment in the shellscript should be c++filt instead of c++filter. c++filt is a part of gcc package.

Add your own comment.