Skip navigation links

Additional languages
30.5.1.6 Using Server Logs to Find Causes of Errors in mysqld »
Section Navigation      [Toggle]

30.5.1.5. Using a Stack Trace

On some operating systems, the error log contains a stack trace if mysqld dies unexpectedly. You can use this to find out where (and maybe why) mysqld died. See Section 5.2.2, “The Error Log”. To get a stack trace, you must not compile mysqld with the -fomit-frame-pointer option to gcc. See Section 30.5.1.1, “Compiling MySQL for Debugging”.

If the error file contains something like the following: 

mysqld got signal 11;
The manual section 'Debugging a MySQL server' tells you how to
use a stack trace and/or the core file to produce a readable 
backtrace that may help in finding out why mysqld died
Attempting backtrace. You can use the following information 
to find out where mysqld died.  If you see no messages after 
this, something went terribly wrong...
stack range sanity check, ok, backtrace follows
0x40077552
0x81281a0
0x8128f47
0x8127be0
0x8127995
0x8104947
0x80ff28f
0x810131b
0x80ee4bc
0x80c3c91
0x80c6b43
0x80c1fd9
0x80c1686

you can find where mysqld died by doing the following:

  1. Copy the preceding numbers to a file, for example mysqld.stack.

  2. Make a symbol file for the mysqld server: 

    nm -n libexec/mysqld > /tmp/mysqld.sym

    If you have not linked mysqld statically, use the following command: 

    nm -D -n libexec/mysqld > /tmp/mysqld.sym

    If you want to decode C++ symbols, use the --demangle, if available, to nm. If your version of nm does not have this option, you will need to use the c++filt command after the stack dump has been produced to demangle the C++ names.

    Note that most MySQL binary distributions (except for the "debug" packages, where this information is included inside of the binaries themselves) ship with the above file, named mysqld.sym.gz. In this case, you can simply unpack it by doing: 

    gunzip < bin/mysqld.sym.gz > /tmp/mysqld.sym

  3. Execute the following line: 

    resolve_stack_dump -s /tmp/mysqld.sym -n mysqld.stack

    If you were not able to include demangled C++ names in your symbol file, use c++filt on the output: 

    resolve_stack_dump -s /tmp/mysqld.sym -n mysqld.stack|c++filt

    This prints out where mysqld died. If this doesn't help you find out why mysqld died, you should make a bug report and include the output from the above command with the bug report.

    Note however that in most cases it does not help us to just have a stack trace to find the reason for the problem. To be able to locate the bug or provide a workaround, we would in most cases need to know the query that killed mysqld and preferable a test case so that we can repeat the problem! See Section 1.7, “How to Report Bugs or Problems”.


User Comments

Posted by Martin Mokrejs on January 12 2004 1:21am[Delete] [Edit]

This is a nice way to resolve the stack trace:

echo "This scripts expects stack trace from your mysql error log file"
echo "placed in the current directory as ./stack"

if [ ! -r ./stack ]; then
echo "./stack file not present or readable"
exit 255
fi

if [ -r /usr/local/mysql/bin/mysqld.sym.gz ]; then
gzip -dc /usr/local/mysql/bin/mysqld.sym.gz > ./symbols
else
echo "nm -n /usr/local/mysql/bin/mysqld > ./symbols"
nm -n /usr/local/mysql/bin/mysqld > ./symbols
fi

echo "/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack"
/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack

# if you have c++ filter, use rather this output, it's even better
echo "if you don't have c++filter, the next command will fail"
echo "/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack | c++filter"
/usr/local/mysql/bin/resolve_stack_dump -s ./symbols -n ./stack | c++filter

Posted by Martin Mokrejs on January 12 2004 1:53am[Delete] [Edit]

In my previous comment in the shellscript should be c++filt instead of c++filter. c++filt is a part of gcc package.

Add your own comment.