WL#2989: CRASH_INSERTION and ERROR_INSERTED for ndb_mgmd

Affects: Server-7.1   —   Status: Complete

ndb kernel has the ability to add bits of code like:

CRASH_INSERTION((10021));

into code and have a test suite make ndbd crash at this point. You can also add

if(ERROR_INSERTED(10023))

to simulate an error in a specific part of the code.

This is used extensively in a bunch of the autotest tests to simulate various
failures and especially in the code designed to recover from them.

We currently don't have these in ndb_mgmd and testing is subsequently manual
work alongside such tests as "did the cluster start and nodes connect okay"

This WL is to add CRASH_INSERTION and ERROR_INSERTED into ndb_mgmd so that more
automated testing can be devised.

There also needs to be a method of inserting an error and getting notification
on when this point has been reached so that a test suite could (for example)
unplug the node from the network or crash the box. This could come in the form
of a mgmapi call for "wait for error".

The bulk of the time will be spent writing the automated tests and putting the
crash and error insertion into existing code.