WL#3704: Have timeouts for mgmapi

Affects: Server-5.1   —   Status: Complete

Add proper timeouts for mgmapi functions.

Many functions in the mgm api will not timeout correctly (or remotely quickly)
on unexpected disconnection from the management server.

Properly auditing and fixing existing functions will probably fix a bunch of
bugs in our mgm api clients as well as in customer applications.
1. have timeout in NdbMgmHandle
2. function to set timeout
3. modify each mgmapi function to return error on timeout, keeping existing API/ABI
4. add new versions of functions if needed with better return codes for timeout
5. review use of mgmapi in our programs
From comments in commits:

Start using the write_timeout we already have in NdbMgmHandle

Add error injection either for this connection or for whole server.

Currently nothing for injecting errors into *another* connection... but that's
perhaps getting tricky-dicky for this point in time. Perhaps needed for events
if we don't do anything fancy.

In ndb_mgm_call, add checks for expired timeout in (Input|Output)Stream.
In case of timeout, we set NdbMgmHandle->last_error and return NULL.

In api calls not using ndb_mgm_call (or using it in conjunction with
own IO), they'll need to check for timeouts manually. Macros are provided
to do this.

Add ndb_mgm_disconnect_quiet(h) to disconnect without checking errors
(so we don't clobber NdbMgmHandle->last_error). This helps us provide
the *consistent* semantic that on timeout we leave the NdbMgmHandle
*disconnected*. We check for this in testMgm.
 
Change CHECK_REPLY in mgmapi to also check for set error in handle->last_error
This will pick up the ETIMEDOUT errors and return them to client (through
returning correct failure code for API call and setting NdbMgmHandle error).
Applications written to MGMAPI before this patch will behave as before,
and even hopefully check get_last_error and report the error back to the
end user!

Adding the last CHECK_TIMEDOUT_RET and delete in ndb_mgm_call() we
slightly change behaviour of mgmapi. Previously, if disconnect
midway through a reply, where there were only optional parameters left,
we'd get a Properties object from ndb_mgm_call() containing NULLs for
the optional parameters, leading to interesting error messages. This
enables the returning of the *real* message and actually improves the API
without breaking compatibility.

ndb_mgm_start_signallog
ndb_mgm_stop_signallog
ndb_mgm_log_signals
ndb_mgm_set_trace
ndb_mgm_insert_error
ndb_mgm_set_int64_parameter [1]
ndb_mgm_set_string_parameter [1]
ndb_mgm_purge_stale_sessions [2]
 - return error code on error during ndb_mgm_call

TODO:
ndb_mgm_report_event [2]

[1] marked for removal, unused.
[2] return codes incorrect in CHECK_HANDLE/CONNECTED. undocumented.


Server side:
 in Services (per session) add macro for injecting timeout error
 (just waiting 10 seconds before continuing... it does work!)

 We inject these errors in a number of critical places - including
 the tricky api functions that don't just use ndb_mgm_call but do
 their own thing (get_config, get_status and friends)

ATRT:
 Expand testMgm to add timout tests for API. Fully automated.
 *THEORETICALLY* timing dependent - an ultra-slow network will
 cause problems and "fake" failures... I welcome other solutions.

 Tests aren't exhaustive, but cover the generics and the tricky bits.
 Also test some calling semantics (incl disconnected on error).

 It is encouraged to add *more* mgmapi tests, not less :)


Also add an ERROR_codes.txt file for mgmd


Default timout of 30secs for ConfigRetriever

Default timout of 5sec for use by Transporter (ports etc).

And Ndb_cluster_connection::set_timeout() api for setting timeout from
NDBAPI applications. Should be called before connect. e.g.

c.set_timeout(4200);
c.connect();