WL#7308: Refactor mysql_upgrade

Currently mysql_upgrade internally invokes a number of external tools, e.g. 
mysqlcheck and mysql client. 
This causes problems in handling options, logins, special server startup modes 
etc.
And makes the server upgrade procedure a multi-step process.

mysql_upgrade should use nothing but the C API to talk to the server. 

And we should also have a version of it that's bundled with the embedded server 
and provide for a single-step upgrade process.

User Documentation

• http://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-5.html
• http://dev.mysql.com/doc/refman/5.7/en/mysql-upgrade.html

1: Fix bugs BUG#65288 and BUG#71579.
2: Use cases should be the same.
3: Any existing usage should be working on new version in the same way.
4: Connecting to service parameters should look like in other tools.
5: Exit codes from mysql_upgrade should be following:

1. Initialization error.
2. Server already upgraded.
3. Bad server version.
4. Error during calling mysql_check functionality.
5. Error during execution of upgrading queries.

6: Verbosity option prints more progress messages and no notes or warnings from updating queries execution (in contrast to old version).
7: -t --tmpdir option is not needed anymore, and is removed.

This tool does queries to MySQL server and uses mysqlcheck tool.
First should be done using C API. Second should be done by calling mysqlcheck 
functionality linked into mysql_upgrade executable.

This task is divided into following work items:
1) To limit excessive copy&paste of command line options list, variables and 
functions making use of these parameters refactoring will be done.
Main idea is to divide these options into functional groups and encapsulate as 
much of parameters in separate objects.
Interface I_options_provider with functions to:
- acquire parameters list: vector<my_option> generate_options(),
- do action per option parsed: void option_parsed(int optid, char *argument),
- process data on parsing done: void options_parsed(),
- set option changed event listener: void 
set_option_changed_listener(I_option_changed_listener* listener) - by extending 
I_option_changed_listener.

It also contains implementation of function for aggregating several instances of 
implementations of Options_provider to one list:
static void aggregate_options(vector<my_option> *options, 
vector<Options_provider*> options_providers).

Abstract class Abstract_options_provider will be created to hide my_option 
structure internally.
It will implement generate_options() and option_parsed(int optid, char 
*argument).
It will expose new set of overloaded protected methods to create instances that 
implement Abstract_option:
- ...* create_new(_*)_option(...) - they will differ in input parameter types 
and will generate expected type of option object.
It will define pure virtual method void create_options(). It's task will be to 
create all options using new methods.
It will implement interface I_option_changed_listener with functions:
- void notify_option_optid_changed(I_option* source, uint32 old_optid) - this 
function is called by option or contained options provider to allow containers 
to check for optid collision.
- void notify_option_name_changed(I_option* source, string old_name) - this 
function is called by option or contained options provider to allow containers 
to check for name collision.

Class Composite_options_provider will be created to contain other instances of 
Abstract_options_provider and aggregate their options - it should register for 
option_changed event in each one.
One instance can be contained in at most one another. This will also be assumed 
by set_optid_changed_listener, as it can store only one listener in state (for 
now, in future when needed this restriction can be removed by making state store 
list of listeners).

Class Abstract_Option will be OO wrapper of my_option, which it will contain and 
expose only to Abstract_options_provider.
There are type-specific classes extending Abstract_option (base class -> class):
I_option: non-template interface with protected functions for use in 
Abstract_options_provider:
void call_callbacks(char* argument)
my_option get_my_option()
void set_option_changed_listener(I_option_changed_listener* listener)

I_option -> Abstract_option: abstract option class to implement all common code, 
that is: wrapping my_option, handling option IDs, callback functors, implement 
all I_option functions.

Abstract_option -> Bool_option: on/off switch type option.

Abstract_option -> Disabled_option: Represents disabled option. When used in 
command line it issues message to user.

Abstract_option -> Abstract_value_option: abstract option to cover options 
accepting value.

Abstract_option -> Simple_option: option that does not get any values. Main role 
is to execute callbacks when it is used.

Abstract_value_option -> Char_array_option: option for handling string values to 
char* variable. Should not be used in new code.

Abstract_value_option -> Abstract_string_option: abstract option for handling 
character string argument.

Abstract_string_option -> String_option: option with simple character string 
argument.

Abstract_string_option -> Password_option: option to handle password parameter. 
Should implement getting password from TTY, warning user when he specified it in 
command line and masking it in Linux command line.

Abstract_value_option -> Abstract_number_option: abstract option to cover 
numeric values. Allows to set number argument restrictions: minimum and maximum 
value boundaries.

Abstract_number_option -> Abstract_integer_number_option: abstract option to 
cover integer numeric values. Allows to set additional number argument 
restriction: value step.

Abstract_number_option -> Abstract_floating_number_option: abstract option to 
cover floating point numeric values.

Abstract_integer_number_option -> Number_option<>: option to cover specific type 
integer numeric values: int32, uint32, int64 and uint64.

Abstract_floating_number_option -> Number_option<>: option to cover specific 
type floating point numeric values: float and double.

Abstract_string_option -> Choice_option: option with argument value from defined 
set. Will cover old GET_ENUM type. This one will not be implemented, as it will 
have no usages for a while.

Abstract_value_option -> Flag_option: option which allows to provide comma-
separated string values contained in set. It covers old GET_FLAGSET type. This 
one will not be implemented, as it will have no usages for a while.

Abstract_value_option -> Set_option: option which allows to provide comma-
separated string values. It has no replacement in old GET_ types. This one will 
not be implemented, as it will have no usages for a while.

Number_option will have additional template parameter TValue stating type of 
value: int32, uint32, int64, uint64, float, double.

Constructors to option classes should have at least following parameters:
- Option name.
- Option description.

And for classes implementing Abstract_value_option also:
- Pointer to data where the value will be stored. Also we assume it contains 
default value at the constructor call.

We expect client tools not to call these constructors explicitly, instead they 
should use methods from a list of Abstract_options_provider::create_new_option() 
methods (all '...' stands for option name and description parameters):
Simple_option* create_new_option(...)
Password_option* create_new_password_option(std::string* value, ...)
String_option* create_new_option(std::string* value, ...)
Number_option<TNumber>* create_new_option(<TNumber>* value, ...) - for TNumber 
in {int32, uint32, int64, uint64, float, double}.
Bool_option* create_new_option(bool* value, ...)
Choice_option* create_new_option(std::string* value, const 
std::vector<std::string> possible_values, ...)
Set_option* create_new_option(std::vector<std::string>* value, ...)
Flag_option* create_new_option(uint64* value, const std::vector<std::string> 
possible_values, ...)

All values that receives argument value pointed to as parameter are expected to 
be members of calling class (that is extending Abstract_options_provider) to 
preserve encapsulation, and to have default value from the moment of creation 
method call. Also member value SHOULD NOT be changed afterwards as it will lead 
to undefined behaviour.

My_option IDs should be assigned basing on some global counter starting with 256 
to allow usage of short codes.
Abstract_options_provider should keep track of used options short IDs and names 
to detect collisions. For that it will collect all optids and names, and it will 
register itself as I_getopt_change_listener in all created I_option objects.

In implementation of Abstract_string_option and Abstract_floating_number_option 
there should be additional private member that will get value assigned from 
handle_options() function before conversion to destination type.

They also may expose chain-able methods to use more specific behaviour of 
option:
- Abstract_value_option::value_optional() - sets arg_type to OPT_ARG,
- Abstract_option::add_callback(I_callable<void, char*> callback) - adds new 
callback for this option for option_parsed() event to callback chain,
- Abstract_option::set_short_character(char code) - sets optid to given 
character to make possible usage of short option alternative,
- Abstract_number_option::set_minimum_value(TValue minimum) - sets minimum valid 
value,
- Abstract_number_option::set_maximum_value(TValue maximum) - sets maximum valid 
value,
- Abstract_integer_number_option::set_value_step(TValue step) - will round value 
to value divisible by step that is not further to zero than original value.

For this chain-able methods we need to introduce template parameter TType to 
abstract option classes so all these methods can use properly typed pointer to 
it as return value.
The my_option.app_type in wrappers should contain reference to wrapping object. 
This will enable us to make static callback,
that then can dispatch execution to proper object without using additional 
static state. Also we require all options in application to implement I_option.
I_callable is an interface to allow creating functors with one argument, as we 
don't have standard ones for now.
I_callable<type_of_return_value, type_of_parameter> exposes type_of_return_value 
operator() (type_of_parameter parameter).
I have one implementation for now, Instance_callback that allows usage of 
instance method to be used as functor.
I_callable and all implementations should be removed once we have 
boost::function<> or std::function<>.

We introduce interface I_connection_factory capable of creating new connections 
to use with MYSQL C API. It will consist of MYSQL* create_connection() function.

Common MySQL tools options groups implementing Abstract_options_provider will be 
created:
- Help_options (for handling -?),
- Debug_options,
- Mysql_connection_options (also will be able to provide MYSQL connection 
according to input parameters by implementing I_connection_factory).
- Mysql_connection_options::Ssl_options used by Mysql_connection_options to 
encapsulate SSL-related options.

Abstract class Abstract_program will be created to handle all common sequential 
work, that is:
- gathering and parsing options providers parameters,
- calling main tool routine (which is to be overridden by extending classes,
- free resources at the end and call at-exit debug routines.
It will extend Composite_options_provider to allow adding multiple options 
providers easily to program.

Also, it allows to get program name (got from command line): const char* 
get_name().
It has 4 pure virtual members:
- char* get_version() - Returns string describing current version of this 
program.
- string get_description() - Returns string describing shortly current program.
- void error(int error_code) - Handles general errors.
- void execute(vector<string> arguments) - Runs main program code. Supplies all 
command line arguments that weren't consumed by handle_options.
Also abstract class Abstract_connection_program extending Abstract_program will 
handle Mysql_connection_options and will implement I_connection_factory.

As there are a lot of objects in global namespace and some of tools would like 
to share functionality with others there will be a dedicated namespace for each 
tool in My::Tools namespace.
All common code for clients should be in Mysql::Tools::Base namespace, and all 
options-related (derived from I_option and I_options_provider) should be in 
Mysql::Tools::Base::Options namespace.
Alternatively names of objects could be prefixed, but that would cause more 
boilerplate and would not make any deeper division.
Also using namespaces will make it easier for community developers to integrate 
with tools and extend them (mysqldump seems to have such possibility in future). 
This is main reason to use Mysql namespace instead of global one for tools.

The main() function should then look like:

static ::Mysql::Tools::Upgrade::Program program;

int main(int argc, char **argv)
{
  program.run(argc, argv);
}

It should be global variable to make it being destructed on exit(), which wouldn't if it was on heap or stack. This is required to Debug_options work properly.

Here are two remaining work items in this task (continuation from first lines):

2) Use MYSQL C API to run queries in run_query routine.

3) Make mysqlcheck code available for use from mysql_upgrade and call required 
routines. main() in mysqlcheck need to be divided, so that part with actual work 
is separated from options parsing, connection creation etc.
Subsequent calls will be parametrized using explicit option variables assignment 
in call to function extracted from main().
This function will be in mysqlcheck_core library, which should be the kind of 
API for whole functionality.
No other objects should be accessed from mysqlcheck source within mysql_upgrade.