8.7.1: What is MySQL Fabric?
8.7.2: Is it necessary to use a MySQL Fabric-specific Storage Engine?
8.7.3: What versions of MySQL are supported by MySQL Fabric?
8.7.4: What connectors support MySQL Fabric?
8.7.5: Are transactions ACID?
8.7.6: How many machines are needed in order to use MySQL Fabric?
8.7.7: Do I need to run an agent for each MySQL Server?
8.7.8: What interface is available to manage MySQL Fabric and its server farm?
8.7.9: How does MySQL Fabric Compare with MySQL Cluster?
8.7.10: How is MySQL Fabric licensed?
8.7.11: What if MySQL Fabric doesn't do what I need it to?
Questions and Answers
MySQL is a framework for managing groups of MySQL Servers and using those servers to provide services. It is designed to be extensible so that over time many different services can be added. In the current version the services provided are High Availability (built on top of MySQL Replication) and scale-out (by sharding the data).
MySQL Fabric is implemented as a MySQL Fabric node/process (which performs management functions) and Fabric-aware connectors that are able to route queries and transactions directly to the most appropriate MySQL Server. The MySQL Fabric node stores state and routing information in its State Store (which is a MySQL database).
No. The MySQL Servers that are being managed by MySQL Fabric continue to use InnoDB (and in the future NDB/MySQL Cluster may also be supported).
Currently MySQL 5.6. New MySQL releases will be fully supported as they reach General Availability status.
Currently PHP, Python & Java. In addition the Hibernate and Doctrine Object-Relational Mappings frameworks are also supported. New connectors will be added over time.
Yes. Because each transaction is local to a single MySQL Server, all of the ACID behavior of the InnoDB storage engine is experienced.
For development, the MySQL Fabric node and all of the managed MySQL Servers can be hosted on a single machine.
For deployment, the minimal HA configuration would need 3 or more machines:
2 to host MySQL Servers
1 to host the MySQL Fabric process (that machine could also be running application code).
No. The MySQL Fabric node is the only additional process and does not need to be co-located with any of the MySQL Servers that are being managed.
A Command Line Interface (CLI) is provided as well as an XML/RPC API that can be used by connectors and/or applications to make management changes or retrieve the routing information - in this way, an application could use MySQL Fabric without a Fabric-aware connector.
MySQL Cluster is a mature, well proven solution for providing very high levels of availability and scaling out of both reads and writes. Some of the main extra capabilities that MySQL Cluster has over MySQL Fabric are:
Faster (automated) fail-over (resulting in higher availability)
Cross-shard joins and Foreign Keys
In-memory, real-time performance
MySQL Fabric on the other hand, allows the application to stick with the InnoDB storage engine which is better suited to many applications.
MySQL Fabric is available for use under the GPL v2 Open Source license or it can be commercially licensed as part of MySQL Enterprise Edition or MySQL Cluster Carrier Grade Edition.
There are a number of options:
Raise feature requests or bug reports
Modify the code to customize the current services. MySQL Fabric is written in Python and is designed to be easy to extend.
Implement new modules that bind into the MySQL Fabric framework to implement new services.
8.7.1: How is High Availability achieved with MySQL Fabric?
8.7.2: How are MySQL Server failures detected?
8.7.3: What happens when the primary (master) MySQL Server fails?
8.7.4: Does my application need to do anything as part of the failover?
8.7.5: Is a recovered MySQL Server automatically put back into service?
8.7.6: Does MySQL Fabric work with semisynchronous replication?
8.7.7: Do I have to use MySQL Replication for implementing HA?
8.7.8: Is the MySQL Fabric node itself fault tolerant? What happens when the MySQL Fabric node is not available?
Questions and Answers
MySQL Fabric manages one or more HA-Groups where each HA-Group contains one or more MySQL Servers. For High Availability, a HA Group contains a Primary and one or more Secondary MySQL Servers. The Primary is currently a MySQL Replication master which replicates to each of the secondaries (MySQL Replication slaves) within the group.
By default, the Fabric-aware connectors route writes to the Primary and load balance reads across the available secondaries.
Should the Primary fail, MySQL Fabric will promote one of the Secondaries to be the new Primary (automatically promoting the MySQL Server to be the replication Master and updating the routing performed by the Fabric-aware connectors).
The MySQL Fabric node has a built-in monitoring function that checks on the status of the master. In addition, the Fabric-aware connectors report to MySQL Fabric when the Primary becomes unavailable to them. The administrator can configure how many problems need to be reported (and over what time period) before the failover is initiated.
The MySQL Fabric node will orchestrate the promotion of one of the Secondaries to be the new Primary. This involves two main actions:
Promoting the Secondary to be the replication master (and any other surviving Secondaries will become slaves to the new master)
Updating the routing information such that Fabric-aware connectors will no longer send any queries or transactions to the failed Primary and instead send all writes to the new Primary.
No. The failover is transparent to the application as the Fabric-aware connectors will automatically start routing transactions and queries based on the new server topology. The application does need to handle the failure of a number of transactions when the Primary has failed but before the new Primary is in place but this should be considered part of normal MySQL error handling.
No, the user must explicitly invoke MySQL Fabric to return a recovered MySQL Server to a HA Group. This is intentional so that the user can ensure that the server really is ready to take on an active role again.
In this version, MySQL Fabric sets up the HA Group to use asynchronous replication. If the user prefers to use semisynchronous replication then they can activate it manually after MySQL Fabric has created the replication relationships.
At present, HA Groups are formed using MySQL Replication; future releases may offer further options such as MySQL Cluster or DRBD.
There is currently only a single instance of the MySQL Fabric node. If that process should fail then it can be restarted on that or another machine and the state and routing information read from the existing state store (a MySQL database) or a replicated copy of the state store.
While the MySQL Fabric node is unavailable, Fabric-aware connectors continue to route queries and transactions to the correct MySQL Servers based on their cached copies of the routing data. However, should a Primary fail, automated failover will not happen until the MySQL Fabric node is returned to service and so it's important to recover the process as quickly as possible.
8.7.1: How is scaling achieved with MySQL Fabric?
8.7.2: Does scaling apply to both reads and writes?
8.7.3: What if I have table data that needs to be in every shard?
8.7.4: How many MySQL Servers can I scale to?
8.7.5: Can MySQL Fabric introduce contention or deadlock?
8.7.6: What happens when my data set or usage grows and a shard grows too big?
8.7.7: Is there extra latency when using MySQL Fabric?
8.7.8: Why does MySQL Fabric route using connector logic rather than via a proxy?
8.7.9: What is the difference between a shard key and a shard identifier?
8.7.10: Does my application need to change when a shard is moved to a different MySQL Server or split into multiple shards?
8.7.11: Is it possible to perform cross-shard unions or joins?
8.7.12: Is the routing of queries and transactions transparent to my application?
Questions and Answers
Horizontal scaling is achieved by partitioning (sharding) the data from a table across multiple MySQL Servers or HA Groups. In that way, each server or group will contain a subset of the rows from a specific table.
The user specifies what column from the table(s) should be used as the shard key as well as indicating whether to use a HASH or RANGE partitioning scheme for that key; if using RANGE based sharding then the user should also specify which ranges map to which shards. Currently the sharding key must be numeric.
When accessing the database, the application specifies the sharding key which the Fabric-aware connector will then map to a shard ID (using the mapping data it has retrieved and cached from MySQL Fabric) and route the query or transaction to the correct MySQL Server instance.
Within a HA group, the Fabric-aware connector is able to direct writes to the Primary and then spread the read queries across all available Secondaries (and optionally the Primary).
Yes. Both reads and writes scale linearly as more HA groups are added. Reads can also be scaled independently by adding more Secondary servers to a HA Group.
A special group can be created called the Global Group which holds the Global Tables. Any table whose data should appear in its entirety in all HA Groups should be handled as a Global Table. For a Global Table, all writes are sent to the Global Group and then replicated to all of the HA Groups. An example might be the department table from an employee database - the contents of the department table being small enough to be stored in every server and where that data could be referenced by any record from one of the sharded employee tables.
Similarly, any schema changes would be sent to the Global Group where they can be replicated to all of the HA Groups.
There is no limit—either to the number of HA Groups or the number of servers within a HA group.
No. A single transaction can only access data from a single shard (+ Global Table data) and all writes are sent to the Primary server within that shard's HA Group. In other words, all writes to a specific row will be handled by the same MySQL Server and so InnoDB's row-based locking will work as normal.
MySQL Fabric provides the ability to either:
Move a shard to a new HA group containing larger or more powerful servers
Split an existing shard into two shards where the new shard will be stored in a new HA Group. In the future, different levels of granularity may be supported for shard splitting.
No. Because the routing is handled within the connector there is no need for any extra "hops" to route the request via a proxy process.
One reason is to reduce complexity; rather than having a pool of proxy processes (a single proxy would represent a single point of failure) the logic is just part of each connector instance. The second reason is to avoid the latency involved in all operations being diverted via a proxy process (which is likely to be an a different machine).
The shard key is simply the value of a column from one or more tables. The shard key does not change if a row is migrated from one shard or server to another. The shard key is mapped to a shard id (using either a HASH or RANGE based mapping scheme); the shard id represents the shard itself.
As an example, if an existing shard were split in two then some of the rows would map to one shard's shard id and the rest to the other's; any given row's shard key would *not* change as part of that split.
Very importantly, shard keys are known to the application while shard ids are not and so any changes to the topology of the collection of servers is completely transparent to the application.
No. Because the application deals in shard keys and shard keys do not change during shard moves or splits.
Not at present; all queries are limited to the data within a single shard + the Global Table data. If data from multiple shards is required then it is currently the application's responsibility to collect and aggregate the data.
For HA, the application simply needs to specify whether the operations are read-only or involve writes (or consistent reads).
For sharding, the application must specify the sharding key (a column from one or more tables) *but* this is independent of the topology of the MySQL Servers and where the data is held and it is unaffected when data is moved from one server to another.
8.7.1: What do I do if I need immediately-consistent reads?
Questions and Answers
Because replication from the Primary (master) to the Secondaries (slaves) is not synchronous, you cannot guarantee that you will retrieve the most current data when reading from a secondary. To force a read to be sent to the Primary, the application may set the *mode* property for the connection to read/write rather than read.