![]() |
MySQL 26.7.0
Source Code Documentation
|
Main scheduling class. More...
#include <scheduler.h>
Public Types | |
| using | Task_return_type = Task_result |
| using | Thread_pool_type = Thread_pool< Task_return_type > |
| using | Thread_pool_ptr = std::shared_ptr< Thread_pool_type > |
| using | Scheduled_task_type = Scheduled_task |
| using | Mutex_type = concurrency::Mutex |
| using | Task_queue_type = std::priority_queue< Scheduled_task > |
Public Member Functions | |
| virtual | ~Scheduler () |
| Destructor. More... | |
| void | deinit () |
| Ends the scheduler if stop was not yet requested. More... | |
| Scheduler (const Scheduler &src)=delete | |
| Copying constructor is deleted. More... | |
| Scheduler (Thread_pool_ptr shared_thread_pool, Scheduler_clock_ptr shared_clock, Dependency_tracker_ptr dependency_tracker, int instance_id=0, std::size_t allowed_task_count=std::numeric_limits< std::size_t >::max(), Scheduler_psi psi_params={}) | |
| Constructor. More... | |
| template<typename FType , typename... Args> | |
| bool | enqueue (Task_schedule_ptr schedule, FType &&task, Args &&...args) |
| Enqueues a task. More... | |
| template<typename FType , typename... Args> | |
| bool | enqueue_after (const Task_id &predecessor, Task_schedule_ptr schedule, FType &&task, Args &&...args) |
| Enqueues a task that should execute after a predecessor task finishes. More... | |
| bool | synchronize (bool force=true, bool print_checkpoint=false) |
| This function waits for currently fired tasks to finish (blocking) More... | |
| std::size_t | get_timeouts () const |
| Access "timeouts" observability statistics - the number of times scheduler timed out while waiting for tasks to schedule. More... | |
| bool | is_error () const |
| Internal function to check on whether error has been set. More... | |
| std::size_t | get_scheduled_tasks_count () const |
| Obtain the number of scheduled, ongoing tasks. More... | |
| bool | register_phase (Scheduler_clock_ptr phase_clock) |
| Register task phase run according to the given clock. More... | |
| void | stop_now () |
| Instruct the scheduler to stop now and withdraw all work. More... | |
| void | notify_scheduler () |
| Function used to notify scheduler thread by the other thread (enqueueing thread / worker poll thread) More... | |
| bool | ensure_space () |
| Ensure space for the next task (eliminates possible wait in the enqueue) More... | |
| void | request_unblock () |
| This function requests unblocking of the scheduler. More... | |
Static Public Attributes | |
| static constexpr Sync_method | sync_method {Sync_method::active} |
Protected Types | |
| using | Task_future = std::future< Task_result > |
| using | Futures_list = std::list< Task_future > |
Protected Member Functions | |
| bool | add_dependency (const Task_id &predecessor, const Task_id &successor) |
| add_dependency More... | |
| template<typename FType , typename... Args> | |
| bool | enqueue_internal (Task_schedule_ptr schedule, FType &&task, Args &&...args) |
| Enqueues a task identified by the task_id. More... | |
| void | end_execution () |
| end_execution More... | |
| void | wait_for_scheduler_thread_to_stop () |
| Waits for scheduler thread to finish its work. More... | |
| void | run_main_thread () |
| Scheduler run function. More... | |
| bool | are_phase_queues_empty () const |
| Checks phase queues and return true if all are empty. More... | |
| bool | is_task_phase_ready () const |
| Checks whether any phase of the task is ready for execution. More... | |
| template<typename Repeatable_task_type > | |
| Task_result | callback (Task_schedule_ptr schedule, Scheduler_clock::Time_point_t task_delay, bool task_error, std::shared_ptr< Repeatable_task_type > repeatable_task, unsigned int current_thread_id) |
| Function used to notify that task ended its execution. More... | |
| Task_result | callback_phase (Task_schedule_ptr schedule, bool task_error, unsigned int phase_id) |
| Function used to notify that task phase ended its execution. More... | |
| template<typename Repeatable_task_type > | |
| bool | enqueue_phase (Task_schedule_ptr schedule, Scheduler_clock::Time_point_t task_delay, std::shared_ptr< Repeatable_task_type > repeatable_task, unsigned int current_thread_id, unsigned int phase_id) |
| Function that enqueues next task phase. More... | |
| void | wait_until_done () |
| void | handle_error () |
| Worker error handling function. More... | |
| void | scheduler_clean_up () |
| Scheduler error handling function. More... | |
| bool | enqueue_helper (Scheduled_task &&task) |
| Enqueue helper. More... | |
| void | synchronize_active (bool print_checkpoint, std::size_t expected, bool force) |
| Synchronizes the scheduler by actively waiting for finished tasks count. More... | |
| void | synchronize_partial (bool print_checkpoint=false) |
| Waits for percent of tasks to finish. More... | |
| void | synchronize_active_partial (bool print_checkpoint) |
| Actively waits for percent of tasks to finish. More... | |
| Task_queue_type & | get_phase_queue (Scheduler_clock_ptr phase_clock) |
| Mutex_type & | get_phase_queue_lock (Scheduler_clock_ptr phase_clock) |
| bool | is_task_ready () const |
| Helper function to check if the task at the top of the task queue is ready. More... | |
| bool | is_stop_requested () const |
| Internal function to check if immediate stop was requested, externally or due to an error. More... | |
Private Types | |
| using | Thread_type = concurrency::Thread |
Private Attributes | |
| std::priority_queue< Scheduled_task > | m_tasks |
| This is a queue which holds task ready for scheduling. More... | |
| bool | m_scheduler_thread_active {true} |
| Variable indicating that scheduler thread finished its execution, protected with m_mutex_end. More... | |
| Mutex_type | m_mutex_notification |
| Mutex protecting access to scheduler notifications. More... | |
| Mutex_type | m_mutex_tasks |
| Mutex protecting access to m_tasks, data structure populated by the enqueuing thread. More... | |
| Mutex_type | m_mutex_phases |
| This mutex protect access to phase. More... | |
| std::atomic< bool > | m_notification {false} |
| True if scheduler has been notified. More... | |
| concurrency::Condition_variable | m_cv_scheduler |
| Cv used by the scheduler main thread to wait on, when no task is available or tasks in m_task queue are not read to execute. More... | |
| Mutex_type | m_mutex_end |
| Mutex used to finish execution of the Scheduler, protects m_scheduler_thread_active and it is used together with m_cv_end. More... | |
| concurrency::Condition_variable | m_cv_end |
| cv used by a thread requesting scheduler to finish its work More... | |
| std::unordered_map< Task_id, Scheduled_task > | m_tasks_waiting_for_deps |
| Internal map that keeps tasks waiting for dependencies defined in the Dependency Graph (distinct from time dependencies implemented in the Scheduler clock) More... | |
| container::Integrals_lockfree_queue< uint64_t > | m_notified_tasks |
| Lock-free queue of task IDs that are ready for execution. More... | |
| Thread_type | m_scheduler_thread |
| Scheduler main thread. More... | |
| std::atomic< bool > | m_scheduler_active {true} |
| Variable that checks whether scheduler should run (true) or end its execution (false) More... | |
| std::atomic< bool > | m_deinitialized {false} |
| Ensures deinit() is executed only once. More... | |
| std::atomic< bool > | m_stop_now {false} |
| Variable that instructs scheduler to withdraw all work and stop. More... | |
| std::atomic< std::size_t > | m_scheduled_tasks_cnt {0} |
| The number of scheduled tasks. More... | |
| Scheduler_clock_ptr | m_scheduler_clock |
| Scheduler clock. More... | |
| std::unordered_map< Scheduler_clock_ptr, Task_queue_type > | m_task_phases |
| Tasks may be divided into phases, each phase works according to a defined clock, registered with "register_phase" function. More... | |
| std::unordered_map< Scheduler_clock_ptr, Mutex_type > | m_task_phases_locks |
| Dependency_tracker_ptr | m_dependencies |
| Task dependencies, used mainly to implement dependencies between events in one transaction or between commit event and parent transaction last event (to disallow a transaction to enter a commit when a parent transaction, i.e. More... | |
| Thread_pool_ptr | m_thread_pool |
| Thread pool with threads that execute tasks. More... | |
| std::atomic_flag | m_is_error = ATOMIC_FLAG_INIT |
| std::atomic< Scheduler_status > | m_status {Scheduler_status::idle} |
| Observability variables. More... | |
| std::atomic< std::size_t > | m_timeouts {0} |
| unsigned int | m_instance_id {0} |
| Instance id. More... | |
| Statistics_instance_monitor_ref | m_stat_monitor |
| Statistics monitoring object for the current instance. More... | |
| std::atomic< std::size_t > | m_allowed_task_count |
| Maximum allowed number of tasks active in the scheduler. More... | |
| bool | m_wait_for_task_limit {false} |
| Information on whether enqueueing thread is waiting for available task count limit. More... | |
| std::atomic< bool > | m_unblock_request {0} |
| Stores unblock requests from external threads, will allow N task to enter phase. More... | |
| Scheduler_psi | m_psi |
| PSI parameters. More... | |
Main scheduling class.
|
protected |
|
protected |
| using mysql::scheduler::Scheduler::Task_queue_type = std::priority_queue<Scheduled_task> |
| using mysql::scheduler::Scheduler::Thread_pool_ptr = std::shared_ptr<Thread_pool_type> |
|
private |
|
virtual |
Destructor.
|
delete |
Copying constructor is deleted.
| mysql::scheduler::Scheduler::Scheduler | ( | Thread_pool_ptr | shared_thread_pool, |
| Scheduler_clock_ptr | shared_clock, | ||
| Dependency_tracker_ptr | dependency_tracker, | ||
| int | instance_id = 0, |
||
| std::size_t | allowed_task_count = std::numeric_limits<std::size_t>::max(), |
||
| Scheduler_psi | psi_params = {} |
||
| ) |
Constructor.
| shared_thread_pool | Shared thread pool |
| shared_clock | Shared scheduler clock |
| dependency_tracker | Dependency tracker |
| instance_id | Unique instance id for statistics monitoring |
| allowed_task_count | Sets the number of ongoing tasks limit to selected number. If the number of tasks reaches this number, scheduler will synchronize to reach 25% of allowed_count tasks |
| psi_params | PSI parameters |
|
protected |
add_dependency
| predecessor | Predecessor task ID |
| successor | Sucessor task ID |
Add dependency function
| true | Dependency added successfully |
| false | Failed to add dependency |
|
protected |
Checks phase queues and return true if all are empty.
|
protected |
Function used to notify that task ended its execution.
| schedule | Task schedule, containing current phase information |
| task_delay | Delay of the whole task w.r.t. scheduler clock |
| task_error | True if enqueued task returned an error, false otherwise |
| repeatable_task | Shared repeatable task state |
| current_thread_id | THP id of the currently executing thread |
|
protected |
Function used to notify that task phase ended its execution.
| schedule | Task schedule |
| task_error | True if enqueued task returned an error, false otherwise |
| phase_id | Executed phase sequence number |
| void mysql::scheduler::Scheduler::deinit | ( | ) |
Ends the scheduler if stop was not yet requested.
|
protected |
end_execution
| bool mysql::scheduler::Scheduler::enqueue | ( | Task_schedule_ptr | schedule, |
| FType && | task, | ||
| Args &&... | args | ||
| ) |
Enqueues a task.
Function adding task to the scheduler
| FType | Functor type |
| Args | Functor arguments parameter pack type |
| schedule | Pointer to task schedule |
| task | Functor to execute |
| args | Functor arguments |
| bool mysql::scheduler::Scheduler::enqueue_after | ( | const Task_id & | predecessor, |
| Task_schedule_ptr | schedule, | ||
| FType && | task, | ||
| Args &&... | args | ||
| ) |
Enqueues a task that should execute after a predecessor task finishes.
Function adding task to the scheduler
| FType | Functor type |
| Args | Functor arguments parameter pack type |
| predecessor | Task id that should execute prior to the task |
| schedule | Pointer to task schedule |
| task | Functor to execute |
| args | Functor arguments |
|
protected |
Enqueue helper.
Puts Scheduled task object into the task queue
| task | Scheduled task rvalue reference |
|
protected |
Enqueues a task identified by the task_id.
Function adding task to the scheduler
| FType | Functor type |
| Args | Functor arguments parameter pack type |
| schedule | Pointer to task schedule |
| task | Functor to execute |
| args | Functor arguments |
|
protected |
Function that enqueues next task phase.
| schedule | Task schedule, containing current phase information |
| task_delay | Delay of the whole task w.r.t. scheduler clock |
| repeatable_task | Shared repeatable task state for this phase |
| current_thread_id | THP id of the currently executing thread |
| phase_id | Enqueued phase sequence number |
| bool mysql::scheduler::Scheduler::ensure_space | ( | ) |
Ensure space for the next task (eliminates possible wait in the enqueue)
|
protected |
|
protected |
| std::size_t mysql::scheduler::Scheduler::get_scheduled_tasks_count | ( | ) | const |
Obtain the number of scheduled, ongoing tasks.
| std::size_t mysql::scheduler::Scheduler::get_timeouts | ( | ) | const |
Access "timeouts" observability statistics - the number of times scheduler timed out while waiting for tasks to schedule.
|
protected |
Worker error handling function.
| bool mysql::scheduler::Scheduler::is_error | ( | ) | const |
Internal function to check on whether error has been set.
|
protected |
Internal function to check if immediate stop was requested, externally or due to an error.
|
protected |
Checks whether any phase of the task is ready for execution.
|
protected |
Helper function to check if the task at the top of the task queue is ready.
| void mysql::scheduler::Scheduler::notify_scheduler | ( | ) |
Function used to notify scheduler thread by the other thread (enqueueing thread / worker poll thread)
| bool mysql::scheduler::Scheduler::register_phase | ( | Scheduler_clock_ptr | phase_clock | ) |
Register task phase run according to the given clock.
| phase_clock | Phase clock for the task phase |
| void mysql::scheduler::Scheduler::request_unblock | ( | ) |
This function requests unblocking of the scheduler.
After this call, the scheduler will try to unblock the workflow
|
protected |
Scheduler run function.
Function for the thread which schedules tasks for execution
|
protected |
Scheduler error handling function.
| void mysql::scheduler::Scheduler::stop_now | ( | ) |
Instruct the scheduler to stop now and withdraw all work.
| bool mysql::scheduler::Scheduler::synchronize | ( | bool | force = true, |
| bool | print_checkpoint = false |
||
| ) |
This function waits for currently fired tasks to finish (blocking)
| force | If enabled, wait for tasks even in case of error or stop requested |
| print_checkpoint | Prints diagnostics during waiting |
| true | Successfully synchronized |
| false | Stopped while waiting |
|
protected |
Synchronizes the scheduler by actively waiting for finished tasks count.
| print_checkpoint | Prints diagnostics during waiting |
| expected | Expected scheduled task count after synchronization |
| force | When true, force synchronization regardless of errors / stop requests |
|
protected |
Actively waits for percent of tasks to finish.
This function is called to reduce contention if clock queue is full. Instead of syncing after one pop(), we are allowing the scheduler to process portion of tasks. To boost performance, we wait only for a percent of tasks to finish
| print_checkpoint | Prints diagnostics during waiting |
|
protected |
Waits for percent of tasks to finish.
This function is called to reduce contention if clock queue is full. Instead of syncing after one pop(), we are allowing the scheduler to process portion of tasks. To boost performance, we wait only for a percent of tasks to finish. Partial waiting is available only for the active synchronization method, which is a default one
| print_checkpoint | Prints diagnostics during waiting |
|
protected |
Waits for scheduler thread to finish its work.
|
protected |
|
private |
Maximum allowed number of tasks active in the scheduler.
Above this threshold, provider will block and wait (synchronize_partial)
|
private |
cv used by a thread requesting scheduler to finish its work
|
private |
Cv used by the scheduler main thread to wait on, when no task is available or tasks in m_task queue are not read to execute.
|
private |
Ensures deinit() is executed only once.
|
private |
Task dependencies, used mainly to implement dependencies between events in one transaction or between commit event and parent transaction last event (to disallow a transaction to enter a commit when a parent transaction, i.e.
commit order parent id, did not finish its execution
|
private |
Instance id.
|
private |
|
mutableprivate |
Mutex used to finish execution of the Scheduler, protects m_scheduler_thread_active and it is used together with m_cv_end.
|
mutableprivate |
Mutex protecting access to scheduler notifications.
|
mutableprivate |
This mutex protect access to phase.
|
mutableprivate |
Mutex protecting access to m_tasks, data structure populated by the enqueuing thread.
|
private |
True if scheduler has been notified.
Used to release mutex before calling notify on cv, which would cause a thread to wake up and block immediately
Protected by m_mutex_scheduler
|
private |
Lock-free queue of task IDs that are ready for execution.
|
private |
PSI parameters.
|
private |
The number of scheduled tasks.
|
private |
Variable that checks whether scheduler should run (true) or end its execution (false)
|
private |
Scheduler clock.
|
private |
Scheduler main thread.
|
private |
Variable indicating that scheduler thread finished its execution, protected with m_mutex_end.
|
private |
Statistics monitoring object for the current instance.
|
private |
Observability variables.
|
private |
Variable that instructs scheduler to withdraw all work and stop.
|
private |
Tasks may be divided into phases, each phase works according to a defined clock, registered with "register_phase" function.
This map holds all registered phases: phase clock (key) and associated phase queue, in which task phases wait for their phase dependencies (defined by the clock) to finish. As an example, task may be divided into "apply" phase and "commit" phase. "commit" phase works with additional start-to-start commit order dependencies (register workers in order of their commit order).
|
private |
|
private |
This is a queue which holds task ready for scheduling.
The lower scheduled time for a task, the higher the task priority. This queue is used by 2 threads, task provider and scheduler main thread. We synchronize it with m_mutex_scheduler
|
private |
Internal map that keeps tasks waiting for dependencies defined in the Dependency Graph (distinct from time dependencies implemented in the Scheduler clock)
|
private |
Thread pool with threads that execute tasks.
|
private |
|
private |
Stores unblock requests from external threads, will allow N task to enter phase.
|
private |
Information on whether enqueueing thread is waiting for available task count limit.
We allow for dirty reads.
|
staticconstexpr |