Configure the runners
- Runners are doing the job:
Runners execute the central logic of each step when the step is ready for execution
It must be active at least one runner.
It is preferable to have multiple runners with the multiprocessing capabilities of Pacemaker
How to execute a runner
A Pacemaker runner is a message queue consumer.
- You need to run the default Magento
queue:consumers:startcommand to start a Pacemaker pipeline runner:
<PATH_TO_MAGENTO_ROOT>/bin/magento queue:consumers:start pipelineRunner
Magento provides the ability to start MQ consumers via cron.
We advise not to use this feature.
Supervisor is a much better option to manage your consumers.
In cases you are using tools like Supervisor to run Pacemaker runners, you must update your
configuration not to start
Otherwise, this will lead to runtime issues because you will have different versions of
probably with different configurations.
Using Supervisor for Pacemaker runners
We recommend using Supervisor to run multiple runners and ensure that all runners are up and running all the time.
- See the following Supervisor configuration example:
[program:pipelineRunner] stderr_logfile = <MAGENTO_ROOT>/var/pipelineRunner.log autorestart = 1 autostart = 1 startsecs = 0 startretries = 0 directory = <MAGENTO_ROOT> numprocs = 4 process_name = %(program_name)s_%(process_num)02d user = www-data command = /usr/bin/env php <MAGENTO_ROOT>/bin/magento queue:consumers:start --max-messages 1 pipelineRunner
Limit runner messages (--max-messages 1)
We recommend using the
1 to avoid stateful PHP code issues.
It depends a lot on how you implement your job executors; vendor code with a state set can still cause problems at any time that are easy to fix but hard to detect.
startsecs config option
startsecs option is required since we have executors running faster than one second.
In this case, the Supervisor would stop restarting the runners.
If you’re using MacOS X please refer to how to setup RabbitMQ on MacOS X