A newer version of this documentation is available: View the Latest Version

Configure the runners

Runners are doing the job:
  • Runners execute the central logic of each step when the step is ready for execution

  • It must be active at least one runner.

  • It is preferable to have multiple runners with the multiprocessing capabilities of Pacemaker

How to execute a runner

A Pacemaker runner is a message queue consumer.

You need to run the default Magento queue:consumers:start command to start a Pacemaker pipeline runner:
<PATH_TO_MAGENTO_ROOT>/bin/magento queue:consumers:start pipelineRunner

Avoid using cron_consumers_runner

Magento provides the ability to start MQ consumers via cron.

We advise not to use this feature.

Supervisor is a much better option to manage your consumers.

In cases you are using tools like Supervisor to run Pacemaker runners, you must update your cron_consumer_runner configuration not to start pipelineRunner.

Otherwise, this will lead to runtime issues because you will have different versions of pipelineRunner running, probably with different configurations.

Using Supervisor for Pacemaker runners

We recommend using Supervisor to run multiple runners and ensure that all runners are up and running all the time.

See the following Supervisor configuration example:
stderr_logfile = <MAGENTO_ROOT>/var/pipelineRunner.log
autorestart = 1
autostart = 1
startsecs = 0
startretries = 0
directory = <MAGENTO_ROOT>
numprocs = 4
process_name = %(program_name)s_%(process_num)02d
user = www-data
command = /usr/bin/env php <MAGENTO_ROOT>/bin/magento queue:consumers:start --max-messages 1 pipelineRunner

Limit runner messages (--max-messages 1)

We recommend using the --max-messages option with value 1 to avoid stateful PHP code issues.

It depends a lot on how you implement your job executors; vendor code with a state set can still cause problems at any time that are easy to fix but hard to detect.

startsecs config option

The startsecs option is required since we have executors running faster than one second.

In this case, the Supervisor would stop restarting the runners.

If you’re using MacOS X please refer to how to setup RabbitMQ on MacOS X