Setup

Pipeline Initialization

After successful installation, you can now create a cleanup pipeline in the Magento backend:

Navigate to Pacemaker Process Pipelines [Add new pipeline entry]

Pipelines can get initialized in the following ways:

Manually with the backend
Manually with CLI commands
Automated with the heartbeat
- Pipeline condition
- Pipeline Initializer

To further process the created pipeline, the following conditions must apply:

A message queue must get installed on the system. Either AMQP (Rabbit-MQ) or MySQL
A Magento consumer must get executed, which processes the pipelines
The heartbeat must get configured

Message Queue

The following configurations are available in the main module:

Queue-Publisher
Queue-Consumer

Queue-Publisher

queue-publisher.xml

<publisher topic="pipeline.step.process">
    <connection
            name="amqp"
            exchange="techdivision-process-pipelines"
            disabled="false"
    />
    <connection
            name="db"
            exchange="techdivision-process-pipelines-db"
            disabled="true"
    />
</publisher>

Queue-Consumer

queue-consumer.xml

<consumer
    name="pipeline.runner.amqp" queue="pipeline_process_steps"
    connection="amqp"
    handler="TechDivision\ProcessPipelines\Model\Runner::process"
/>
<consumer
    name="pipeline.runner.db" queue="pipeline_process_steps"
    connection="db"
    handler="TechDivision\ProcessPipelines\Model\Runner::process"
/>

Depending on the installed message queue, the necessary connections can get set in app/etc/env.php.

Example

Rabbit-MQ is installed and activated as AMQP:

app/etc/env.php

'queue' => [
    'config' => [
        'publishers' => [
            'pipeline.step.process' => [
                'connections' => [
                    'amqp' => [
                        'name' => 'amqp',
                        'disabled' => true
                    ],
                'db' => [
                        'name' => 'db',
                        'disabled' => false
                    ]
                ]
            ]
        ]
    ],
],

Magento Consumer/Pipeline Runner

Pipeline Runner

At least one Magento consumer must now get started, which processes the pipeline(s).

Local this can get executed with the following command:

bin/magento queue:consumers:start pipeline.runner.ampq

When using the MySQL MQ, the likewise provided runner can get started:

bin/magento queue:consumers:start pipeline.runner.db

In productive environments, the use of Supervisor gets recommended to control better the consumer processes concerning the processing of messages (--max-messages=1)
Alternatively, the Supervisor Consumer module is available, which controls the consumers with a Cron Daemon

Heartbeat

As the last step, the heartbeat must be executed to determine the next step to be processed and pass it to the running consumer with the message queue.

With the following command, the pipeline heartbeat can get executed:

bin/magento pipeline:heartbeat

The heartbeat is a minutely cron job, which can get configured on productive systems, e.g., with the crontab
After the execution of the heartbeat, the pipeline was successfully processed

Setup

Pipeline Initialization

Message Queue

Queue-Publisher

Queue-Consumer

Example

Magento Consumer/Pipeline Runner

Pipeline Runner

Heartbeat

WIR HÄTTEN IHNEN DIESE MELDUNG AUCH GERNE ERSPART ...