CLI commands

Besides the pipeline control and display in the Magento backend, different pipeline actions can also get executed with CLI commands:

Listing

Description

Command Description

Command	Description
pacemaker:pipeline:list	The `pacemaker:pipeline:list` command is applied to find out which pipelines are installed and available in the system. All pipelines listed from the CLI command are also available for manual creation in the pipelines select menu

pacemaker:pipeline:list

The pacemaker:pipeline:list command is applied to find out which pipelines are installed and available in the system.

All pipelines listed from the CLI command are also available for manual creation in the pipelines select menu

Command

bin/magento pipeline:list

Usage:
  pacemaker:pipeline:list
  pipeline:list

Output

Display with TechDivision Translation Generator installed

+--------------------------------------+--------------------------------------------------------+-------+
| pipeline | description | steps |
+--------------------------------------+--------------------------------------------------------+-------+
| pacemaker_import_catalog | Pacemaker Product Import | 1 |
| clean_up_pipelines | Pack the pipelines and make cleanup by configuration | 3
| pacemaker_translation_index_serial | Translation Generator Export Index Generation | 1 |
| pacemaker_translation_index_parallel | Translation Generator Parallel Export Index Generation | 3
+--------------------------------------+--------------------------------------------------------+-------+

Initialization

Description

Command

Description

pacemaker:pipeline:init

In addition to creating pipelines in the Magento backend and dynamically with the pipeline initializer logic, pipelines can also get generated with a CLI command.

A defined pipeline name must get passed as an argument to generate the correct pipeline

Command

bin/magento pipeline:init

Usage:
  pacemaker:pipeline:init [pipeline_name]
  pipeline:init

Example:

Creation of a cleanup pipeline

bin/magento pipeline:init clean_up_pipelines

Output

Create pipeline "clean_up_pipelines" with 3 steps: "Pack the pipelines and make cleanup by configuration"

Status

Description

Command	Description
pacemaker:pipeline:status	All Pipelines get listed in the Magento Backend as well as with a dedicated CLI Command on the command line.

Command

Description

pacemaker:pipeline:status

All Pipelines get listed in the Magento Backend as well as with a dedicated CLI Command on the command line.

Command

bin/magento pacemaker:pipeline:status

Description:
  Display status of given pipelines
  Default (without options): Pipelines of the last 12h are returned

Usage:
  pacemaker:pipeline:status [options]
  pipeline:status

Options:
  -s, --status-filter=STATUS-FILTER      List of allowed pipeline status [default: "pending,running,error,canceled,success"]
  -w, --watch[=WATCH]                    Start watch, which updates every X seconds
  -f, --created-at-from=CREATED-AT-FROM  Filter for created at field (from)
  -t, --created-at-to=CREATED-AT-TO      Filter for created at field (to)
  -c, --contains[=CONTAINS]              Word/String which is part of the arguments or logs in at least one steps of the pipeline
  -g, --created-at[=CREATED-AT]         Filter for created at field
                                         Examples:
                                         relative time --created-at=-3days, e.g. -g-3days
                                         relative time --created-at=-12hours, e.g. -g-12hours
                                         range => --created-at=-3days,-2days, e.g. -g-3days,-2days
                                         absolute time => --created-at='2022-02-03 12:00:00', e.g. -g'2022-02-03 12:00:00'
  -b, --started-at[=STARTED-AT]         Filter for started at field
  -l, --finished-at[=FINISHED-AT]       Filter for finished at field
  -e, --expires-at[=EXPIRES-AT]         Filter for expires at field

The time zone configuration in the backend is causing impact to the output

Output

Display of the previously initialized cleanup pipeline

+----+--------------------+---------+-------------+---------------------+------------+------------+-------------+
| ID | Name               | Status  | Steps       | Created at          | Expires at | Started at | Finished at |
+----+--------------------+---------+-------------+---------------------+------------+------------+-------------+
| 21 | clean_up_pipelines | pending |  P - P - P  | 2021-08-02 13:07:44 |            |            |             |
+----+--------------------+---------+-------------+---------------------+------------+------------+-------------+

Details

Description

Command Description

pacemaker:pipeline:details

For more information about a pipeline, use the Magento backend step visualization or the dedicated CLI command pipeline:details

The ID of the pipeline must get passed as argument
The pipeline ID can get received, e.g., with the status command

Command

bin/magento pacemaker:pipeline:details

Description:
  Display table with step information for given pipeline

Usage:
  pacemaker:pipeline:details [options] [--] <pipeline-id>
  pipeline:details

Arguments:
  pipeline-id             Pipeline ID which can be retrieved with pacemaker:pipeline:status command

Options:
  -log, --logs            Display steps logs after details table, for more details to logs use pacemaker:pipeline:step:show-log
  -arg, --with-arguments  Display steps arguments in table

Example:

Detail display of the previously initialized cleanup pipeline, e.g., pipeline ID 21

bin/magento pacemaker:pipeline:details 21

Output

+-----+--------------------------------+---------+---------------------+------------+-------------+
| ID | Name | Status | Created at | Started at | Finished
+-----+--------------------------------+---------+---------------------+------------+-------------+
| 171 | compress_pipeline_data | pending | 2021-08-02 13:07:44
| 172 | delete_pipeline_data | pending | 2021-08-02 13:07:44
| 173 | delete_pipelines_from_database | pending | 2021-08-02 13:07:44
+-----+--------------------------------+---------+---------------------+------------+-------------+

Execution Step

Description

Command

Description

pacemaker:pipeline:step:process

Individual steps can get executed or restarted with the CLI.

Any arguments regarding execution limit and other conditions get ignored
The Step ID to get processed must get passed as an argument
The Step ID can get received, e.g., with the details command

Steps that have already been processed or completed can also get executed again, regardless of the pipeline status

Only by a running of the Magento Consumer / PipelineRunner, does the step gets processed technical, the data of the step is transferred to the message queue, regardless of any conditions

Command

bin/magento pipeline:step:process
Usage:
  pacemaker:pipeline:step:process <step_id>
  pipeline:step:process

Example:

Manual execution of a step with subsequent detail output, e.g., Step ID 173

bin/magento pipeline:step:process 173
bin/magento pacemaker:pipeline:details 21

Output

+-----+--------------------------------+----------+---------------------+-------------+
| ID | Name | Status | Created at | Started at | Finished at
+-----+--------------------------------+----------+---------------------+---------------------+-------------+
| 171 | compress_pipeline_data | pending | 2021-08-02 13:07:44
| 172 | delete_pipeline_data | pending | 2021-08-02 13:07:44
| 173 | delete_pipelines_from_database | enqueued | 2021-08-02 13:07:44 | 2021-08-04 13:06:42
+-----+--------------------------------+----------+---------------------+---------------------+-------------+

Log output Step

Description

Command Description

pacemaker:pipeline:step:show-log

During the processing of steps, the step log gets also written

Processing processes and other information get logged
The step log can get viewed in the detailed display of a step in the Magento backend

The ID of the desired step must get passed as an argument
The step ID can get read out, e.g., with the details command

Command

With the following CLI command, a step log can get read

bin/magento pipeline:step:show-log

Example:

Display of a step log, e.g., Step ID 173

bin/magento pipeline:step:show-log 173

Output

--- ATTEMPT 1 ---
[2021-08-04 13:26:37] executor.INFO: [Skipped] Skip delete pipelines, as feature is configured as disabled

Heartbeat

Description

Command	Description
pacemaker:pipeline:heartbeat	A pipeline heartbeat allows the following listed actions The heartbeat performs Steps The heartbeat creates pipelines The heartbeat updates pipelines The execution gets provided as a CLI command.

Command

Description

pacemaker:pipeline:heartbeat

A pipeline heartbeat allows the following listed actions

The heartbeat performs Steps
The heartbeat creates pipelines
The heartbeat updates pipelines

The execution gets provided as a CLI command.

Command

bin/magento pacemaker:pipeline:heartbeat

Description:
  Checks pending steps and pipelines in order to create messages for them.

Usage:
  pacemaker:pipeline:heartbeat [options]
  pipeline:heartbeat

Options:
      --pulse[=PULSE]   [DEPRECATED] In minutes. Determines how pipeline step cron conditions are evaluated.
                        Should match the system cron expression used to execute the heartbeat to avoid missed schedules.
                        The --pulse option is outdated, because the value is calculated automatically in the meantime.
                        Examples:
                        */2 * * * * bin/magento pipeline:heartbeat --pulse 2
                        * * * * *  bin/magento pipeline:heartbeat
                         [default: "1"]

Example / Output

Execution of a pending pipeline

CLI command

bin/magento pacemaker:pipeline:status

Output

+----+--------------------+---------+-------------+---------------------+------------+---------------------+---------------------+
| ID | Name | Status | Steps | Created at | Expires at | Started at | Finished
+----+--------------------+---------+-------------+---------------------+------------+---------------------+---------------------+
| 21 | clean_up_pipelines | success | S - S - S | 2021-08-02 13:07:44 | | 2021-08-04 13:26:37 | 2021-08-04 13:26:37
| 22 | clean_up_pipelines | pending | P - P - P | 2021-08-04 14:27:29
+----+--------------------+---------+-------------+---------------------+------------+---------------------+

CLI command

bin/magento pacemaker:pipeline:details 22

Output

+-----+--------------------------------+---------+---------------------+------------+-------------+
| ID | Name | Status | Created at | Started at | Finished at
+-----+--------------------------------+---------+---------------------+------------+-------------+
| 174 | compress_pipeline_data | pending | 2021-08-04 14:27:29 |
| 175 | delete_pipeline_data | pending | 2021-08-04 14:27:29
| 176 | delete_pipelines_from_database | pending | 2021-08-04 14:27:29
+-----+--------------------------------+---------+---------------------+------------+-------------+

CLI command

bin/magento pacemaker:pipeline:heartbeat

CLI command

bin/magento pacemaker:pipeline:details 22

Output

+-----+--------------------------------+----------+---------------------+-------------+
| ID | Name | Status | Created at | Started at | Finished
+-----+--------------------------------+----------+---------------------+-------------+
| 174 | compress_pipeline_data | enqueued | 2021-08-04 14:27:29 | 2021-08-04 14:28:41
| 175 | delete_pipeline_data | pending | 2021-08-04 14:27:29
| 176 | delete_pipelines_from_database | pending | 2021-08-04 14:27:29
+-----+--------------------------------+----------+---------------------+---------------------+-------------+

CLI command

bin/magento pacemaker:pipeline:status

Output

+----+--------------------+---------+-------------+---------------------+------------+---------------------+
| ID | Name | Status | Steps | Created at | Expires at | Started at | Finished
+----+--------------------+---------+-------------+---------------------+------------+---------------------+---------------------+
| 21 | clean_up_pipelines | success | S - S - S | 2021-08-02 13:07:44 | | 2021-08-04 13:26:37 | 2021-08-04 13:26:37
| 22 | clean_up_pipelines | running | R - P - P | 2021-08-04 14:27:29 | | 2021-08-04 14:28:41
+----+--------------------+---------+-------------+---------------------+------------+---------------------+

Consumer

Description

Command	Description
queue:consumer:start	To process steps in a message queue, a Magento consumer gets required. The consumer handles the communication between the Magento system and the configured message queue The pending steps in the MQ are fetched by the runner(s) and processed with the executor logic

Command

Description

queue:consumer:start

To process steps in a message queue, a Magento consumer gets required.

The consumer handles the communication between the Magento system and the configured message queue

The pending steps in the MQ are fetched by the runner(s) and processed with the executor logic

Command

bin/magento queue:consumer:start

Example:

E.g. Rabbit-MQ

bin/magento queue:consumer:start pipeline.runner.amqp

E.g. MySQL-MQ

bin/magento queue:consumer:start pipeline.runner.db

Example / Output

Description

The first step of pipeline 22 gets passed to the message queue (here Rabbit-MQ) for processing
The executor gets now instantiated and processed with the information received from the arguments of the step
Processing parameters can also get passed with the step arguments

Processing pipelines with AMQP PipelineRunner / Magento Consumer

bin/magento pacemaker:pipeline:status

+----+--------------------+---------+-------------+---------------------+------------+---------------------+---------------------+
| ID | Name | Status | Steps | Created at | Expires at | Started at | Finished
+----+--------------------+---------+-------------+---------------------+------------+---------------------+---------------------+
| 21 | clean_up_pipelines | success | S - S - S | 2021-08-02 13:07:44 | | 2021-08-04 13:26:37 | 2021-08-04 13:26:37
| 22 | clean_up_pipelines | running | R - P - P | 2021-08-04 14:27:29 | | 2021-08-04 14:28:41
+----+--------------------+---------+-------------+---------------------+------------+---------------------+---------------------+

bin/magento queue:consumer:start pipeline.runner.amqp

bin/magento pacemaker:pipeline:status

+----+--------------------+---------+-------------+---------------------+------------+---------------------+
| ID | Name | Status | Steps | Created at | Expires at | Started at | Finished at
+----+--------------------+---------+-------------+---------------------+------------+---------------------+
| 21 | clean_up_pipelines | success | S - S - S | 2021-08-02 13:07:44 | | 2021-08-04 13:26:37 | 2021-08-04 13:26:37
| 22 | clean_up_pipelines | success | S - S | 2021-08-04 14:27:29 | | 2021-08-04 14:28:41 | 2021-08-04 14:42:29
+----+--------------------+---------+-------------+---------------------+------------+---------------------+

Output of active steps

Description

Command

Description

pacemaker:pipeline:step:activecount

A separate CLI command can get used to output the number of running steps together with the number of enqueued steps.

As a recommendation, a deployment should not get rolled out until all ongoing processes within the pipelines have been completed

Command

bin/magento pacemaker:pipeline:step:activecount

Example:

Display that two steps are currently running or waiting to get executed

bin/magento pacemaker:pipeline:step:activecount

Example / Output

Found 2 active steps (running /enqueued)

Search for pipelines

Description

Command	Description
pacemaker:pipeline:search	A separate CLI command to search for pipeline infos with given search term in pipeline logs.

Command

Description

pacemaker:pipeline:search

A separate CLI command to search for pipeline infos with given search term in pipeline logs.

Command

bin/magento pacemaker:pipeline:search

Description:
  Search for pipelines by given string/word

Usage:
  pacemaker:pipeline:search [options] [--] <word>

Arguments:
  word                             String/word to search for, which can be retrieved with pacemaker:pipeline:search command

Options:
      --finished-at[=FINISHED-AT]  Filter for finished at field
      --created-at[=CREATED-AT]    Filter for created at field
                                   Examples:
                                   relative time --created-at=-3d
                                   relative time --created-at=-12h
                                   range => --created-at=-3d,-2d
                                   absolute time => --created-at='2022-02-03 12:00:00'
      --started-at[=STARTED-AT]    Filter for started at field

Example:

Display pipeline steps which have the search term "index" in their logs

bin/magento pacemaker:pipeline:search index

Example / Output

+-------------------+---------+---------------------------+---------------------+---------------------+---------------------+
| Pipeline Name     | Step ID | Step Name                 | Started at          | Created at          | Finished at         |
+-------------------+---------+---------------------------+---------------------+---------------------+---------------------+
| pacemaker_indexer | 83      | pacemaker_force_reindex   | 2022-12-20 03:56:07 | 2022-12-20 03:52:14 | 2022-12-20 03:56:22 |
| pacemaker_indexer | 84      | pacemaker_import_reindex  | 2022-12-20 03:56:22 | 2022-12-20 03:52:14 | 2022-12-20 03:56:23 |
| pacemaker_indexer | 85      | pacemaker_indexer_cleanup | 2022-12-20 03:56:23 | 2022-12-20 03:52:14 | 2022-12-20 03:56:23 |
+-------------------+---------+---------------------------+---------------------+---------------------+---------------------+

Cancel Pipelines

Now, with the command cancel, a developer can abort all or certain pipelines with CLI
When the command gets executed, it fetches the steps from the database and cancels the execution of the given steps
For steps in Running or Enqueued or Paused state, the consumer and all child processes get terminated

Description

Command	Description
pacemaker:pipeline:cancel	Cancel given pipelines and abort all running processes, if this option is not disabled.

Command

Description

pacemaker:pipeline:cancel

Cancel given pipelines and abort all running processes, if this option is not disabled.

Command

bin/magento pacemaker:pipeline:cancel

Description:
  Cancel given pipelines and abort all running processes, if this option is not disabled

Usage:
  pacemaker:pipeline:cancel [options] [--] [<pipeline-id>...]
  pipeline:cancel

Arguments:
  pipeline-id           Pipeline ID(s) (optional); If several pipelines are to be cancelled, then specify the values comma-separated

Options:
  -f, --force-abort     Abort all step-states, even those that are already running/enqueued
  -a, --all             Cancel/abort all pipelines, based on the --force-abort option
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Example:

Cancel Pipeline for the given Pipeline ID 21 and force abortion option

bin/magento pacemaker:pipeline:cancel 21 -f

Output

Output if answer to Question is "no" for a pipeline with 3 Steps, where one is running

Are you sure you want to abandon all running and enqueued pipelines? [y/n] n
Following Steps were canceled:
63 (Pipeline-ID: 21)
64 (Pipeline-ID: 21)

Output if answer to Question is "yes" for the same pipeline as above

Are you sure you want to abandon all running and enqueued pipelines? [y/n] y
Following Steps were canceled:
63 (Pipeline-ID: 21)
64 (Pipeline-ID: 21)
65 (Pipeline-ID: 21)

Pause Pipeline

Description

Description:
Just pipelines with the status "pending/enqueued" are paused with their steps.

Stop processing of a given pipeline - set status to "paused"

Usage:
  pacemaker:pipeline:pause [options]

Options:
  -i, --pipeline-ids=PIPELINE-IDS      Pipeline ID(s) (optional); If several pipelines are to be paused, then specify the values comma-separated
  -m, --pipeline-names=PIPELINE-NAMES  Comma separated list of pipeline names
  -a, --all                            Pause all pending and enqueued pipelines in system
  -h, --help                           Display help for the given command. When no command is given display help for the list command
  -q, --quiet                          Do not output any message
  -V, --version                        Display this application version
      --ansi|--no-ansi                 Force (or disable --no-ansi) ANSI output
  -n, --no-interaction                 Do not ask any interactive question
  -v|vv|vvv, --verbose                 Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

/ [START] tab: command example +

Command

With the following CLI command, a step log can get read

bin/magento pacemaker:pipeline:pause

Usage:

pacemaker:pipeline:pause --pipeline-ids <id1,id2...>
pacemaker:pipeline:pause --pipeline-names <pipeline_name1,pipeline_name2,...>
pacemaker:pipeline:pause --all=1

Example:

Indication of a failed pause with shortcuts variant

bin/magento pacemaker:pipeline:pause -i 123,124

bin/magento pacemaker:pipeline:pause -m pacemaker_indexer,clean_up_pipelines

bin/magento pacemaker:pipeline:pause -a

The Pipelines that are paused may be canceled with the command.
see The Cancel Pipelines.

Output

bin/magento pacemaker:pipeline:pause --pipeline_ids 12,54 --pipeline-names pacemaker_indexer

No Pipeline with "pending/enqueued" status was found!

Resume Paused Pipeline

Description

Description:

Just pipelines with the status "paused" are resumed

Continue processing of a given pipeline - set status to "pending"

Usage:
  pacemaker:pipeline:resume [options]

Options:
  -i, --pipeline-ids=PIPELINE-IDS      Pipeline ID(s) (optional); If several pipelines are to be pending, then specify the values comma-separated
  -m, --pipeline-names=PIPELINE-NAMES  Comma separated list of pipeline names
  -a, --all                            append all paused pipelines in system
  -h, --help                           Display help for the given command. When no command is given display help for the list command
  -q, --quiet                          Do not output any message
  -V, --version                        Display this application version
      --ansi|--no-ansi                 Force (or disable --no-ansi) ANSI output
  -n, --no-interaction                 Do not ask any interactive question
  -v|vv|vvv, --verbose                 Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Command

With the following CLI command, a pipeline with their step can set on pending

bin/magento pacemaker:pipeline:resume

Usage:

pacemaker:pipeline:resume --pipeline-ids <id1,id2...>
pacemaker:pipeline:resume --pipeline-names <pipeline_name1,pipeline_name2,...>
pacemaker:pipeline:resume --all=1

Example:

bin/magento pacemaker:pipeline:resume 123,124

Output

Output of a failed command when resuming a paused pipeline

bin/magento pacemaker:pipeline:resume --pipeline-ids 1,4 --pipeline-names clean_up_pipelines

No Pipeline with "paused" status was found!

Step Condition Check

Description

Description:
  Check and info step conditions

Usage:
  pacemaker:pipeline:step:condition-check <step_id>

Arguments:
  step_id               ID of an existing step entry in DB

Command

With the following CLI command, the conditions of the given step will be checked. Please note that the command can check only "pending" steps.

bin/magento pacemaker:pipeline:step:condition-check <<ID>>

Output

Output of the condition check of a pending step

+---------------------------------------------------------------------------------+---------+
| Condition-Type                                                                  | isReady |
+---------------------------------------------------------------------------------+---------+
| TechDivision\PacemakerIndexer\Virtual\Condition\NoConflictingIndexStepProcess   |  x      |
| TechDivision\PacemakerImportBase\Model\Condition\Step\NoImportExecStepIsRunning |  ✓      |
| TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsFinished       |  x      |
| TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1        |  ✓      |
+---------------------------------------------------------------------------------+---------+

CLI commands

Listing

Initialization

Status

Details

Execution Step

Log output Step

Heartbeat

Consumer

Output of active steps

Search for pipelines

Cancel Pipelines

Pause Pipeline

Resume Paused Pipeline

Step Condition Check

WIR HÄTTEN IHNEN DIESE MELDUNG AUCH GERNE ERSPART ...