Pipeline Monitor

The Pipeline Monitor is a simple onboard tool to monitor the pipeline’s status and inform about status changes with mail.

The pipelines monitoring and their schedules can get executed with the following CLI command:

bin/magento cron:run --group pacemaker

Cron run workflow:

Setting the pipelines to be observed
- The pipelines get set with the pipeline observables in the DB table pacemaker_pipelines_monitor_observable
- Every 15 minutes, run’s a check
- The schedule is configurable

Set the pipeline status
- After the monitor knows the observables, completed pipelines are checked and written to the DB table pacemaker_pipelines_monitor_status
- The time of the status note is saved and checked at the next execution of the schedule (default = every 5 minutes)

Notification by email
- The initial statuses are transmitted by email to the configured sender
- Sending takes place according to the configurable schedule

Status change and notification detection
- If a pipeline fails, the status change of the respective pipelines (by name) gets set and the notification is determined and sent

The following status monitoring is available as of version 2.1.0 of the pipeline monitor

If a pipeline expires, the expiring info gets set and the notification is determined and sent.
- The expiration date is normally defined in the pipeline XML
- The information is processed and provided by the heartbeat
If a pipeline gets a waiting status, a step is in "enqueued" status and is not processed by consumers/executors.
- The time span, how long a step may have the enqueued status can be configured.
A pipeline gets the ghosted status, if it contains steps, which have the status "running", but no process is present and therefore would always remain in this status.
- Termination of processes may happen during deployment processes or system shutdowns.

The following status monitoring is available as of version 2.4.0 of the pipeline monitor

If a pipeline has reached an end status but there is no finished_at date, then the missing_finished_at status will get set
- Pipeline end status are success and canceled
With this status, you can see pipelines in your system, which are finished, but do not have a finished_at date in the database
- This most likely happens through manual intervention

The following status monitoring is available as of version 2.7.0 of the pipeline monitor

If a pipeline failed, the failed_pipelines status will get set and you will be notified
- Pipeline error status are canceled, error and abandoned

The backend view was introduced in version 2.6.0 of the pipeline monitor

You can find this view in the backend under Pacemaker → System → Pipelines Monitor
There you have an overview of all pipeline status changes on your system
- Per default it is sorted by status id in descending order