Pipeline XML

The structure of a pipeline gets defined with a pipeline.xml file.

  • The implementation and processing of dynamic pipelines can get realized with the pipeline initializer logic

pipelines are based on a schema that can get viewed at vendor/techdivision/process-pipelines/etc/pipeline.xsd:
Extract from a pipeline.xml (vendor/techdivision/process-pipelines/etc/pipeline.xml)
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:module:TechDivision_ProcessPipelines:etc/pipeline.xsd"> (1)
    <pipeline name="clean_up_pipelines" description="Pack the pipelines and make cleanup by configuration" use-working-directory="true" expire-in="2 hours"> (2)
        <conditions> (3)
            <pipeline_condition type="TechDivision\ProcessPipelines\Helper\Condition\Pipeline\CleanUp" description="Cron expression check for message import"/> (4)
        </conditions>
        <step name="compress_pipeline_data" executorType="TechDivision\ProcessPipelines\Model\Executor\CompressExecutor" sortOrder="10" description="" expireIn="2 hours"> (5)
            <conditions>
                <step_condition type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit5" description="Try step up to 5 times"/> (6)
            </conditions>
        </step>
        ... (7)
    </pipeline>
    (8)
</config>
1 A Pipeline.XML must always be validated against the applicable pipeline schema
2 A pipeline must be created with the <pipeline> XML tag and contain the name XML attribute
  • To work with files in the pipeline, the use-working-directoy should get defined

  • To cancel a pipeline that exceeds a certain execution time, the expire-in should get defined

3 A pipeline can be equipped with several conditions
4 A pipeline operation must always contain a type that implements the \TechDivision\ProcessPipelines\Api\PipelineConditionInterface
5 A pipeline must consist of at least one step and must meet the following criteria
  • The name of the step must be set and must be unique within a pipeline

  • To cancel a pipeline when a step exceeds a certain execution time, the expireIn at step level should get defined

  • To process steps within a pipeline in the correct order, the sort order should get defined

  • The executorType must get set

    • Here the executor of the step is defined, which must implement the \TechDivision\ProcessPipelines\Api\ExecutorInterface

6 Like pipelines, steps can also have execution conditions
  • If conditions are defined, they must be declared with type and implement the \TechDivision\ProcessPipelines\Api\StepConditionInterface

7 A pipeline can consist of several steps
8 Multiple pipelines can be defined and structured in Pipeline.XML
  • The pipeline XML is subject to the Magento logic and can get extended based on Magento

  • Additional steps can get defined in a different pipeline.xml file for an existing pipeline

  • A replacement or exchange (e.g., of steps or conditions]) is currently NOT possible

  • As an option, a completely new pipeline could get created

  • All pipelines defined with Pipeline.XML are available for selection in the backend for manual creation

All defined pipelines can get listed with the CLI command bin/magento pipelines:list