Migration

The indexer in the current Pacemaker version has a newer processing logic compared to older Pacemaker installations

The following adjustments must be made to ensure error-free execution:

Removal or deactivation of the TechDivision_IndexSuspender module
1. for example with the command
  1. composer remove techdivision/indexsuspender
2. Now that indexing is covered by Magento’s standard Index Pipeline functionality, the Suspender logic is no longer needed
Removal or deactivation of the TechDivision_IndexTriggerSuspender module
1. e.g. with the Composer command
  1. composer remove techdivision/index-trigger-suspender
2. Since the indexing is now covered by the Magento standard functionality with index pipeline, the trigger logic is no longer needed
Testing of all customized/project-specific import pipelines
1. Removal of the steps index_suspender_start or steps with the executor type TechDivision\PacemakerImportBase\Model\Executor\IndexSuspender\Start, because this logic is not used anymore
  1. The executor still exists but only logs info
2. Removal of the steps index_suspender_stop or steps with the executor type TechDivision\PacemakerImportBase\Model\Executor\IndexSuspender\Stop, because this logic is not used anymore
  1. The executor still exists, but only logs info
3. Removal of steps with executor type TechDivision\PacemakerImportBase\Model\Executor\TriggerSuspender\Start, because this logic is no longer applied The executor still exists, but only logs info
4. Removal of steps with executor type TechDivision\PacemakerImportBase\Model\Executor\TriggerSuspender\Stop, because this logic is no longer applied The executor still exists, but only logs info
5. Removal of the steps reindex_full or steps with the executor type TechDivision\PacemakerImportBase\Model\Executor\ConfiguredReindex or TechDivision\ProcessPipelines\Model\Executor\Reindex, because this logic is no longer applied and replaced by the force reindex step of the indexer pipeline
6. Equipping the import steps with extended conditions depending on the indexer
  1. NoConflictingIndexStepProgress
  2. NoConflictingIndexProcess
  3. NoExeedMaxTime
    
    The logic for index suspending was replaced and optimized with the pipeline and step conditions of the Pacemaker indexer
Adaptation and extension of all adapted (project-specific) import pipelines
1. Integration of an index setter step to optimize the index behavior
  1. A separate virtual executor should be derived for each import
2. Integration of an indexer spawn step to ensure that an indexer pipeline is created and executed on demand after the import to shorten the waiting time until the next heartbeat

Example

The following price-import pipeline serves as an example to illustrate the changes within the XML definition:

Old logic

Price-Import Pipeline without Pacemaker Indexer (old logic)

<pipeline name="pacemaker_import_price"
          description="Pacemaker Price Import" use-working-directory="true" expire-in="6 hours">
    <conditions>
        <pipeline_condition type="TechDivision\ProcessPipelines\Helper\Condition\Pipeline\NoAutoSpawn"
                            description="No automatic start for this pipeline"/>
    </conditions>
    <step name="move_files"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\MoveFilesToWorkingDirectory"
          sortOrder="10" description="Move files to working directory.">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
        </conditions>
    </step>
    <step name="price_transformation"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\TransformationExecutor"
          sortOrder="20" description="Transform files into correct format and map values.">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
        </conditions>
    </step>
    <step name="index_suspender_start"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\IndexSuspender\Start"
          sortOrder="30" description="Suspend delta indexer" >
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
        </conditions>
    </step>
    <step name="price_import"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\ImportExecutor"
          sortOrder="40" description="Import price">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\PacemakerImportPrice\Virtual\Model\Condition\Step\CouldSkipByNoFiles"
                    description="Check files to import."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
            <step_condition
                    type="TechDivision\PacemakerImportPrice\Virtual\Condition\NoConflictingStepsInProcess"
                    description="Avoid conflicts between import steps."/>
            <step_condition
                    type="TechDivision\PacemakerIndexer\Virtual\Condition\NoConflictingIndexStepProcess"
                    description="Avoid conflicting steps for reindex process."/>
            <step_condition type="TechDivision\PacemakerImportBase\Model\Condition\Step\NoConflictingIndexingProcess"
                            description="Avoid conflicting index processes for reindex process."/>
            <step_condition
                    type="TechDivision\PacemakerImportBase\Model\Condition\Step\NoExceedMaxTime"
                    description="Prefer an indexer run if the configured time has been exceeded."/>
        </conditions>
        <arguments>
            <argument key="command" value="import:products:price" />
            <argument key="operation" value="add-update" />
        </arguments>
    </step>
    <step name="tier_price_import"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\ImportExecutor"
          sortOrder="50" description="Import tier price">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
            <step_condition
                    type="TechDivision\PacemakerImportPrice\Virtual\Condition\NoConflictingStepsInProcess"
                    description="Avoid conflicts between import steps."/>
        </conditions>
        <arguments>
            <argument key="command" value="import:products:price:tier"/>
            <argument key="operation" value="add-update"/>
        </arguments>
    </step>
    <step name="reindex_full"
          executorType="TechDivision\PacemakerImportPrice\Virtual\Executor\ConfiguredPriceReindex"
          sortOrder="70" description="Reindex full">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit2"
                    description="If something went wrong while reindexing, try it a second time"/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsFinished"
                    description="Previous step needs to be finished."/>
            <step_condition
                    type="TechDivision\PacemakerImportBase\Virtual\Condition\NoConflictingReindexStepsInProcess"
                    description="Avoid conflicting steps for reindex process."/>
        </conditions>
    </step>
    <step name="index_suspender_stop"
          executorType="TechDivision\PacemakerImportPrice\Virtual\Executor\IndexSuspender\Stop"
          sortOrder="71" description="Resume delta indexer" runAlwaysStep="true">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsFinished"
                    description="Previous step needs to be finished."/>
        </conditions>
    </step>
</pipeline>

New logic

Price Import Pipeline with Pacemaker Indexer (new logic)

<pipeline name="pacemaker_import_price"
          description="Pacemaker Price Import" use-working-directory="true" expire-in="6 hours">
    <conditions>
        <pipeline_condition type="TechDivision\ProcessPipelines\Helper\Condition\Pipeline\NoAutoSpawn"
                            description="No automatic start for this pipeline"/>
    </conditions>
    <step name="move_files"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\MoveFilesToWorkingDirectory"
          sortOrder="10" description="Move files to working directory.">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
        </conditions>
    </step>
    <step name="price_transformation"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\TransformationExecutor"
          sortOrder="20" description="Transform files into correct format and map values.">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
        </conditions>
    </step>
    <step name="price_import"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\ImportExecutor" sortOrder="40"
          description="Import price">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\PacemakerImportPrice\Virtual\Model\Condition\Step\CouldSkipByNoFiles"
                    description="Check files to import."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
            <step_condition
                    type="TechDivision\PacemakerImportPrice\Virtual\Condition\NoConflictingStepsInProcess"
                    description="Avoid conflicts between import steps."/>
        </conditions>
        <arguments>
            <argument key="command" value="import:products:price" />
            <argument key="operation" value="add-update" />
        </arguments>
    </step>
    <step name="tier_price_import"
          executorType="TechDivision\PacemakerImportBase\Model\Executor\ImportExecutor" sortOrder="50"
          description="Import tier price">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
            <step_condition
                    type="TechDivision\PacemakerImportPrice\Virtual\Condition\NoConflictingStepsInProcess"
                    description="Avoid conflicts between import steps."/>
        </conditions>
        <arguments>
            <argument key="command" value="import:products:price:tier"/>
            <argument key="operation" value="add-update"/>
        </arguments>
    </step>
    <step name="index_price_setter"
          executorType="TechDivision\PacemakerImportPrice\Virtual\Model\Indexer\PriceIndexSetter"
          sortOrder="60" description="Set index info for related indexer">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
        </conditions>
    </step>
    <step name="spawn_indexer"
          executorType="TechDivision\PacemakerIndexer\Model\IndexerSpawnExecutor" sortOrder="99"
          description="Spawn new indexer pipeline after import.">
        <conditions>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1"
                    description="Try once."/>
            <step_condition
                    type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted"
                    description="Previous step needs to be finished."/>
        </conditions>
    </step>
</pipeline>

Migration

Example

WIR HÄTTEN IHNEN DIESE MELDUNG AUCH GERNE ERSPART ...