FAQs
Why is an import pipeline or import step not executed?
There are several reasons for this:
-
Another import step is currently being processed
-
An index step is being processed
-
An index process (
bin/magento indexer:status) is currently running -
The import logic is disabled by configuration
-
The maintenance mode of the process pipelines is active.
-
There are no or too few "consumers" running.
You can check many of the above reasons using log information.
Why is an indexer pipeline or index step not executed?
There are several reasons for this:
-
Another import step is currently being processed
-
An index step is being processed
-
An index process (
bin/magento indexer:status) is currently running -
The indexer is disabled by configuration
-
The indexer is suppressed by configuration
-
The maintenance mode of the process pipelines is active.
-
There are no or too few "consumers" running.
You can check many of the above reasons using log information.
It is possible to bypass index operations or to suppress the indexer?
Yes, that is possible.
-
You can simply deactivate the Pacemaker indexer by configuration. Please note that the cron processing of the indices is then set to "active" again.
-
Another possibility is to suppress the active Pacemaker Indexer by configuration.
Why do indexer steps get skipped?
The main reasons why indexer steps are skipped are the suppress configuration and performance optimizations during the index run. Please check the log output of the respective steps.
Is it possible to always trigger a full re-index after an import?
There are a few possibilities here. Three of them are roughly described below:
-
You can set all indexer modes to "on save". In this way, a full re-index is obtained by the "purpose check" after each import.
The problem then are the continuous full index runs, which are non-performant in this scenario.
To compensate this, the schedule for the indexer pipeline could be configured from execution every minute to execution once a day. The configuration
Time between two indexer pipelinesshould then be set to 25h.Don’t worry, the index setter ensures that an index pipeline is created after an import.
-
As a second alternative, the threshold limits could be lowered by configuration.
In this case, attention would also have to be paid to indexer behaviour in the minute-by-minute execution.
-
A more code-based approach could be the adaption of the import pipeline itself.
To do this, the steps Index-Setter and Index-Spawner can be removed via XML and replaced with a custom step like the default executor
TechDivision\ProcessPipelines\Model\Executor\Reindexof the Process Pipelines component.This option is not recommended because it is non-performant depending on the import and questions the reasonable standard Magento delta indexing logic.
Is it possible to speed up the import run times?
There are a few options that can be considered to optimize the import run times.
-
Adjustment of the indexer modes
Setting the indexer modes to "on-save" can reduce the import runtime, as no CL tables are supplied. However, there are also disadvantages.
ON-SAVE ON-SCHEDULE Very performant, as no DB triggers are raised with reference to the CL table provisioning.
Full-Reindex must be executed (already available by default in the indexer pipeline)
No data updates may take place during import (indexing can cause import to crash)
Better indexing performance by delta indexing with Pacemaker Indexer
Manual data changes can be made during Imports
Up to a factor of 5 slower in comparison to import run times with
ON-SAVE -
Enable import cache
It is possible to cache the import data during the import. This minimises the file accesses and reduces the runtime. However, more memory (RAM) is used during the import.
-
Use "change-set" detection
With the change-set detection, only actual updates of the data are carried out. This reduces the number of executed SQL update statements to the minimum.