Cleanup Pipeline
The functionality of the cleanup pipeline is fully implemented and works without any programmatic adjustments to the component.
-
At regular intervals the cleanup pipeline ensures that the directories of pipelines are
-
cleaned up
-
compressed
-
deleted
-
-
In the backend are various existing setting possibilities for the above listed actions
Example configuration
- The following example, a cleanup’s creation/execution time is daily at
08:00
(GMT+2): -
-
All pipelines older than two days should be closed and their contents compressed
-
All pipelines older than three days should also be deleted and removed from the database
-
![be example configuration](../_images/cleanup-pipeline/be-example-configuration.png)
Cleanup execution
- No cleanup gets created or executed before
08:00
:
![cleanup execution](../_images/cleanup-pipeline/cleanup-execution.png)
- The pipeline directories look like this before the cleanup:
![system log](../_images/cleanup-pipeline/system-log.png)
- When the heartbeat runs every minute at 08:00, a cleanup pipeline gets created and executed according to the configuration:
![created cleanup pipeline](../_images/cleanup-pipeline/created-cleanup-pipeline.png)
- The pipeline directories are well ordered and compressed:
![27 tar](../_images/cleanup-pipeline/27-tar.png)
Cleanup results
The results of the pipeline cleanups can get reviewed in the individual steps of the pipeline.
Cancel
- To optimize the data storage of pipelines and to trigger the optimization process, pipelines get canceled:
-
-
After their
Expire Date
-
On the basis of the Cancel configuration with the heartbeat
-
Compress Step
The compress step cleans empty pipeline directories and compresses pipeline directories with existing files.
The compress step is recommended for import functions |
![compress step](../_images/cleanup-pipeline/compress-step.png)
Delete Step
The delete step deletes all existing directories of pipelines based on the time span stored in the configuration.
-
If the configuration also specifies that a pipeline cleanup takes place in the database, the pipeline entities and their steps get deleted as well
![delete step](../_images/cleanup-pipeline/delete-step.png)