Validation

For large CSV files, as an example > 100 MB, the validation can drastically slow down the import process.

For this reason, the validation is highly customizable and can be turned off completely if needed

Switch-Off validaton

To disable the validation of your import CSV files, you need to overwrite the corresponding shortcuts with a snippet, e.g., in the custom etc/configuration/shortcuts.json.

It is strongly recommended that, by default, the value for custom-configuration-dir is <magento-install-directory>/app/etc/configuration
As soon as a default configuration is copied, changed, extended, or modified and stored under the path specified in custom-configuration-dir, only this configuration takes effect and always overwrites the original configuration
The configuration handling is mainly based on the PHP framework Symfony, and therefore unexpected effects maid occur in different configuration constellations
- Consequently, it is always recommended to test modified configurations concerning unwanted side effects

Example

Disable validation from the add-update process of your product import:

Remove the general/catalog_product/validate operation from the corresponding shortcuts.json file:

{
  "shortcuts": {
    "ce": {
      "add-update": [
        ...,
        "general/eav_attribute/add-update.swatch-values",
        "general/catalog_product/validate", (1)
        ...
      ]
    }
  }
}

1	Line to remove to disable the catalog product validation

The custom snippet file etc/configuration/shortcuts.json must now look like the following code:

{
  "shortcuts": {
    "ce": {
      "add-update": [
        "general/general/global-data",
        "general/general/move-files",
        "general/catalog_product/collect-data",
        "general/eav_attribute/convert",
        "general/eav_attribute/add-update.options",
        "general/eav_attribute/add-update.option-values",
        "general/eav_attribute/add-update.swatch-values",
        "general/catalog_category/convert",
        "ce/catalog_category/sort",
        "ce/catalog_category/add-update",
        "ce/catalog_category/add-update.path",
        "ce/catalog_category/add-update.url-rewrite",
        "general/catalog_category/children-count",
        "ce/catalog_product/add-update",
        "ce/catalog_product/add-update.variants",
        "ce/catalog_product/add-update.bundles",
        "ce/catalog_product/add-update.links",
        "ce/catalog_product/add-update.grouped",
        "ce/catalog_product/add-update.media",
        "general/catalog_product/add-update.msi",
        "general/catalog_product/add-update.url-rewrites"
      ]
    }
  }
}

Custom validations

If overriding this definition, e.g., with a custom snippet like custom-configuration-dir>/operations.json, you gain complete control over what is validated and how the validation works.

The validation operation gets generally based on a validator subject, an observer, some listeners, and various callbacks
The validations get implemented as callbacks that allow you to register one or more validation callbacks for each column
Pacemaker Community Edition comes with additional specialized callbacks
- Only for the corresponding columns and the user-defined regex validator and therefore following for the integration of user-defined regex-based validation additional specialized callbacks get used

Example

See the product import validation operation declared in the operations.json of the repository techdivision/import-product.

For display we removed the unnecessary parts for you in this example:

{
  "operations": {
    "general": {
      "catalog_product": {
        "validate": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  ...
                  "params" : {
                    "custom-validations" : { (1)
                      "sku" :  [ "/.+/" ],
                      "product_type": [ "simple", "virtual", "configurable", "bundle", "grouped", "giftcard" ],
                      "visibility": [ "Not Visible Individually", "Catalog", "Search", "Catalog, Search" ]
                    }
                  },
                  "callbacks": [ (2)
                    {
                      "sku": [ "import.callback.custom.regex.validator" ],  (3)
                      "store_view_code": [ "import.callback.store.view.code.validator" ],
                      "attribute_set_code": [ "import.callback.attribute.set.name.validator" ],
                      "product_type": [ "import.callback.custom.array.validator" ], (4)
                      "tax_class_id": [ "import_product.callback.validator.tax.class" ],
                      "product_websites": [ "import.callback.store.website.validator" ],
                      "visibility": [ "import.callback.visibility.validator" ],
                      "related_skus": [ "import_product.callback.validator.link" ],
                      "upsell_skus": [ "import_product.callback.validator.link" ],
                      "crosssell_skus": [ "import_product.callback.validator.link" ],
                      "created_at" : [ "import.callback.validator.datetime" ],
                      "updated_at" : [ "import.callback.validator.datetime" ],
                      "special_price_to_date" : [ "import.callback.validator.datetime" ],
                      "special_price_from_date" : [ "import.callback.validator.datetime" ],
                      "custom_design_to" : [ "import.callback.validator.datetime" ], (5)
                      "custom_design_from" : [ "import.callback.validator.datetime" ], (6)
                      "new_to_date" : [ "import.callback.validator.datetime" ],
                      "new_from_date" : [ "import.callback.validator.datetime" ],
                      "price" : [ "import.callback.validator.number" ],
                      "special_price" : [ "import.callback.validator.number" ],
                      "map_price" : [ "import.callback.validator.number" ],
                      "msrp_price" : [ "import.callback.validator.number" ],
                      "qty" : [ "import.callback.validator.number" ]
                    }
                  ]
                }
              ]
            }
          }
        }
      }
    }
  }
}

1	user-defined `"custom-validations" :` section for regex-based validation
2	User definded callbacks section
3	User definded `import.callback.custom.regex.validator` for the field `sku`
4	User definded `import.callback.custom.array.validator` for the field `product_type`
5	User definded field `"custom_design_to" :` with the provided default validator `import.callback.validator.datetime` for `datetime` fields
6	User definded field `"custom_design_from" :` with the provided default validator `import.callback.validator.datetime` for `datetime` fields

In case of problems with a validator or if you do not need a validator, e.g. because the import file does not contain a particular column, you can remove the corresponding callback from your custom snippet

Regex validator

Example

To check and validate the SKU column, we use a regex validator with a regular expression:

The regular expression /.+/ validates that any character with at least one character is allowed for the SKU check

If you want to allow only digits, you can change the regular expression in the params/custom-validations array from:

/.+/

to:

/\+d/

Our recommendation for a good regex builder to test and build our regular expressions is regex 101

Custom array validator

The custom array validator will be a great choice if you do not want to use a regular expression but instead want to use a list of allowed values.

In general we use it the custom array validator to check the product type.

Example

To add your custom product type my_product_type, expand the list in the params/custom-validations array as follow:

...
"params" : {
  "custom-validations": {
    ...,
    "product_type": [
      "simple",
      "virtual",
      "configurable",
      "bundle",
      "grouped",
      "giftcard",
      "my_product_type" (1)
      ]
    }
  },
  "callbacks": [
...

1	Your custom product type `my_product_type`, added to the list of already existing product types

Validation

Switch-Off validaton

Example

Custom validations

Example

Regex validator

Example

Custom array validator

Example

WIR HÄTTEN IHNEN DIESE MELDUNG AUCH GERNE ERSPART ...