FAQs
- Composer runs into auth issues on my Mac OS X machine
- The performance on the production/stating system is worse than on my local machine
- Timestamp Detection does not work
- Supervisor does not restart the runners any more
- What are the general rules
- What type of product for items with features (e.g., color)?
- How do I connect profiling articles?
- How must the file names be structured?
- How must the categories get assigned?
- How must the categories get assigned?
- How can products get assigned to individual websites?
- Values sometimes do not work; why?
- Why are images not displayed even though the path in the file matches the real path?
- The
Product.csvvisibility is not set correctly - URLs are not generated correctly
- Special characters in categories
category.csvandproduct.csv - I get the following SQL error during import after adding a field to one of the Magento standard tables
- Product import - ERROR:
Can’t load product with SKU - Product import - ERROR:
Store Scope is overwritten by default - Product import - ERROR:
Cron Job indexer_update_all_views Integrity constraint Duplicate entry - Attribute import - ERROR:
PreLoadOptionId must be of type array - Image import - Images are not visible after import
- Sorting options controlled by the configurable_variations field, e.g., "Color", "Size", for configurables?
Composer runs into auth issues on my Mac OS X machine
|
As a Solution partner or customer, you have received the ext12345 username along with a token to access the
necessary libraries via Composer.
The received credentials need to get added to the auth.json, which can get located, e.g., in your project’s
source directory like src/auth.json.
<magento-install-dir>/auth.json{
"http-basic": {
"gitlab.met.tdintern.de": {
"username": "ext12345",
"password": "asaZIjkUIollKSADnrlm"
}
}
}Each time Composer gets invoked, these credentials get used for the HTTP download that Composer performs.
In some cases, you will receive a message from Composer that you do not have the necessary permissions to install Pacemaker or any of its packages .
Solution
-
macOS stores credentials in the keychain at the host level. See host entry in the example (e.g., gitlab.met.tdintern.de).
The first time Composer gets invoked, the first host entry from the keychain gets used rather than the local auth.json.
That can cause Pacemaker libraries to fail to load due to lack of access.
Pacemaker requires that you disable caching of system credentials for HTTPS calls.
git config --local --unset-all credential.helper \
&& git config --global --unset-all credential.helper \
&& git config --system --unset-all credential.helpergit config --global --add credential.helper "" && composer clear-cacheFinally, search for met in the keychain tool (see example) and delete the entry gitlab.met.tdintern.de.
The performance on the production/stating system is worse than on my local machine
|
|
The performance between your local and any other system differs significantly, thats why the performance between your local and another system may differ significantly?
Solution
-
The MySQL transaction log different settings
-
The option
innodb_flush_log_at_trx_commithas the value1as default
That is why the transaction log gets rewritten by MySQL after each commit.
The innodb_flush_log_at_trx_commit option controls the balance between strict ACID
compliance for commit operations and higher performance possible when commit-related I/O
operations are reordered and batched.
If you set the value to 2, you can achieve better performance, but then you might lose transactions in case of a crash.
0 |
write and flush once per second |
1 |
write and flush at each commit |
2 |
write at commit, flush once per second |
| ID | Pipeline | Created | Finished | Duration |
|---|---|---|---|---|
219 |
xxx_import_catalog |
Oct 24, 2019 2:47:04 PM |
Oct 24, 2019 4:53:14 PM |
02:06:10 h |
221 |
xxx_import_catalog |
Oct 24, 2019 5:02:02 PM |
Oct 24, 2019 5:05:31 PM |
00:03:29 h |
Timestamp Detection does not work
|
|
I am using Pacemaker Professional Edition > 3.8.0.
-
Everything seems to work fine when trying to enable timestamp detection with
--use-timestamp=true -
Performance is still at the same level after enabling the
change-setdetection to--use-change-set=true -
The log file and the console do not contain any error messages
Solution
-
The date format in your CSV file might differ from the default format, or the necessary date format is not set correctly in the configuration.
Y-m-d H:i:s, create the snippet <magento-install-dir>/app/etc/configuration/operations.json with following content to override the given date format:{
"operations": {
"ee": {
"catalog_product": {
"validate": {
"plugins": {
"subject": {
"id": "import.plugin.subject",
"listeners": [
{
"plugin.process.start": [
"import.listener.reset.loader.eav.attribute.option.value"
],
"plugin.process.success": [
"import.listener.stop.validation"
]
}
],
"subjects": [
{
"id": "import.subject.validator",
"create-imported-file": false,
"date-converter": {
"source-date-format": "Y-m-d H:i:s" (1)
},
"file-resolver": {
"prefix": "product-import"
},
"listeners": [
{
"subject.artefact.header.row.process.start": [
"import.listener.validate.header.row"
]
}
],
"params": {
"custom-validations": {
"sku": [
"/.+/"
],
"product_type": [
"simple",
"virtual",
"configurable",
"bundle",
"grouped",
"giftcard",
"designyourown"
],
"visibility": [
"Not Visible Individually",
"Catalog",
"Search",
"Catalog, Search"
]
}
},
"observers": [
{
"import": [
"import_product.observer.composite.base.validate"
]
}
],
"callbacks": [
{
"sku": [
"import.callback.custom.regex.validator"
],
"store_view_code": [
"import.callback.store.view.code.validator"
],
"attribute_set_code": [
"import.callback.attribute.set.name.validator"
],
"product_type": [
"import.callback.custom.array.validator"
],
"tax_class_id": [
"import_product.callback.validator.tax.class"
],
"product_websites": [
"import.callback.store.website.validator"
],
"visibility": [
"import.callback.visibility.validator"
],
"related_skus": [
"import_product.callback.validator.link"
],
"upsell_skus": [
"import_product.callback.validator.link"
],
"crosssell_skus": [
"import_product.callback.validator.link"
],
"created_at": [
"import.callback.validator.datetime"
],
"updated_at": [
"import.callback.validator.datetime"
],
"special_price_to_date": [
"import.callback.validator.datetime"
],
"special_price_from_date": [
"import.callback.validator.datetime"
],
"custom_design_to": [
"import.callback.validator.datetime"
],
"custom_design_from": [
"import.callback.validator.datetime"
],
"new_to_date": [
"import.callback.validator.datetime"
],
"new_from_date": [
"import.callback.validator.datetime"
],
"price": [
"import.callback.validator.number"
],
"special_price": [
"import.callback.validator.number"
],
"map_price": [
"import.callback.validator.number"
],
"msrp_price": [
"import.callback.validator.number"
],
"qty": [
"import.callback.validator.number"
],
"is_returnable": [
"import_product_ee.callback.rma.validator"
]
}
]
}
]
}
}
},
"replace": {
"plugins": {
"subject": {
"id": "import.plugin.subject",
"subjects": [
{
"id": "import_product_ee.subject.bunch",
"date-converter": {
"source-date-format": "Y-m-d H:i:s" (1)
},
"file-resolver": {
"prefix": "product-import"
},
"params": {
"copy-images": false
},
"observers": [
{
"import": [
"import_product_ee.observer.composite.base.replace"
]
}
]
}
]
}
}
},
"add-update": {
"plugins": {
"subject": {
"id": "import.plugin.subject",
"subjects": [
{
"id": "import_product_ee.subject.bunch",
"date-converter": {
"source-date-format": "Y-m-d H:i:s" (1)
},
"file-resolver": {
"prefix": "product-import"
},
"params": {
"copy-images": false,
"clean-up-media-gallery": true,
"clean-up-empty-image-columns": true,
"clean-up-website-product-relations": true,
"clean-up-category-product-relations": true,
"clean-up-empty-columns": [
"special_price",
"special_price_from_date",
"special_price_to_date"
]
},
"observers": [
{
"import": [
"import_product_ee.observer.composite.base.add_update"
]
}
]
}
]
}
}
}
}
}
}
}| 1 | Use the provided date-converter source-date-format to change the date format to Y-m-d H:i:s |
Supervisor does not restart the runners any more
|
|
The Supervisor worked without problems for an extended period, but suddenly the Runner consumers did not restart.
-
The Supervisor log displays an error such as
FATAL Exited too quickly(the process log may contain details).- Solution:
-
Even if a process terminates with an expected exit code, Supervisor considers the start as a failure if the process runs faster than with the option
startsecsis set.
-
The default value is
1second.
You must set the option to startsecs=0 in the Supervisor configuration accordingly to avoid the behavior described.
What are the general rules
|
|
- Configurable and Simple:
-
-
Use columns
configurable_variationsandconfigurable_variation_labels -
The assignment takes place on the level of the
Configurable-
The SKUs of simples and the reference to the attributes appropriate attribute code get specified with the attribute’s value
-
-
It is configurable via the backend
-
- General:
-
-
File names can be split into several blocks
-
If only one file exists, it must end with
01.csv -
Use comma instead of a semicolon as a separator
-
How must the file names be structured?
How must the categories get assigned?
|
|
Solution
-
the categories must start with the default category
-
the fields
is_anchorandinclude_in_menumust get set to the value1, so categories are clickable and displayed in the menu structure -
the field
store_view_codemust get filled with the valuesde,fr,uk,nl,it,at,ch,es,se,dk-
already corresponds to the conversion as implemented in the base setup
-
How must the categories get assigned?
|
|
Solution
-
the categories must start with the default category
-
the fields
is_anchorandinclude_in_menumust get set to the value1, so categories are clickable and displayed in the menu structure -
the field
store_view_codemust get filled with the valuesde,fr,uk,nl,it,at,ch,es,se,dk-
already corresponds to the conversion as implemented in the base setup
-
How can products get assigned to individual websites?
|
|
Solution
Products get assigned to individual websites in the
product_websites field of
the Product.csv file.
-
Values are comma-separated and contain the existing
website codes(base,additional_base) from Magento
URLs are not generated correctly
|
|
Solution
The product URLs cannot be generated correctly and cause an error.
One reason for this may be that the value in the url_key
field is empty. That leads to Magento automatically generating the URL, but only the product’s name gets used.
Since this value is identical in the file for Configurable and Simples, this situation leads to duplicate key errors when generating the URL rewrites.
Special characters in categories category.csv and product.csv
I get the following SQL error during import after adding a field to one of the Magento standard tables
|
|
SQLSTATE[HY093]:-Invalid parameter number: number of bound variables does not match the number of tokens when executing SQL "INSERT INTO catalog_product_entity_media_gallery
(attribute_id,value,media_type,disabled,modified_at)
VALUES (90,/w/p/wp12-blue_main.jpg,image,0,:modified_at)"
in file `media_20161021-161909_04.csv` on line 407Solution
Pacemaker uses the default features of Magento 2.
-
If you use modules that add additional columns to tables or add them yourself that Pacemaker uses, those columns should permanently get assigned a default value.
-
Importing data into tables with custom columns without a default value will always fail and lead to errors because Pacemaker does not know what type of data it is.
-
If a module adds columns without a default value, you must manually provide values for these columns to the import CSV file at any time.
Product import - ERROR: Can’t load product with SKU
|
|
Exception: Can't load product with SKU "..." in /var/www/<my.domain.xx>/releases/0.16.0/vendor/techdivision/import-product/src/Observers/GenericSkuEntityIdMappingObserver.php:117
Stack trace:
#0 /var/www/<my.domain.xx>/releases/0.16.0/vendor/techdivision/import-product/src/Observers/AbstractProductImportObserver.php(56): TechDivision\Import\Product\Observers\GenericSkuEntityIdMapping
Observer→process()Product import - ERROR: Cron Job indexer_update_all_views Integrity constraint Duplicate entry
|
|
- After the import the following error occurs in Magento:
Cron Job
indexer_update_all_views has an error: SQLSTATE[23000]: Integrity
constraint violation: 1062 Duplicate entry '980-2458-9-95258-980'(bsp) for
key 'catalog_product_index_eav_temp.PRIMARY', query was: INSERT INTO
`catalog_product_index_eav_temp` (`entity_id`,`attribute_id`,`store_id`,`value`,`source_id`)...Solution
-
To find the attribute, you can use the error message. The key is formed from all fields, meaning that the second value (in example 2458) is the attribute id
-
Finding the product can be done in the same way as for the attribute The product id is the
entity_id; this is the first value (in example 980)-
Check if the attribute
is_global=1, and the values are imported onStore_Viewlevel in the product import file -
Check if the attribute is a multiselect, and the exact value is imported several times
-
The example in the
additional_attributesis for the attributeattribute=2006-today|2006-today|2006-today
-
Attribute import - ERROR: PreLoadOptionId must be of type array
|
|
Uncaught TypeError: Argument 1 passed to TechDivision\Import\Attribute\Observers\PreLoadAttributeOptionIdObserver::preLoadOptionId() must be of the type array, bool given, called in /var/www/<my.domain.xx>/releases/0.16.0/vendor/techdivision/import-attribute/src/Observers/PreLoadAttributeOptionIdObserver.php on line 100 and defined in /var/www/<my.domain.xx>/releases/0.16.0/vendor/techdivision/import-attribute/src/Observers/PreLoadAttributeOptionIdObserver.php:123 Stack trace: #0 /var/www/<my.domain.xx>/releases/0.16.0/vendor/techdivision/import-attribute/src/Observers/PreLoadAttributeOptionIdObserver.php(100): # TechDivision\Import\Attribute\Observers\PreLoadAttributeOptionIdObserver->preLoadOptionId() # 1 /var/www/<my.domain.xx>/releases/0.16.0/vendor/techdivision/import-attribute/src/Observers/AbstractAttributeImportObserver.php(55): # TechDivision\Import\Attribute\Observers\PreLoadAttributeOptionIdObserver->process() #2 /var/www/<my.domain.xx>/releases/0.1Solution
- Check if the attribute options with the admin scope contain the same number of elements as in other scopes:
-
-
It can be easely get checked on the system in the
option-value-import.csvfile -
In that file, it must get checked if the attribute
admin_store_valueis present without value
-
|
The Attribute value |
- Check if there are attribute options with the attribute type
select/multiselect: -
-
To verify, the attributes must be identified first
-
- The easiest way to check is to extend the class
TechDivision\Import\Attribute\Observers\PreLoadAttributeOptionIdObserver classafter the following code section:
TechDivision\Import\Attribute\Observers\PreLoadAttributeOptionIdObserver class$attributeOption = $this->loadAttributeOptionByEntityTypeIdAndAttributeCodeAndStoreIdAndValue($entityTypeId, $attributeCode, $storeId, $value);- with the following lines of code:
if ($attributeOption === false || $attributeOption === true) {
$this->getSystemLogger()->critical($attributeCode);
}Therefore, after the following code (log call) is additionally inserted into the existing code, the attribute code and its associated errors can be logged for the affected pipeline step.
Sorting options controlled by the configurable_variations field, e.g., "Color", "Size", for configurables?
|
|
How can I sort for existing configurables, options that are controlled by the field configurable_variations,
e.g. color, size?
Solution
Add the field configurable_variations_position to the file product-import_<FILENAME>_<COUNTER>.csv.
With the new field configurable_variations_position it is possible to control the order of the
configurables options.
-
The values in field
configurable_variations_positionandconfigurable_variation_labelsmust have the same order
- check out the following example:
| … | configurable_variation_labels | configurable_variations_position | … |
|---|---|---|---|
… |
color=Color,size=Size |
color=1,size=2 |
… |