Data Upload

Owned by Ondřej Tesař
Last updated: Dec 04, 2024
5 min read

Using data uploads, you can send your own CSV / XLSX file (even zipped) into a specific partition.

Supported Entities

The following partition entities can be updated using data uploads: 

  • BoM Data

  • Company Parameters

  • Competition Data

  • Condition Records

  • Customer

  • Customer Extension

  • Data Source

  • Live Price Grid (LPG) – only overridable fields are supported for update

  • Live Price Grid Matrix

  • Manual Price List – only overridable fields are supported for update

  • Matrix Price List – only overridable fields are supported for update

  • Price Lists

  • Product

  • Product Extension

  • Product Reference

  • Seller

  • Seller Extension 

  • Users (to import users to a partition)

Note: 

  • During the Data Upload only the correct types are sent from PlatformManager to Pricefx, since PlatformManager can get the information on used types in Pricefx.

  • All simple types that can be defined on a partition and are not bound to specific partition config are supported in PlatformManager.

Support for Creating New Table

For the bellow listed entities you can create the corresponding table directly from PlatformManager during Data Upload, without having to log into the partition. This can be handy when you find out that the table you need is not on the target partition. This is a two-step process: first you set the new table general properties, then you can change field types (for CX, PX and SX).

New tables can be created for: 

Supported Formats & Requirements

PlatformManager supports the following file formats for upload: 

  • CSV
    Files which contains the BOM character are also supported.

  • XLSX if they meet the following criteria:

    • The file has only one sheet.

    • The data start in the first cell of the first column.

    • The first row is a header.

    • No macros are present in the file.

    • No formulas are present in the file.

    • Maximum size 20 MB.

All of these files can also be zipped. 

The preferred file encoding format is UTF-8; maximum file size is 512 MB.

Data Upload Types

There are two types of Data Upload: 

  • Regular Data Upload – After you select the file to upload, you need to define the parsing options, map the fields from your CSV / XLSX file to the Pricefx fields, manipulate the fields if needed, define time of the upload and various other options.
     For step-by-step instructions see How to Upload Data to Partition.

  • Quick Data Upload – Allows you to skip the parsing and field mapping step. The only required inputs are the CSV / XLSX file (can be zipped) and selecting the entity to update.

    • The condition is that your file is prepared to exactly match the target tables at the Pricefx partition. It also must use a comma as a delimiter and must contain a header.

    • There is validation: if the fields do not match, the upload ends up with an error message.

    • The file size limit is 2048 MB.

Data Uploads Overview

In the Data Uploads section you can see an overview of all existing data uploads.

DataUploads07.png

For each data upload you have the following options (available in the row menu):

  • Show history of the selected data upload.
    For failed records you can expand the line and view more details. The first 50 failed records are displayed here; if you want to get their full list, click the Export CSV button on the right. 
     Since version 1.52.0, information about failed records is saved to PlatformManager database (instead of ELK) and retrieved from there (to avoid calls to ELK and load the page faster). For older Data Uploads which had failed records info stored in ELK, this info is no longer visible in this section.

  • Send file – Upload a new file for an existing data load. It follows the same steps as when creating a new data upload.

    • It is possible here to lock the data upload configuration options and allow users only to upload a new file. To do that, assign the users the permission Partition Data Upload - use.

    • Only users with the permission Partition Data Upload - edit can modify data upload configuration options.

  • Copy the selected data upload to one or more partitions. It allows you to upload a new CSV / XLSX file while keeping all the mapping options of the original data upload.

  • SFTP User Management – Allows you to define a new SFTP user for each Data Upload. Then you can connect via this new user to the SFTP server, upload data (ZIP, CSV, XLSX) directly to the SFTP and this data are processed via the given Data Upload.

    • When you open the edit form, you will see the connected SFTP server. When adding a new user, you need to specify a name and authentication method for the user: password or SSH Public Key is mandatory.

    • You can also enter one or more email addresses of users who should be notified when the Data Upload is completed.

    • Each SFTP client may have its own settings to be able to upload and process a file via Data Upload successfully. For example:

      • WinSCP tries to set permissions for the uploaded file. Here is a solution.

      • When you upload a large file via WinSCP, it is moved with .filepart extension. Here is a solution.

  • Delete the selected data upload.

In addition, there is the Upload Queue section which shows you a list of all running/waiting Data Uploads to the partition and you can stop or cancel the upload there.

Validation

The following types of validation take place:

  • Before data upload – If the CSV / XLSX file has correct types String, Number, Date and DateTime. In case of an error, the file is not uploaded and the affected lines are displayed.


  • During data upload – If the business key length is within limit. You have the option to restrict the length of individual attributes used in the business key or reduce the number of used attributes.

PlatformManager version 1.75.0