FAQ & Troubleshooting – Part 1

Owned by Ondřej Tesař
Last updated: Oct 16, 2024 by Ioana Kocurova
32 min read

pfx-rest:post accepts only String type payload not InputStream

When using a marshaler, e.g. JSon, the output is InputStreamCache. Pfx-rest:post does not support streams and requires the payload to be String or byte[].

You can simply convert the body to String using following the line:

 

<convertBodyTo type="String"/>

 

After upgrade to IM 2.x you have to specify the date format

In IntegrationManager 2.x the Groovy Joda framework is deprecated. To get the current date time, use the simple language. You have to specify the ISO mask.

${date:now:yyyy-MM-dd'T'HH:mm:ssZ}

How to add row number to an exported file

Sometimes you need to add a row number to an exported file.

First you need to save the properties PfxRecordIndex (internal Pricefx Mapper property) and CamelSplitIndex in the first mapper. Usually you are inside a split.

<pfx:loadMapper id="IndexPriceCalculationLogicSimulation_TestingMapper"> <pfx:body in="Currency" out="TI_COPY_RECORDS_WAERS" /> <pfx:body in="Quantity" out="TI_COPY_RECORDS_KPEIN" /> <pfx:body in="UOM" out="COPY_RECORDS_KMEIN" /> <pfx:property in="PfxRecordIndex" out="FifcoPfxRecordIndex" /> <pfx:property in="CamelSplitIndex" out="FifcoCamelSplitIndex" /> </pfx:loadMapper>

Then you can calculate the row number in the second mapper. Just set includeUnmappedProperties=true to include all fields mapped in the first mapper.

You need to call api-model twice then. That's all.

Refresh Integration Information in PlatformManager

Sometime, we see information in PlatformManager which does not match existing IM on the server. To refresh the information:

  1. In PlatformManager, go to Account > Integrations > Monitoring tab.

  2. Turn off the 'Monitored' option and click 'Save'. 

  3. Wait for about 10 seconds. 

  4. Turn on the 'Monitored' option again and click 'Save'.

Integration Account on PROD Partition

Integration is a special account which IM uses to communicate with a PFX partition.

It should follow this convention:

Removing IntegrationManager

If IM is no longer used, we need to remove it from the servers and release resources that it occupies.

To remove IM:

  • Stop the service.

  • Remove all installed files from the folders.

  • Remove the linked file in /etc/init.d.

  • Remove the Jenkins jobs.

  • Remove the IM configuration from PlatformManager.

Using Special Delimiter Characters

Example usage of Hex 14 for inbound and Hex 0A for outbound (core 1.1.18.3):

Sorting by DESC Order in Filter

To sort the result set in a descending order, put the following before the field to sort by:

Duplicated Rows in Result Set When Fetching Data from PFX Table in Batch Mode

In some cases, there are duplicated rows in the result set when fetching data from a PFX table in the batch mode. If this happens, check if there is the sortBy attribute added in the filter for fetching.

HTTPS Protocol

The latest IM version (1.1.16.5) uses only the HTTPS protocol for monitoring by PlatformManager.

The previous versions use HTTP.

Invalid Command Parameters Error

This error "org.hibernate.exception.SQLGrammarException: could not execute statement" is raised when IM uses an old API pfx:dsLoad to load data into a PX table. 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

One of the reasons for this error is a missing parameter businessKeysMaxLengths. This parameter was not mandatory before but now it is. 

It is good to switch to the new API pfx-api:loaddata but here businessKeysMaxLengths is mandatory.

Error 404 displays on Logfile page in PlatformManager

When register a new IM instance in PlatformManager and you get an error as shown bellow, add this line into the IM properties file (if it is not there yet):

logging.file=main.log

Missing roles when calling refreshCustomerDS

When you encounter the following error:

Try adding these roles to the integration account:

header.PfxTotalInputRecordsCount is empty when using split loading

To fix it:

Using delimiter \t in inbound CSV files

The delimiter \t can cause errors if the core 1.1.16 is used. To avoid this error, use 1.1.16.1

It is caused by:

How to prevent pfx:csv-to-list to double each line when it ends with cr and lf (\r\n)

Request the file owner to fix ends of lines from "\r\n" to "\n".

How to call (from IM) dataload with Type=Calculation and TargetType=DMDS

Not authorized for command: class net.pricefx.server.commands.configurationmanager.Set

Got this error:

You need to check the role in PFX UI: General Admin (without User Management)

If you use fetch/AP or update/AP, you should be ok just with the Data Integration role.

How to skip records which cannot be parsed during import

Q: I have to import some old exports with no way of correcting the data and I need to skip records which cannot be parsed (mapper throws error). Is there a simple way to do that?

A: It is supported since the version 1.1.16.

Is there a way to lookup DF name based on label?

How to set "exclude hidden attributes" in export to XLS files in IM code

We can get metadata of the table, it may have the information about hidden attributes of a column.

In case of a price list, we can get and see this log:

How to update an approved PLI

You don't need key2. 
getAttributeName gets the attributeXX for the name of the element. 

How to track changes in a PX table

The audit log works only for a PP table. Change tracking in PX, CX tables is not supported.

We could filter changed rows by LastUpdatedDate > last_check_date condition and last_check_date could be stored in a PP table.

Data Integration role and changing an approved quote via API

In PFX 3.6.2 anyone can change anything as it used to be. In 3.6.3 this will be restricted only to users with the dataintegration role. 

Triggering PADATALOAD_COMPLETED

Q: When/how can we catch an event that fired after Datamart refresh?

A: IM code and/or PFX UI button clicked event handling creates JOBS for Datamart refresh. The scheduler (of PFX) will schedule execution of the job according to the available resources. You will receive an event when the jobs finish successfully and this is reason why event has the name XXXX_COMPLETED.

Could not locate PropertySource: I/O error on GET request for http://localhost:8888/application

Set the configuration value:

net.pricefx.integration.api.NonRecoverableException: Not authorized for command: net.pricefx.domain.EventTask

The problem is access rights of the user – your user cannot read event data from PFX. jsonWebToken should have no impact to this. 

applicationContext is null in PfxSalesforceProducer class

Try to add into your properties file:

Data fields in IM UI configuration

  • Instance Management URL – Address where your IM listens to requests from IM UI.
    Example : http://int1.eu.pricef.eu:8080
    Address – Your IM server address and port. Port can be set by server.port property

  • Instance Management Username – Set by security.user.name

  • Password – Set by security.user.password property in your IM config file

Can I start calculation of CFS and pass parameters to the underlying calculation?

As of 28/02/2019, this feature is not available.

https://qa.pricefx.eu/pricefx-api/json/develop.html#!/cfs/calculate

How to fetch data from Datamart and iterate over it and update some values

The correct API is:

Java:

How to get property within Groovy in Camel

As in

 

should be

Example of route calling Datamarts refresh

How to track value changes in a PP table

We need to audit the changes in price parameters (PP) tables.

  • Load the whole PP at the start and keep it in memory (if it's not too big).

  • Data Change Request (DCR) can be also solution, we use it on Bosch. 

    SE can set up DCR.

How to access Advanced Configuration in Pricefx

Who is responsible for archiving/cleaning data in the outbound folder?

It should be the customer. The main reason is that we don't know exactly if the files we sent (outbound) are processed at their side or not. For instance, if we clean-up/archive a file but the customer did not pick it up yet, it will be a "data missing" issue. And we will give the client the file delete permission.

However, it's flexible. For example, in SENETIC, IM does the clean-up/archive job. We asked the customer to move the outbound files to another place after they processed them. After that, IM archives the outbound files at 12:00 AM every night.

Examples of projects using REST client/server

  • REST server example: project firthnz
    file outbound/quoteRoutes.xml

  • REST server example: project Haefele
    file net.pricefx.integration.processor.PfxBasicAuthProcessor

How to avoid processing files that have not been fully transferred yet?

Use one of the following:

  • Done file or other file parameter

  • Bean - net.pricefx.integration.filter.FileModifiedFilter

    ​In URL use the filter parameter:

    In camelcontext:

  • Readlock option is only for SFTP component (on the FILE component on Linux it does not work)

How to get notified when e.g. quote is deleted?

You can use the ITEM_UPDATE_Q event type, in deletion, its 'operation' is 'DELETE'.

How to change log levels on your project in runtime

Note that it will create more data in log files / send more data to ELK and it could slow down the application.  

Splitting list of messages into 5000 messages per chunk

Q: How to split a list of messages (body) into e.g. 5000 messages per chunk by a standard Camel component? I would like to export CSV files containing no more than 5000 rows into two different locations (SFTP, audit)?

A: On Dana we can see usage of:

DsUniqueName in pfx-api:loaddata

Q: For the pfx-api:loaddata component, there must be DsUniqueName specified when loading objectType=DM? dataSourceName is not sufficient?

A: Yes there must be DsUniqueName. It works same way as the dmLoad object.

Events exist in PFX but are not fetched to IM (missing events)

PFX partition uses the PULL event model, so you need to check that the Advanced Configuration Option disabledEventProcessing.

Notification when a quote is deleted

Q: Is there a way to get notified when e.g. a quote is deleted? Is it possible to trigger some code to generate a custom event or something like this?

A: You can use the ITEM_UPDATE_Q event type, in deletion, its 'operation' is 'DELETE'.

Different handling of a property substitution in the property file

You could run into issues when IM starts correctly but during processing throws an exception like this:

Spring resolves the substitution using ${ } as a property placeholder. The ​format {{ }} uses Camel when it resolves endpoints URI.

Example

Magic properties for an instance of the PFX client

You do not need to instantiate the PFX client bean in your camel-context.xml. It is enough to have properly named properties for PFX URL.

For details see PfxClient. 

Example

Incoming REST request implementation

Q: Do you have an implementation of an incoming call of a REST service to IM?

A: Yes, see the example below.

Route

Calling a Camel processor in a separate transaction

Q: Can I call a Camel processor in its own transaction?

A: See a Camel example here: https://github.com/apache/camel/blob/master/components/camel-spring/src/test/java/org/apache/camel/spring/interceptor/MixedTransactionPropagationTest.java

You have to call the processor in a new route.

Camel Processor Snippet

LoadData with businessKeys with more than 5 columns

Q: There is an error in dsLoad when the businessKeys attribute has more than 5 columns.

Error message

A: Since Pricefx 3.3 Cosmopolitan, you can use an optional request param to adjust the length of columns defined in the businessKeys attribute. For more details see  https://pricefx.atlassian.net/browse/PFCD-3594

Removing old data automatically

To remove old data automatically, you can use backup routes. 

Connecting to SAP using WSDL

The old way to connect was to use RFC (direct connection to SAP) and this is no longer recommended. 

New projects use only web services where we don't call directly SAP ERP but we call SAP PI and it calls SAP ERP. 

We also export Idocs to SFTP; customers SAP PI processes such data and again calls SAP. 

In SAP, they should be able to wrap SAP RFC with web services as well.

Why Q_START_DATE field format changes?

Q: Any suggestions why the Q_START_DATE field format changes?

A: Check the original value in the CSV file and the convertor for the date type. 

Convertor example

IntegrationManager version 5.8.0