FAQ & Troubleshooting - Part 2

Multiple PFX connections in one IM

Q: I need to process an event in one IM and save data into another partition / message other IM.

A: You can connect from one IM to many partitions and call API. You need to set partitionPfxApi. As a reference, check out the Toys "R" Us project.

This code will call the tru-ce PFX connection.

So you can load a list of partition names and just iterate through them with split and set up this header before store logic. 

Backing up whole directory structure

Q: I would expect that the whole directory structure is backed up under integration.backup.source but it seems that it only accepts files. Is it possible to backup the whole directory tree?

A: Version 1.1.1 does not have support for nested directories. In the next version there will be support for WS and a list of directories. If WS is enabled, backup creates a backup route for all WS automatically.

Getting dashboard data

Q: How can IM fetch calculation result from a dashboard?

A: priceFxClient.getFormulaApi().executeformula("DashboardName", new FormulaExecuteRequest());

Response sample

Architecture implementation – document sample

For a description how to implement architecture on a project see the /wiki/spaces/CLI/pages/83656777.  

Solenis uses SAP integration.

IOException occurs when quoteChar (" by default) is inside field value

Q: How do we solve a problem with double quotes (" ") inside a cell value which encapsulated by a quote (")?

A: The solution is to escape all the double-quote characters inside the field value or disable the quote mode to make IM accept the quote " as a part of the string.

DisabledQuote mode

Support for fetch to core web services

Since the version 1.1.3 the following entities can be fetched: P, PX, PPV, C, CX and Damatart (Datamart, DataSource, Data Feed).

Using header values in Groovy expression in loadMapper

Q: Is it possible to use the header values in a Groovy expression in loadMapper? For example:

A: Try one of these options:

• headers.loadDateCompetitors

• The following code sample:

Using from(file).to(file) to transfer large files

Q: I have an issue with from(file).to(file) used to transfer large files. Apparently, this method tries to load the whole file into the memory (or at least some parts) and it results in the "OutOfMemoryError: Requested array size exceeds VM limit" error.

A: Use .split(body().tokenize("\n")).streaming() instead.

A2: You can use direct linux command (example for scp to sftp):

Adding templateName param to quote > fetchpdf API

Q: How can I add the templateName parameter to quote > fetchpdf API? I need to select a template for PDF export but I cannot find any API supporting this.

A: There are the following options:

• Use invokeAPI:

• Or as a workaround, take the fetchPDF method from pricefx-api, modify it and use in your processor. 

• Or ask the IM team to add this method into IM.

Dynamic mapping CSV headers to PX fields – NOT SUPPORTED

Q: Is there a way to dynamically map CSV headers to PX fields? I.e. a header with the name "attribute1" will get mapped to attr1 in PX, regardles of the position in the CSV.

A: No, this is not supported. you can submit a Jira ticket for the IM team to implement it. 

A2: Implemented in Schneider project: DynamicHeaderCvsParserProcessor

Switch in Camel

Errror message 'sun.security.validator.ValidatorException: PKIX path building failed'

Q: We are getting the below error when we try to send data to the Pricefx QA server.

Error Message:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Is this a problem with the certificate? I tried to upload data and it worked fine. 

A: It is about a new certificate. We have to import new one. Contact the Marketing team (responsible for communication with customers) and let them inform the customers.

For your reference only, you can download a new certificate (in Chrome) and import it with keytool. If the customer uses our web services, they need to update the certificate of our HTTP proxy.

Invalid command parameters: Lock wait timeout exceeded

Q: How can I fix the following error which I got when trying to load data to a PX table.

A: I would avoid a parallel loaddata to the same target table, although feeding different (physical) tables in parallel is probably completely ok. 

The change at the Solenis project was twofold:

1. A DB level index that matches the business key must exist (someone must create it manually). It probably applies only to PX/CX since all other types have fixed keys (and hence indexes exist).

2. The fix I committed: When doing loaddata based on join keys, PFX was generating a join condition which caused in some cases the DB planner not to pick the right DB index. We changed it and now a simpler query should be generated (with some DB specific things).  Now the planner can always select (and use) the right index. (This fix is in the master and will be part of the next hot patch, probably 3.3.6). 

According to my tests, Oracle based instances behaved the same way as MariaDB instances. 

On Solenis, loaddata took seconds to minutes, now it takes 150 ms per request.

Using PIE for JMS consuming

Q: Does anybody use PIE for JMS consuming as in MS?

A: Bosch uses PIE with JMS and ActiveMQ.

Configuring number of consumers on PIE

If there is only one customer, you will see this:

Pure PIE JMS consumer:

Inserted product does not display

Q: I have an issue with: <pfx:dsLoad id="loadProduct" objectType="P" businessKeys="sku" mapper="productMapper"/>

The CSV contains 2 updates and 1 insert, update works fine, but the insert does not go through. Any ideas?

A: Avoid caching by changing browser:

• Check the filter on the user perspective.

• Check the filter on preferences:

  Open

  

   

businessKeys for PX

Q: If I have businessKeys for PX defined in PFX, is it enough to commit them in integration to work?

A: Yes, if you use detectJoinFields = true. 

Running DM Calculation after DM Refresh

Q: I need to run DM Calculation after DM Refresh. In IM, how can I make sure DM Calculation is called when DM Refresh is done? Or do I need to call dmCalculation because it includes DM Refresh?

A: Use an event system. 

You can add "status":"READY".

Workaround for large files and convertBodyTo

Q: What is the workaround for large files and <convertBodyTo type="java.lang.String" charset="UTF-8"/>? This is just CSV processing, but the file is from the FTP and there is a comment in the source: FTP files currently need this in order to do split + tokenize.

A: Just set a temp file on the FTP component and you do not need to use convertBodyTo. The trick is that it will download a file and you can use the split immediately. The convertBodyTo line was used to "download the file" to be able do the split. It looks like there originally was a bug in Camel which was fixed later on. Now you can just use the local temp file which will do the trick.

What is <bean id="PricesDSExtractor" parent="pExtractor" p:constantKey="${Prices_DM_DSname}" />?

It is a PIE component, Parameter Extractor, it sets contantKey to the header for the next PIE component processing, it sets a DS name in exchange and it is used in connection with the PIE component. 

In new PFX components (not released yet) you would just do <pfx:fetch?dsUniqueName=Prices&filter=dsFilter/>. 

How to trigger CFS after PP change?

There is a new event to do that.

OutOfMemory error

Q: Does anyone have experience with the error "Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded"? It happened with [net.pricefx.integration.command.pl.FetchPriceListItems] when the pricelist has more than 500K items.

A: You have to batch it. 

Fetch it is used as your filter. outputUri is the output URI.

Scanner aborted because of IOException

Q: Any idea what to do with this?

A: Check if you have:

• Correct format of CSV and none of the columns is missing.

• The same number of columns for for each line.

• Correct encoding.