IE Useful Information and Components

Notes and examples on usage of various Pricefx Camel components related to Integration Engineering.

Reference

Integration Route Considerations

Whenever possible, use the standard, unmodified Platform Manager (Pfm) Templates to enable integration with Pricefx.
When an integration challenge seems to exceed the capability of the provided Pfm Templates, consider using a simple Template and managing the post processing of the uploaded / downloaded data through use of a process such as Calculated Field Set (CFS) or Calculation Dataload to make the requisite data changes.
When neither step 1 or 2 seem to be acceptable, consider whether the layout of the Pricefx tables should be adjusted to accommodate the required processing.
When none of the above processes seem to be acceptable, consider modification to he Template generated Camel Routes.

pfx-api Component

Reference https://knowledge.pricefx.com/space/IEKB/3879436488/pfx-api

This is a central Pricefx specific Camel component with a variety of options to enable loading and extraction of data associated with Pricefx partitions.

Terms

Serialize to byte stream - Byte stream is just a stream of binary data
Deserialize to original object - extracting a data structure from a series of bytes

Camel marshal - Serialize the body - as in create an xml file from a Java Object
Camel unmarshal / Deserialize the body - as in create a Java Object from an inbound XML file

from("file:inbox/xml") .

unmarshal().jaxb() .to("bean:validateOrder") .

marshal().jaxb() .to("jms:queue:order");

Camel split EIP Component

See https://camel.apache.org/components/3.20.x/eips/split-eip.html

The Splitter component is used to split a message body into multiple elements. Consider an order which has multiple lines. Each line may require special processing while being processed through the integration Route. The Splitter EIP provides a means to break the order into multiple separate lines for individual processing. Generally, the Splitter EIP breaks out a composite message into a series of individual messages, each containing data related to one item.

Note:

The Split EIP will by default split the message body based on the value type:

java.util.Collection - splits by each element from the collection/list/set.
java.util.Map - splits by each Map.Entry from the map.
Object[] - splits the array by each element
Iterator - splits by the iterator
Iterable - splits by the iterable
org.w3c.dom.NodeList - splits the XML document by each element from the list
String - splits the string value by comma as separator

Example:

<choice>
    <when>
        <groovy>
            headers.objectType == 'DMDS'
        </groovy>
        <log message="Using import"/>
        <toD uri="pfx-api:import?${headers.parsedPfxApiSettings}{{pfx:import-product-perf-csv-from-ftp.pfx-csv.settings}}&amp;mapper={{pfx:import-product-perf-csv-from-ftp.import.csv.from.ftp.mapper}}&amp;connection={{pfx:import-product-perf-csv-from-ftp.pfx-connection}}"/>
        .
        .
        .
        <split streaming="true">
                    <tokenize token="\n" group="5000"/>
                    .
                    .

In the above example, the pfx-api component with the import option is used to load a message body consisting of multiple lines. The <split component is then used to break the message body into individual lines using a newline character ( \n ) as the basis for the line break.

In the case of the <split above, the group option is used to group N parts together. In this case groups of 5000 lines are grouped together to assist in processing large input files.

Note that, by default, split will keep all sub message that are split in memory. On large files, this can prove troublesome. The alternate is to use the streaming mode of split to avoid sub messages being maintained in memory.

Example

<route>
    <from uri="direct:teststream"/>
    <split streaming="true">
        <tokenize token="|"/>
        <to uri="activemq:streamed-parts"/>
    </split>
</route>

pfx-api unmarshal EIP Component

See https://camel.apache.org/components/3.20.x/eips/unmarshal-eip.html

Camel unmarshal / Deserialize of the body

<split streaming="true">
    <tokenize token="\n" group="5000"/>
    <toD uri="pfx-csv:unmarshal?{{pfx:import-product-perf-csv-from-ftp.pfx-csv.settings}}&amp;skipHeaderRecord={{pfx:import-product-perf-csv-from-ftp.skip.header}}"/>
    <choice>
        <when>

In the above example, the Camel body is split based on newline ( \n) characters and then rendered into the data structure that was originally placed into the message ( object, text, etc. )

pfx-api loaddata EIP Component

This is a Pricefx specific component which is used primarily to bulk load data into Pricefx tables. By bulk load we are referring to cases where the volume of data exceeds about 7500 rows.

pfx-api integrate EIP Component

This is a Pricefx specific component which is used primarily to load relatively small amounts of data into Pricefx tables. Note that integrate is restricted to MasterData tables only.

pfx-api fetch EIP Component

See https://pricefx.atlassian.net/wiki/spaces/IM/pages/5867143667

This Pricefx specific component is used to retrieve data from Pricefx tables. Since many tables may contain a large volume of data, this component supports a process known as batching to retrieve a smaller portion of the required data. The component then allows the developer to iterate over and retrieve the remaining data using a special fetchiterator component.

When pfx-api:fetch batching is used to retrieve batches of records, the Pricefx component pfx-api:fetchiterator is used to iterate over the batches of retrieved records.

Sample code using api:fetchIterator

Camel multicast EIP Component

See https://camel.apache.org/components/3.20.x/eips/multicast-eip.html

The Multicast EIP allows a message to be routed to a number of different endpoints and potentially processed in a different way.

Example multicast message to same Route :

Example Multicast message to different Route :

Camel filter EIP Component

See https://camel.apache.org/components/3.20.x/eips/filter-eip.html

This component exists in both standard Camel and as an option on a number of Pricefx specific Camel components. This component allows definition of a subset of data to be processed.

Note that while the filter may be constructed as in the example below, the filter may also exist in the Pricefx filters folder of the standard Pricefx Template Route. In the case of the filter(s) being defined in the filters folder, the end user would have used the Platform Manager Filters option on the Integration instance:

Filter defined in Platform Manager

Filter defined in Route

Sample use of Filter

pfx-api loadmapper EIP Component

This Pricefx component option is used to map either the inbound object fields to corresponding fields on the Pricefx table or map Pricefx table fields to element names in the outbound file.

This mapping may be defined either inside the Route or may be defined in a file under the mappers folder of the standard Pricefx Template project Route.

Example Mapper defined as part of the Route

Example Mapper defined in Mapper folder of Project

Sample usage of the Mapper

The mapper loadmapper definition is referenced by the mapper option on the Pricefx component in use.

Note that when the mapper definition is coming form the Integration Route mappers folder created by a Template, the definitions and files in this folder will have been automatically created by Platform Manager.

Camel wiretap EIP Component

See https://camel.apache.org/components/3.20.x/eips/wireTap-eip.html

One main purpose of WireTap is for logging or auditing a message outside of main thread while not slowing the main thread.

Difference between multicast and wiretap is:

WireTap is "one-way fork". Message goes to wire-tapped endpoint in different thread parallel to main thread and there is no way to get response back from it to main thread.
With Multicast it is possible to "split-then-aggregate" results in main thread after parallel or sequential processing.
Also multicast is not limited as regards how many endpoints will be multicasted, while WireTap can have only one endpoint to send to.

Sample wiretap code in Route

Camel aggregate EIP Component

See https://camel.apache.org/components/3.20.x/eips/aggregate-eip.html

The purpose of the aggregator EIP is to allow the combining of multiple messages into one single message.

One typical usage of this pattern is to collect and store individual messages until a complete set of related messages has been received. Then, the Aggregator publishes a single message distilled from the individual messages.

The aggregator EIP has numerous features and options and may be implemented in numerous ways.

Example:

Assume following message body being directed to Camel endpoint simpleexample

Assume following Route :

Assume following Aggregations c;lass :

Expected Result :

LasgnaisHeavenly

Camel processor interface

See https://camel.apache.org/manual/processor.html

A Camel Processor is an interface implementation which allows you to manipulate the Camel Exchange. A processor implementation is capable of taking an inbound message and modifying the Camel body, header and properties. Note that using a Camel bean can often achieve the same results.

Example - Route to use UpperBodyProcessor class intercept body and convert body to Upper Case

Example - UpperBodyProcessor

Example - Sort body of Exchange on a specific attribute

Introduction to Pricefx

IE Useful Information and Components

Analytics

Related content