Import CSV File (1.2.0 and higher)

Open

Steps:

1. Read the CSV file from the defined URI:

   <from id="from1" uri="file:{{cars-fromUri}}"/>

    

2. Split the records to batches to increase efficiency of the loading. In this particular example, there is grouping into batches of 5000 records. 
   There is an assumption that the records are separated by the new line character.

   <split id="by_rows_grouped"> <tokenize group="5000" token="\n"/>

    

3. Unmarshal each batch using the defined header with parameters depending on formatting of the particular CSV. 

   <to id="to1" uri="pfx-csv:unmarshal?header=sku,label,price,date,year&skipHeaderRecord=true&delimiter=,"eCharacter=~"/>

    

4. The unmarshaled (POJO) batch is loaded into Pricefx by pfx-api:loaddata, using a custom mapper. In this particular case:

    

5. Flush the loaded data from the Data Feed to a Data Source on completion. 

6. Flush the completed data by the property: integration.events.event-to-route-mapping.PADATALOAD_COMPLETED=direct:eventPADataLoadCompleted

7. Truncate DMF.Product. 

    

Open

Example of a Camel route intended for CSV import using pfx-api:

routes/carRoutes.xml

Example of the imported file:

Example definition of a mapper used in the above Camel route: 

mappers-filters.xml

Sample route for treating events:

eventRoutes.xml

Example of using both the above Camel resources in a Camel root context:

camel-context.xml

Data Source import table:

DS import tabel definition