IntegrationManager 1.1.18

This document summarizes major improvements and fixes introduced in the Pricefx IntegrationManager release version.
VersionIntegrationManager 1.1.18
Release dateApr 3, 2020

In this section:

Migration Steps

HTTPS

Since version 1.1.18 IM endpoints have been secured by SSL (HTTPS). If an IM instance is connected to PlatformManager, its management URL (in PlatformManager) has to be updated to HTTPS. For example:

Integration Project Changes

  1. Make sure you have Java OpenJDK 11 installed. Oracle JDK is not allowed due to a commercial license.

Maven Project File Changes (pom.xml)

  1. Change the property version.Java:
    <version.Java>11</version.Java>
  2. Change pricefx-im-version:
    <pricefx-im-version>1.1.18</pricefx-im-version>
  3. Add the property spring-boot.version:
    <spring-boot.version>2.1.5.RELEASE</spring-boot.version>
  4. Use the property spring-boot.version in the plugin definition of spring-boot-maven:
    <plugin> 
    <groupId>org.springframework.boot</groupId> 
    <artifactId>spring-boot-maven-plugin</artifactId> 
    <version>${spring-boot.version}</version>  

Property Files Changes (application.properties)

  1. Remove the spring.main.sources property.
  2. Apply the prefix spring. for the following list properties security.user.name and security.user.password.
  3. Change the names of the properties containing routeErrorHandling to route-error-handling.
  4. Add the integration.mappers.default-converter-strategy-type=MANUAL property.
  5. All error handling properties are in the camelBack format instead of the dash format.

    integration.route-error-handling.enabled=true
    integration.route-error-handling.enableSendingMail=true
    integration.route-error-handling.maximumRedeliveries=10
    integration.route-error-handling.maximumRedeliveryDelay=30000
    integration.route-error-handling.redeliveryDelay=2000
    integration.route-error-handling.retryAttemptedLogLevel=error
    integration.route-error-handling.backOffMultiplier=2
    integration.route-error-handling.logHandled=true
    integration.route-error-handling.logRetryAttempted=true
    integration.route-error-handling.logStackTrace=true
    integration.route-error-handling.useExponentialBackOff=true
    integration.route-error-handling.retry.httpStatus=409
    integration.route-error-handling.retry.exceptions=com.sun.jersey.api.client.ClientHandlerException
    
    integration.route-error-handling.email.smtpHost=${smtp.host}
    integration.route-error-handling.email.smtpPort=${smtp.port}
    integration.route-error-handling.email.smtpAuth=${smtp.auth}
    integration.route-error-handling.email.smtpAuthUsername=${smtp.auth.username}
    integration.route-error-handling.email.smtpAuthPassword=${smtp.auth.password}
    integration.route-error-handling.email.mailFrom=integration_customer@pricefx.eu
    #integration.route-error-handling.email.mailTo=integration@pricefx.eu
    integration.route-error-handling.email.mailTo=integration@pricefx.eu
    integration.route-error-handling.email.mailSubject=Camel processing error

You can validate your application properties file against spring-boot_2.1.5.RELEASE_common-application-properties.

Camel Context Definition Changes (camel-context.xml)

  1. In the camel-context.xml remove the following line:
    <import resource="classpath:routeErrorHandling.xml"/>

Jenkins Changes

You need to change JDK in the Jenkins job to "OpenJDK11".

Also ask the Operations team to modify the deploy scripts to use OpenJDK11.

Possible Issues

java.lang.NoSuchMethodError: 'java.lang.String javax.servlet.ServletContext.getVirtualServerName()"

Remove all unnecessary dependencies. They can cause an exception during startup.

<dependencies>
	<dependency>
		<groupId>net.pricefx.integration</groupId>
		<artifactId>integration-manager-starter</artifactId>
	</dependency>
</dependencies>

IM fails to start with an exception "password has been cleared"

This is likely caused by the Bouncy Castle issue. Some of your dependencies changed the order of loading classes. The issue and workaround is described here:

PFIMCORE-529 - Getting issue details... STATUS

Supported Components

  • Java 11
  • Camel 2.24.0
  • Databases
    • Oracle
    • MySQL
    • Derby
    • PostgreSQL

Improvements

New Feature DescriptionID
When a user sends a wrong payload (invalid XML), it is newly saved at a specified directory and available for further analysis.PFIMCORE-84
There is a new pfx-excel component which has the following methods: unmarshal (converts Excel data into an array of map) and marshal (converts an array of map into the Excel format).PFIMCORE-280
IntegrationManager is now ready to integrate with the Snowflake data warehouse. PFIMCORE-317
There is now the IntegrationManager HTTP client available which allows you to easily call IntegrationManager endpoints.PFIMCORE-320
Support for Apache Freemarker has been added to avoid version conflicts.PFIMCORE-326
The StringToDecimal converter can now set precision and rounding mode.PFIMCORE-331
It is possible to send data to a Data Source directly: for the pfx-api:loaddata component use the option direct2ds: true.

PFIMCORE-336
The BigDecimal to String converter has been added.PFIMCORE-337
As IntegrationManager runs Groovy scripts that could contain malicious code, the Groovy scripts are now evaluated inside a sandbox that prevents malicious code. If the sandbox detects a forbidden code, it throws an exception of the type net.pricefx.integration.groovy.GroovyEvaluationException.PFIMCORE-344
The pfx-api component now supports PYR (Payout Records) as an object for the fetch operation.PFIMCORE-350
There is a new component pfx-odata2 which is used as a connector to  systems that implement OpenData v2 API (such as SAP S/4HANA).PFIMCORE-354
PFIMCORE-287
Communication between IntegrationManager and PlatformManager is now encrypted.PFIMCORE-380
An index number has been added to records inside a batch while loading data.PFIMCORE-385
The possibility to send data to a Data Source directly has also been added for Datamart web service.PFIMCORE-388
It is possible now to set up your own error email template. The email can also contain names of the invalid files to help you with troubleshooting.PFIMCORE-389
PFIMCORE-305
There is a new Camel component pfx-gp which can load data into GreenplumDB on a remote server leveraging the gpload binary.PFIMCORE-396
There is a new web service for fetching rebate records.PFIMCORE-407
There is a new web service for fetching payout records.PFIMCORE-409
It is now possible to remove a deployed property via REST API (to be able delete a property which no longer exists in PlatformManager).PFIMCORE-421
It is now possible to remove a deployed connection via REST API (to be able delete a connection which no longer exists in PlatformManager).PFIMCORE-422
There is a new pfx-rest component which can be used to connect to any REST API. The component deals mainly with accessing the resource and providing raw data without any transformations. The supported authentication methods are: OAuth2, JWT, Basic or no authentication.PFIMCORE-424
The configuration for PlatformManager environments "qa" and "prod" has been improved. There is now a new parameter integration.platform.environment=prod which can easily be changed to "qa".

PFIMCORE-426
There is now a size limit for data to be sent to an to invalid records index.PFIMCORE-429
For the pfx-api component there is a new method "update" based on typedId.PFIMCORE-434

Fixed Issues

Bug DescriptionID
It is not possible to define a tab delimiter \t because the delimiter component property supports only characters.PFIMCORE-288
Cache for Price Parameter is used but if IntegrationManager is connected to multiple partitions this caching can cause issues (the same names on multiple partitions). Therefore there is a new parameter to enable or disable Price Parameter IDs caching: integration.pricing-parameters.use-cache=falsePFIMCORE-349
In pfxCommonTypes.xsd, some elements use a misspelled tag "xds" instead of "xsd".PFIMCORE-365
The parameter dtoFilter for pfx-api:truncate is invalid when validating a route against XSD on version 1.1.16.2.PFIMCORE-376
The cluster name is sometimes not recognized based on the pfx.partition parameter URL.PFIMCORE-401

IntegrationManager version 5.0.0