IntegrationManager 5.0.0

This version has been released only for testing in dedicated projects. Once it is fully validated, it will become available for general use.

This document summarizes major improvements and fixes introduced in the Pricefx IntegrationManager release version.

Release date

Dec 4, 2023

Camel version

4.1.0

Spring boot version

3.1.5

Last Pricefx Swagger API update

Feb 3, 2023

In this document:

Migration Steps

This version introduces major upgrades to the used components. This comes with breaking changes. The largest updates relate to the following components:

  • Camel 4.1.0

  • Spring Boot 3.1.5

  • Java 17

  • Groovy 4.0.15

It is strongly advised to read carefully the official migration guide from Camel. Start with the version 3.11.0 (https://camel.apache.org/manual/camel-3x-upgrade-guide.html), through https://camel.apache.org/manual/camel-4-migration-guide.html and up to 4.1.0 https://camel.apache.org/manual/camel-4x-upgrade-guide.html.

Complete List of Breaking Changes

Covered by Upgrade Tool

Breaking Change Description

Mitigation

Covered by Upgrade Tool

Breaking Change Description

Mitigation

N/A

Java 17 is required.

For local development, you must install Java 17 JDK.

No action needed for provisioned instances.

For manual instances, Java 17 must be be installed on the servers.

N/A

javax was migrated to jakarta.

If you use xml libraries, you are required to update to jakarta counterparts. Otherwise XML functionality from 3rd party libraries will not work.

Also, migrate imports from javax to jakarta for any custom components.

N/A

Groovy was updated to 4.0.15.

Groovy scripts that use deprecated or non-standard API might fail. Check all Groovy scripts for most recent syntax.

N/A

httpclient dropped in favor of httpclient5.

If any of your custom components rely on httpclient, you must rewrite them using httpclient5 imports.

N/A

Mailing support has been migrated from commons-mail to spring-mail because commons-mail does not support Java 17.

If custom components use commons-mail, the logic must be rewritten using spring-mail.

N/A

https://pricefx.atlassian.net/wiki/spaces/INTG/pages/4749951003 is now temporarily disabled. The reason is that Spring removed the jolokia endpoint which does not yet support Servlet API 5 https://github.com/jolokia/jolokia/issues/568. Estimated duration of this measure is 6-12 months.

N/A

N/A

Camel now reports Health: DOWN when Camel is being stopped, during the graceful shutdown process. This can result in IM showing DOWN.

If you use multiple Camel contexts and stop them in runtime, IM might become unresponsive. You should use only one Camel context and should not change its state.

N/A

Default TLS protocol has been changed from TLSv1.2 to TLSv1.3.

Review your existing integrations with old systems. If necessary, you should lower the protocol back to TLSv1.2.

N/A

When referencing dataSource in components, you can no longer use standard dataSource=myDataSource. You need to specify it as: dataSource=#myDataSource

Example of usage:

sql:insert into cars (sku,price) values (:#sku,:#price) on duplicate key update price=??batch=true&dataSource=#dataSource

N/A

Route statistics were removed from Camel API. A helper method RouteCheckerService.extractRouteInfoFromMBean can be used to retrieve route statistics.

If you need to get route details, use API GET /api/routes/details which provides the required information.

N/A

https://camel.apache.org/manual/stream-caching.html is now enabled by default. That means that if streaming methods are used, the content is always downloaded to a temporary file, from which it is streamed and can be streamed multiple times.

The type of the body changes – if you previously relied on InputStream, you should change the type to StreamCache.

N/A

camel-rest: The deprecated option componentName was removed. Use either producerComponentName or consumerComponentName.

For more information see https://camel.apache.org/components/4.0.x/rest-component.html. The most often picked is producerComponentName.

N/A

REST DSL no longer allows inline route into REST service definition (you have to set <to uri="direct:xxx"/> and define the route <route><from uri="direct:xxx"/>...</route> elsewhere).

Examples are in REST Module | Examples.

N/A

camel-caffeine-lrucache was removed. At places where it was used, we used Camel default cache.

No action needed. If performance issues occur, contact support.

pom.xml of provisioned instances must have updated versions: Java 17, Spring Boot 3.1.5 and spring-cloud-starter-boostrap 4.0.4.

This is covered by the upgrade tool, so the fix is applied automatically.

Following elements has been changed:

 

Old value

New value

strategyMethodName

aggregationStrategyMethodName

strategyMethodAllowNull

aggregationStrategyMethodAllowNull

aggregationRepositoryRef

aggregationRepository

aggregationControllerRef

aggregationController

executorServiceRef

executorService

timeoutCheckerExecutorServiceRef

timeoutCheckerExecutorService

redeliveryPolicy

redeliveryPolicyDefinition

strategyRef

aggregationStrategy

<RemoveHeader headerName=xxx/>

<RemoveHeader name=xxx/>

removeProperty

name

propertyName

name

zipfile

zipFile (and many others, the full list is at https://camel.apache.org/manual/camel-3x-upgrade-guide-3_15.html#_data_formats)

New Features and Improvements

New Feature Description

ID

New Feature Description

ID

When triggering a Dataload, it is possible now to set the incremental mode to false explicitly.

PFIMCORE-929

Camel was migrated to version 4.1.0.

PFIMCORE-1538

If there are issues with converterExpression, the converterExpression value is now logged.
If there are issues with a Groovy script, the reasons for failing are also logged and displayed PlatformManager UI in Invalid Records.

PFIMCORE-1683

Statuses for retry route were extended with 502 BAD_GATEWAY.

PFIMCORE-1745

New pfx-smtps component has been added.

PFIMCORE-1771

All IM services have been updated to use Java 17, Spring Boot 3.1.5 and Spring Cloud 2022.0.4. Other dependencies have also been updated.

PFIMCORE-1773

Upgrade script for Camel 4.1.0 migration was created.

PFIMCORE-1792

Documentation for REST module has been updated to reflect changes in new Camel.

PFIMCORE-1794

There is a new upgrade script which will add a bootstrap library to pom.xml if the library is missing.

PFIMCORE-1829

Components library versioning has been changed. Now the versions mimic IM versions, but without hotfixes.
Example: IM version 4.1.1 has complib version 4.1.0, IM version 5.0.0 has complib version 5.0.0.

PFIMCORE-1831

JWT token is no longer printed out to the log when Pricefx connection debug is enabled.

PFIMCORE-1840

Fixed Issues

Bug Description

ID

Bug Description

ID

ZIP files can now be parsed in a new IM flow. Dataload routes have been  changed to support ZIP files and operate on streams (instead of loading objects to memory).

PFIMCORE-1756

Events polling via properties does not work in provisioned IMs.

PFIMCORE-1762

In case of an error loading file from Git, the error message does not say which file causes the problem.

PFIMCORE-1763

An invalid field "STATUS" appears in .gitlab-ci.yml file. 

PFIMCORE-1770

REST module caching does not validate passwords.

PFIMCORE-1802

RouteXMLValidator does not run when route data is saved to the repository, it should run before the route definition is loaded into context.

PFIMCORE-1803

Camel spooling to disk is not enabled in Integration > Application bootstrap.yaml.

PFIMCORE-1809

Pricefx connection stops working after upgrading to IM 4.10.1.

PFIMCORE-1810

In Data Upload, when you submit a large file, not all records are uploaded and the Data Upload stays in the UPLOADED state.

PFIMCORE-1817

A new property spring.autoconfiguration.exclude is set when event-driven configuration is disabled. This is needed in order to disable auto-startup of MQ broker in updated Spring.

PFIMCORE-1821

Webservice proxy does not return WSDL definition on provisioned IMs.

PFIMCORE-1822

Wrong versioning of component library results in requiring a version which does not exist.

PFIMCORE-1830

When there are any parsing errors when calling the IM client parseDataloadPreview method, IMException is thrown but it does not contain information about the errors.

PFIMCORE-1842

Upgrade of Provisioned IM stops at the last step of the Upgrade Tool in case Gitlab API cannot be contacted.

PFIMCORE-1844

When there is an error during parsing CSV file (invalid data for a given config), an error 500 is returned. But it should return a proper IMException with an error message.

PFIMCORE-1845

Upgrade tool: Code Modification fails on 500 Internal Server Error when there are no changes.

PFIMCORE-1848

IntegrationManager version 5.8.0