IntegrationManager 1.4.0 LTS
In this section:
Migration Steps
Major Version Changes
- From now on, IntegrationManager is released together with PlatformManager (approx. 1 release per month).
For details see /wiki/spaces/INTG/pages/1492975782 (Pricefx only link).
Component Changes
pfx-csv:marshalpropertyrecordSeparatornow requires the separator definition\nasLFand\rasCR. You may use various combination of separators.- Previously:
<pfx-csv:marshal?recordSeparator=\r\n/> - Now:
<pfx-csv:marshal?recordSeparator=CRLF/>
- Previously:
- Notable changes in parameters naming:
| Previously | Currently |
|---|---|
| DM.Refresh.dataMartName=xxx | pfx-api:refresh?objectType=DM&targetName=xxx |
| ResetColumn.columnName=xxx | pfx-api:resetColumn?columnName=xxx |
Dependency Changes
camel-emailwas added as a default dependency – you do not need to provide the dependency anymore.
Property Changes
- Default integration.route-error-handling.email.mailFrom property is now based on the integration name. E.g. if the integration.name property value is db-integration, integration.route-error-handling.email.mailFrom will be db-integration@pricefx.eu. You do not need to provide that property anymore.
Other
- It is not recommended to use classes in the PFX namespace like Loaddata and Integrate which are used in XML definitions. Commands like Fetch, Save, Integrate, which were not part of the PFX namespace, but were accessible, are now removed. Please use the pfx-api Component instead.
org.joda.timeis no longer used within IntegrationManager. The only place whereorg.joda.timeis still allowed is Groovy Sandbox. It will be removed in next LTS release.
- Simple language can be used in criterions like:
simple:exchangeProperty[MyProp]. The previous requirement was$simple{exchangeProperty[MyProp]}. MSDynamicsservice was added to IntegrationManager. Instead of using custom HTTP client, it is now defineable as an OAuth2 connection; see MS Dynamics 365 Configurations. Please, remove your MSDynamics service if it is used in integration, and use pfx-rest instead:<to uri="pfx-rest:system?connection=qa_msdynamics_michalstepan_oauth&systemName=msDynamics&systemConfigurationItem=getRecord&recordType=rebate&recordId=2&apiVersion=v9.1"/>- See pfx-rest Component – method 'system'.
Improvements
| New Feature Description | ID |
|---|---|
| MS Dynamics service has been added to System Configurations. | PFIMCORE-413 |
| Pricefx API usages have been moved to the pfx-api-client module. The pfx-api component now implements all calls to pfx-api. XML definition now uses PfxApi producers instead of commands. | PFIMCORE-569 |
| IM no longer uses org.joda.time. The only usage is in Groovy Sandbox where the Joda support will be dropped in next LTS release. | PFIMCORE-585 |
| Pricefx debug can now be set in runtime via REST API. IM HTTP client has been updated. | PFIMCORE-586 |
| Manual instances can now be started or stopped. | PFIMCORE-605 |
| IM can now handle general filter structure and filter preconfigurations via XML. | PFIMCORE-608 |
| The 'preferencesSource' field has been added into the Swagger definition manually. | PFIMCORE-611 |
| When the sandbox denies evaluation of a class, the script is listed in exceptions whenever possible. | PFIMCORE-612 |
| pfx-rest now supports streaming of content into a file. | PFIMCORE-619 |
| Converters documentation has been enhanced. | PFIMCORE-620 |
| PriceFxConnectionDto was created to allow instantiation of the connection from outside. Such connection is converted into PriceFxClient automatically. This is used in the dataload functionality as this connection cannot be created via connections API. | PFIMCORE-622 |
| There is a new property `integration.hostname` which overrides the default hostname resolver when filled in. | PFIMCORE-623 |
| Swagger client has been refreshed to have the latest core API. | PFIMCORE-625 |
| There is a new module 'System Configurations' which allows users to call external systems and retrieve data/metadata and perform standard mappings. Only JSON is supported. | PFIMCORE-626 PFIMCORE-632 |
| There is a new RoutesCheckerService job which checks for struct routes. The job is by default disabled. To enable it, set the property integration.schedulers.routes-checker.enabled=true. You must also specify the input type of the route ('from' definition) by setting the property integration.schedulers.routes-checker.input-types (by default sftp, ftp). | PFIMCORE-633 |
| The Filter component now supports more operators (to match Pricefx core 6.0). | PFIMCORE-636 |
New classes were enabled in Sandbox:
| PFIMCORE-637 |
Properties loaded via REST API are now loaded as property resource. This allows them to be used in the context startup (beans) as the property load is performed in an earlier init phase. The whole logic is enabled only if integration.configuration.enabled=true. Note: You should restart IM after setting the property, so that the changes take effect. Integration configuration directory structure is created depending on integration.name property. Although it works with spring.application.name property, it's recommended not to use it, since this property is internal and necessary for backward compatibility. | PFIMCORE-639 |
| Maven modules 'integration-starters' and 'integration-platform' have been removed. Most of the small modules have been merged into 'pricefx-integration'. | PFIMCORE-641 |
| Dataload now sends progress information. | PFIMCORE-642 |
| AWS AutoConfiguration and SystemConfigurationsAutoConfiguration have been added. System Configurations can now be loaded either from AWS S3 or local resources. | PFIMCORE-643 |
| HTTP client now offers methods to get data from a connection. | PFIMCORE-646 |
| pfx-rest logging has been enhanced. Set loglevel to DEBUG for more information. | PFIMCORE-648 |
| pfx-rest can now handle file uploads. | PFIMCORE-652 |
| StringToDate now accepts string dates. It also allows you to set inputFormat to determine the format of the input string date. Also, the parameter 'nullValue' has been added to further parametrize what is considered a null input after which 'defaultValue' is returned. | PFIMCORE-654 |
| There is a new method pfx-api:internalCopy which requires a label. | PFIMCORE-655 |
| pfx-config now has the option toHeader which stores the result of the retrieval into a given header and keeps the exchange body intact. There is also a new option defaultValue which is returned if no value for the key is found. | PFIMCORE-656 |
| The [not] operator for filters has been implemented. | PFIMCORE-657 |
| Filter option [not] is now also supported in filters deployed via REST API. | PFIMCORE-659 |
| camel-email dependency has been added as a default dependency. | PFIMCORE-661 |
| The default mailFrom property is now based on the integration name. E.g. if the integration.name property value is db-integration, mailFrom will be db-integration@pricefx.eu. | PFIMCORE-668 |
| pfx-api:datamartImport has been implemented. | PFIMCORE-671 |
| pfx-api:datamartExport has been implemented. | PFIMCORE-672 |
| Web services via Spring-WS have been implemented, as well as the web service Delete. It is now possible to define a filter within the web service structure. | PFIMCORE-673 |
| Asynchronous delete has been implemented for all object types. | PFIMCORE-684 |
| Several not important log messages have been moved to the DEBUG level. | PFIMCORE-685 |
| IM HTTP client has been updated to transform deprecated structures for /camel/routes and /camelroutes to uniform API. | PFIMCORE-686 |
| Line separator in dataload templates is no longer hardcoded but it is now detected from the input data. | PFIMCORE-687 |
| There is now an option to set a business key without setting the length; that is done using the asterisk * character. | PFIMCORE-688 |
| pfx-xml:marshal has been implemented. Allowed body types are String and Map. | PFIMCORE-689 |
| Mappers defined via the Spring bean can now be used in pfx components in routes. | PFIMCORE-690 |
| System Configurations (pre-set configurations) now contain the Salesforce - Product mapper. | PFIMCORE-692 |
| The multilevel mapper now offers an aggregate and non-aggregate modes. | PFIMCORE-694 |
| FETCH for JLTV, JLTV2, MLTV, MLTV2, MLTV3, MLTV4, MLTV5 and MLTV6 now uses lookuptablemanager.fetch instead of (general) fetch. | PFIMCORE-695 |
| General mapper has been enhanced with generic metadata definition via the transformer function. Configuration has been migrated to XML and aggregated with the FILTER configurations. DATA configurations have been renamed to SELECT configurations. Configuration extensions have been separated from the mapper/filter/connection API. HTTP client has been updated. | PFIMCORE-696 |
| Instance update event is now sent to the Kafka topic if a connection is added or removed. | PFIMCORE-708 |
| There is a new method pfx-rest:system which elevates the possibilities of system configurations into pfx-rest fetching capabilities. | PFIMCORE-712 |
| System configurations are now resolved in runtime with caching driven by the property integration.schedulers.system-configurations.cache.ms. | PFIMCORE-713 |
| Log masking: properties with the name 'secret-key' are also masked now. | PFIMCORE-717 |
| System Configurations can now define converterExpression or Groovy in mapper rows. | PFIMCORE-719 |
Fixed Issues
| Bug Description | ID |
|---|---|
| The web service Integrate throws a NullPointerException sometimes. | PFIMCORE-509 |
| IntegrationManager does not send cluster and server in a startup event for provisioned instances. | PFIMCORE-539 |
| IM HTTP client does not properly return Object for the method convert. | PFIMCORE-621 |
| pfx-api:fetch on the PX objectType causes an exception when SKU is missing in the filter condition. | PFIMCORE-624 |
| Retry mechanism does not handle Pricefx core exceptions. | PFIMCORE-635 |
| pfx-csv default separator is set to \n which is problematic at some platforms (macOS). | PFIMCORE-653 |
| Health check shows DOWN when one route is DOWN. | PFIMCORE-658 |
| Event is processed before it is fully saved into a file. | PFIMCORE-660 |
| Setting error-handling to false causes the application to fail on start. | PFIMCORE-665 |
| Web service filter definition cannot be properly deserialized. | PFIMCORE-674 |
| Filter deployment fails when the notNull operator is used. | PFIMCORE-675 |
| When loaddata fails, it logs an error about 'integrate' failure. | PFIMCORE-676 |
| No Camel context is loaded if IM is started without a profile. | PFIMCORE-697 |
| Simple language as value="simple:exchangeProperty[...]" defined in criteria fails on deploy. | PFIMCORE-705 |
| It is not possible to set recordSeparator for the pfx-csv:marshal command. | PFIMCORE-714 |
| Route used for invocation of a connection stays deployed when finished. | PFIMCORE-718 |
IntegrationManager version 5.0.0