This module handles storage of IM objects into a repository.

Properties

Property	Default Value	Description

Property

Default Value

Description

integration.configuration.enabled

true

If set to true, the repository will be enabled.

integration.configuration.base-directory

Unix: /var/pricefx/integration

Windows, macOS: user home dir

Directory where the files are stored.

integration.configuration.type

fs

Selects the type of storage:

fs – FileSystem storage.
s3 – AWS S3 storage.

Storage choice is independent from the application logic.

Storage Implementation

There are multiple storage options driven by the property integration.configuration.type. When the application starts, all objects are loaded from the repository.

The following object types can be stored:

Type	Path/Key	Description

Type	Path/Key	Description
Filters	filters	Filters definitions.
Routes	routes	Routes definitions. One route per file.
Mappers	mappers	Mappers definitions.
Connections	connections	Connections definitions.
Properties	config	application.properties with properties. It is marked as a property source for Spring.
Certificates	certs	Certificates definitions.
Classes	classes	Custom classes subjected to rules. Currently supported Groovy classes.

FileSystem storage

This storage implementation will store the objects in the file system into a directory specified by the property integration.configuration.base-directory.

S3 storage

This storage implementation will store the objects into AWS S3 buckets. By default, it will put each object type into their own bucket. For the S3 storage to work properly, you need to set the property integration.aws.enabled=true and the credentials – for details see How to Connect to AWS in IM.

The implementation assumes it has access to the file system temporary folder.

NoOp storage

This storage will not store objects into any repository (objects will be kept in memory only). This implementation will only kick in when integration.configuration.enabled=false.

Groovy Classes

Note that this functionality is in an early development stage and might contain serious bugs.

If standardized Groovy scripts are not sufficient for your use case, you can load classes dynamically. IM will compile the class, link it via the class loader, store it as a Spring bean (singleton) and provide it to Camel.

If some of the classes fail to find/compile/validate/bind, the class will be skipped and an error will be logged.

Limitations

You must provide uncompiled .groovy classes.
Classes cannot use dependencies outside of the IM scope. This is due to the fact that the class is dynamically compiled with IM classpath.
Approach is tested for Camel routes and Spring beans.
You must implement one of the interfaces:
- org.apache.camel.Processor
- org.apache.camel.AggregationStrategy
- net.pricefx.integration.csv.schema.validators.Validator

If all requirements are met, this is the log output for successful loading of a class:

10:31:20.596 [pool-1-thread-1] INFO  n.p.i.c.task.LoadClassObjectsTask - Class loading task started. Going to load 1 classes: [/tmp/junit17404521069772355861/junit738258571191905368/MyGroovyProcessor.groovy]
10:31:21.013 [pool-1-thread-1] INFO  o.a.s.c.u.s.b.BouncyCastleSecurityProviderRegistrar - getOrCreateProvider(BC) created instance of org.bouncycastle.jce.provider.BouncyCastleProvider
10:31:21.991 [main] INFO  n.p.integration.util.SpringUtils - New bean registered: [myJavaProcessor] with definition: Generic bean: class [MyGroovyProcessor]; scope=; abstract=false; lazyInit=null; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null
10:31:21.995 [main] INFO  n.p.i.configuration.BeanLoader - All [1] classes were loaded. Loaded classes: [/tmp/junit17404521069772355861/junit738258571191905368/MyGroovyProcessor.groovy]
10:31:21.996 [main] INFO  n.p.i.configuration.BeanLoader - Done loading 1 classes with 0 errors.

Example

Let’s have one Groovy class.

Groovy [MyGroovyProcessor.groovy]:

import org.apache.camel.Exchange
import org.apache.camel.Processor

class MyGroovyProcessor implements Processor {

    @Override
    void process(Exchange exchange) throws Exception {
        println "Groovy executed!"
    }
}

Directory content:

mst@michal-desktop:/var/pricefx/integration/integration-manager/classes$ ll
total 16
drwxrwxr-x 2 mst mst 4096 Jun  4 14:13 ./
drwxrwxr-x 9 mst mst 4096 Jun  2 10:43 ../
-rw-rw-r-- 1 mst mst  327 Jun  4 14:08 MyGroovyProcessor.groovy

Route:

When the app starts, these are the relevant parts of the logs:

Route execution output: