Continuous Integration or Continuous Delivery (CI/CD) are common and widely used practices in software development.

Why to Use It

When deploying a configuration and doing a QA release (from DEV) or a PROD release (from QA), it is a strongly recommended best practice to use the GitLab pipelines (or alternatively any other CI/CD tool) prior to manual deployment from Pricefx Studio. This will reduce potential sources of errors significantly.

By enabling the automated deployment it also will be ensured that the repository matches the partition 1:1. This is especially useful to avoid situations where a hotfix gets manually deployed to PROD and is never streamlined back to DEV, causing code loss and regression issues after new rollouts.

The pipelines can also be used to run unit tests (TDD4C) and code quality tools (Sonar).

How it Works

When a commit is made to QA or PROD branch (or potentially to any additional environement), the pipeline is triggered. The pipeline executes a pfxpackage command line tool that reads all the Pricefx objects in the repository and deploys them to the target partition. If the object does not exist in the target partition, it gets created and if it already exists, it gets updated. A special account (typically named “pfx.deploy”) should be created on the partition with the access of a Configuration Engineer.

The automated deployment to the DEV environment typically does not make sense because each commit would most probably overwrite the configuration work in progress.

Packaging Tool deploys the objects in a smart way, that means it compares the object with the one from the partition and deploys it only if there is a change. The objects that were not changed and therefore are out the scope are listed in the log as “skipping”.

What is Part of Scope

• Deployment to QA and PROD partitions (potentially additional ones).

• Creation of logics, tables, and other objects that you see in Studio can be deployed using pipelines.

What is Out of Scope (To Be Deployed Manually)

• Deployment to DEV partitions. The DEV partitions are deployed by configuration engineers while developing the solutions. Setting up auto-deployment would ruin their development in progress.

• .*.csv files: Data Files should not be kept in git. The recommended approach is to create system-required rows manually during the release or by writing a migration script using Calculated Field Sets or Data Loads.

• Deletion of the objects that were removed from GIT is still to be done manually after the deployment; either directly on the partition or in the Fetch tool window in Studio.

• CF_.*+Deployed.json files: Calculation Flow definitions in Deployed status should not be kept in git. The issue is that right after those objects get deployed, the calculation flow job will be triggered because the startingDate parameter will always be in the past.

How to setup

Create a user account for auto-deployment

The user account for the automated deployment with the relevant access rights needs to be created in the partition. The recommendation is to create an account named “pfx.deploy” and assign the “Configuration Engineer” role to it, the steps to do it are described below.

In Studio:

1. Navigate to Security > Business roles in Deploy tool window

2. Select the “initial-security-setup” in the module/component picker.

3. Select the target partition

4. Select the “Configuration Engineer” role

5. Click Deploy

In the target partition:

1. Open to Administration > Access Admin > User Admin.

2. Add a new user named “pfx.deploy” and set some strong password.

3. Assign the “Configuration Engineer” business role to this user.

Now you can use this user account in your pipeline setup.

Set up the pipelines as per your GIT service

In Studio:

1. Right click on the project root > Pricefx > CI Deployment and select the desired GIT service:

   • Create ‘.gitlab-ci.yml’ for Gitlab, or

   • Create ‘.github-ci.yml’ for Github, or

   • Create ‘.bitbucket-pipelines.yml’ for BitBucket, or

   • Create ‘.azure-pipelines.yml’ for Microsoft Azure Devops

2. In the dialog, select the branch for each environment and partition which you want to automaticaly deploy.

3. Click Create 'xxx' file

Once you commit the created file to GIT, it will be picked up.

Important Information and Limitations

Primary Key Fields Changes

If there is a change in the key field structure during the release, the data gets truncated. So you should potentially back up the data first, adjust and then import. Or alternatively modify the table, but create another one using a different name.

User Groups after GO LIVE

During development, the initial configuration of user groups is designed by a Configuration Engineer. After GO LIVE the security setup is owned and maintained by the customer. So it has to be ensured that the security adjusted by the customer in the partition is not overwritten by the old user groups from Git. There is a switch “keepUserGroups” in Packaging Tool that does that.

Shifting Attributes

When shifting an attribute, e.g. “Country” from e.g. attribute5 to attribute1 (P, PP, PX, C, CX, S, SX), the deployment will fail because there is a unique check on the Label and in certain moment there would be two attribute fields attribute5 and attribute1 with the same Label (“Country”) and that is not allowed. The solution for CP tables is to inactivate the previous version. Now for P,PX,C,CX tables you can delete all attributes' meta-data and deploy from Studio to first generate a clean state and then run the pipeline.

Running Dataload Locks DS or DM Deployment

When a Data Source (DS) or Datamart (DM) schema change is being deployed and at the same time some dataload locked the DS or DM, the deployment of the schema will fail with an unknown error.

Schema Change Between Pricefx Releases

If there is a new Pricefx major release, there is a big probability that there was a field added in some object. Such a field existing on the partition is recognized as a change by Packaging Tool and therefore is deployed every time. This also pollutes the pipeline log with a lot of warning messages. So it is recommended to fetch from the target partition to Git to have the new fields in Git as well. This will help run the pipeline faster and make the log human readable again.

Troubleshooting

General Error

General error means that backend has crashed. Check the server log to find out why.

Not Authorized for Command

In case you get an error message “Not authorized for command”, make sure your deploy user account has the required user roles to manage/administer the given object.

Follow the “Create a user account for auto-deployment” section above to use the Configuration engineer business role.

Here is the list of permissions required for every entity.

Note that General Admin contains various permissions like Configuration Wizards and Message Templates.

Timeout

If the deployment pipeline performed by pfxpackage tool is taking too long, it might be worth extending the timeout threshold - 120s is set in the pipeline templates as default. This may happen momentarily if the latency between the worker server running the pfx-package tool and the pricefx cluster has increased, or if your project has too many artifacts to be deployed at once, especially during rollouts of new phases.

Objects locked

When a calculation job is running, the table is locked. So you may need to re-run the pipeline.

Further reading:

See also: Git Version Control in Pricefx