• It required time, as any manual process. The person doing the release needed to have an environment ready to use the Maven commands, and then follow the documented steps one by one.

• It was not a 'pleasant' job to do. Because the process was very mechanical in nature, it was not something that had room for creativity. On top of that, to ensure that everybody was familiar with the process, a different person was chosen every time to do the release.

• It was error prone, as every manual job is as well.

This release process was a particularly suitable candidate to be automated. The goal was to have a reliable 'push button' that anyone could use when needed, without any other intervention.

What Kind of Releases Are Done

We have a structure of Git branches similar to this:

Releases are done only from the 'stable' branch once QA gives the greenlight.

We have two types of releases:

• Hotfix releases. It's a small release that contains only bugfixes that are considered important for a customer. They are done as needed.

• Minor releases. It's a bigger release, that contains both bugfixes and new backward compatible features. They are done once a month.

Occasionally we also do ‘major’ releases, but that’s out of the scope here.

Starting to Automate the Release Process

Before the automation, and to have an effective automated release process, one change to our internal versioning system needed to be made. 

Semantic Versioning

Initially versions consisted of two digits. When tags were created, they actually contained three digits. When creating such tags, the person who was doing the release needed to check what was the latest hotfix version released, and use that information accordingly.

So one change that was done was to switch to Semantic Versioning. This way, the hotfix version could be stored in the project, and could be used automatically. 

Maven Release vs. Manual Tagging

Our software is built with Apache Maven, which already provides a release plugin that could do some of the automation we want. However, this plugin was designed to do a series of fixed steps which are not customizable. The plugin documentation states the following:

To be clear, this is all the Release plugin does, it isn't a substitute for organizational process, it wasn't designed to be customized to meet your specific process.

The release plugin also violates the Single Responsibility Principle – one command changes versions, creates tags, deploys artifacts and site documentation.

In our case, we wanted to have more control over the process, so we went for a script that would run only the commands we needed. Something similar to what we initially had in our releases documentation. 

Working on a Release Script

The script that was prepared is simple, about 130 lines. The operation of the script is roughly described in the following diagram:

This diagram applies to minor releases. The merge request is still a manual step, so it can be reviewed if needed before the automate steps take control. For hotfix releases, there is no merge request involved, but the staging branch is still kept up-to-date at the end.

Also, the following points were considered during the design of the script.

Fail Fast

The script needs some preconditions to work, for example a GitLab token needs to be set as an environment variable. All the preconditions are checked at the very beginning, and if any of these preconditions fail, the script fails immediately, without doing any actual changes in the repository.

Make Changes Atomic

Perhaps one of the most important conditions of the script is to never leave the branches in an inconsistent state. Because of this, no changes are pushed until the very end. Git offers the --atomic parameter, which ensures that either all the changes in all the branches changed will be pushed, or none will. This approach has also another good side, and that is that in case the script fails, whatever error happened can be corrected, and the operation can be just retried again without any other side effects.

Propagating Changes

As any other branching model, commits in 'lower' branches must be present in 'higher' branches as well. Once a release is completed in the 'stable' branch, all the commits should be propagated also to 'staging', the immediate superior. This is as simple as merging the 'stable' branch into the 'staging' branch. 

Dealing with Conflicts

However, conflicts are almost always sure to happen. As in every hotfix release the version in 'stable' is changed after it was branched from 'staging', one possible merge conflict will be the version.

On the other hand, if we knew that the only possible conflict was going to be the version, then an 'empty' merge commit could be done, and this way, any manual intervention could be avoided. Git offers a merge strategy named 'ours', and that's what the script uses.

But it's very important to make sure that the version is indeed the only possible conflict. Otherwise, other changes would be lost in this empty merge commit. Because of this, one of the preconditions of the script is to check if the 'stable' and 'staging' branches are in sync. This is, if there are commits in 'stable' which are not in 'staging'. If that's the case, the script fails just as it would do with any other precondition, and it's up to someone to correct the situation first.

How It Is Used

As described in the previous diagram, the process is expected to start after the staging branch is merged into the stable branch, in the case of minor releases. Right now, after the changes that the script brings, to start the release process, we have a Gitlab job that anyone can trigger without any further training or documentation to read. And for hotfix releases, the process is even a bit simpler as no merge request is involved.

...

This is essentially a one-click button to do releases, and exactly what we were aiming for. Of course, there are still some aspects that can be improved even more, and that is something that we will be looking into in the future.

Future Work

Everything can be improved. Automation is not a goal, but more of a journey. And during that journey, new ideas can come up. For example: should integration tests be run in the pipeline build for the tag? Usually, the only thing one is interested in in such builds are the artifacts. But on top of that, it should not be possible to do a release if there are broken tests in the branch. Right now, the person who does the release can see whether the branch is in good shape or not before pushing the release button, but this can also be enforced with GitLab rules.