The Aggregation data integration pattern is the act of taking or receiving data from multiple systems and inserting consolidating them into one. For example, customer transaction data integration could reside in three different systems, and a data analyst might want would like to generate a report which uses data from all of them. One solution could be to create a daily migration process from each of those systems system to a data repository and then perform an aggregation query that against that database. But, then there we would be another database to keep track of and keep synchronized.In addition, as things change in the three other systems, the data repository would have to be constantly kept up to date. Another need to maintain this aggregate structure and maintain its synchronizations.
An additional downside is that the data would be a snapshot that is a day old. Thus, so for today’s real-time reportsnumbers, the analyst would have to either manually initiate the migrations manually or wait another day. One Or, they could set up three broadcast applications, achieving a situation where the reporting aggregate database is always up to date synchronized with the most recent changes in each of the systems. But there would still be a , once again we would need to maintain this database which only stores replicated data so that it can be queried every so often. In addition, there will be a number of wasted API calls to ensure that the database is always up to X minutes from reality.
This is where the scenario that depicts the advantages of the aggregation pattern comes into play. If you build an application, or use one of our templates that is built on it, you will notice that you can on demand query multiple systems, merge the data set, and do as you please with it.
For example, you can build an integration app which queries the various systems, merges the data and then produces a report. This way you avoid having a separate database and you can have the report arrive in a format like .csv or the format of your choice. You can place the report in the location where reports are stored directly.
...
Value of Aggregation
The aggregation pattern derives its value from allowing you to extract and process data from multiple systems in one united application. This means that the data is up to date at the time that you need it, does not get replicated, and can be processed or merged to produce the dataset you want.
...
Usefulness of Aggregation
The aggregation pattern is valuable if you are creating orchestration APIs to “modernize” legacy systems, especially when you are creating an API which gets data from multiple systems, and then processes it into one response. Another use case is creating reports or dashboards that pull data from multiple systems and create an experience with that data.
...