Transformation is a key part of integration, information in one format needs to be used in another format and eventually processed in some way. While working with a client a while ago we did a successful PoC to move their integrations to Azure and want to share the findings regarding mapping. We needed to transform and the alternatives Liquid and the Enterprise Integration Pack (EIP) did not fit well. Transformations were far to complex for Liquid and maps with EIP did not offer a natural way between XML and Json. Important for the decision was also that the customer already used MapForce from Altova in their current solution, so they already had an investment in licensing and knowledge. This post is intended to show how we solved the transformations with MapForce and not a detailed step by step description. We generated code from MapForce and performed the transformations in Azure Functions.
Example mappings
One of the things we needed in the PoC was a lookup changing some identifiers in one system to their counterparts in the other, my examples are two flavors of lookups. All code can be found in the GitHub repository MappingFunctions. So you get a picture of the effort to get started, the maps in the example took me about two hours to find out how to do them.
Steps
These are the steps in short I took for the example.
- Create a MapForce project
- Create the transformations
- Generate code (C#)
- Create Azure Functions project
- Add the generated code to the Functions solution
- Write code to execute the mappings in the functions
- Call the functions from Logic Apps to test (excluded in the repository)
Simple lookup example
In the simple lookup example, the data is included in the value-map shape, thus to add new values we will need a new release. This can be ok in cases when data changes very seldom.
Advanced lookup example
Sometimes data changes more often, then we don’t want the data to be hard coded in the map. In the advanced lookup example, the data is provided as a second input that could come from a database or a blob, thus being able change data without a new release. Note that one input is XML and the other is Json
The lookup is performed in a user function that uses the second input.
Code generation in MapForce
Code can be generated in several languages, C#, Java, XSLT, etc., choose the one that suites your project best. As different languages have different feature sets, not all constructs works on all languages. In the PoC I wrote about we used Java as that was the language that team preferred and used. The example project uses C# (.NET Core 3.1), with the settings in the image.
The projects MapForce generated could be added without changes to the Azure Functions solution. This is good as if you need to update your transformation it’s possible to just overwrite them and keep them together in source control.
The code to execute the transformation is quite straightforward. In the example the documents are small and string variants are used, for larger documents there are variants that use streams.
public static class SummaryAdvanced
{
[FunctionName("SummaryAdvanced")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("Starting SummaryAdvanced.");
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
if (string.IsNullOrEmpty(requestBody))
{
return new BadRequestResult();
}
// TODO: Add more error handling
// Create the mapper object
SummaryLookupMapToCatalog_Summary_Schema mapper = new SummaryLookupMapToCatalog_Summary_Schema();
// Sources
Altova.IO.Input booksSource = new Altova.IO.StringInput( requestBody);
Altova.IO.Input shelfsSource = new Altova.IO.StringInput(MappingFunctions.Properties.Resources.Shelfs);
// Mapping result
StringBuilder responseMessageBuilder = new StringBuilder();
Altova.IO.Output Catalog_Summary_SchemaTarget = new Altova.IO.StringOutput(responseMessageBuilder);
// Execute transformation
mapper.Run(booksSource, shelfsSource, Catalog_Summary_SchemaTarget);
return new OkObjectResult(responseMessageBuilder.ToString());
}
}
Final thoughts
Having one tool that can assist creating the maps between different formats is valuable, it saves time not needing to change tooling. I find this a good way to perform mappings and it was reasonable effort to get started. That said the documentation could be better and it can take some time to be up and running with complex transformations. I have not dived deep in all features, see this post as a starting point. Generating code in different languages is good but I assume a team will stick to what suites their environment best.
Note: These are my own thoughts and I don’t have any business contact with Altova. I used a 30-day trial version of MapForce that can be downloaded here.
You can download source code from the GiHub repo 