Logic Apps Debatching

SplitOn Command

Logic Apps offer the splitOn command that can only be added to a trigger of a Logic App. In this splitOn command, you can provide an expression that results in an array. For each item in that array, a new instance of the Logic App is fired.

Debatching JSON Messages

Logic Apps are completely built on API’s, so they natively support JSON messages. Let’s have a look on how we can debatch the JSON message below, by leveraging the splitOn command.

Create a new Logic App and add the Request trigger.  In the code view, add the splitOn command to the trigger.  Specify the following expression: @triggerBody()[‘OrderBatch’][‘Orders’]

Use Postman to send the JSON message to the HTTP trigger.  You’ll notice that one input message, triggers 3 workflow runs.  Very easy way to debatch a message!

Debatching XML Messages

In old-school integration, XML is still widely spread. When dealing with flat file or EDI messages, they are also converted into XML. So, it’s required to have this also working for XML messages. Let’s consider the following example.

Update the existing Logic App with the following expression for the splitOn command: @xpath(xml(triggerBody()), ‘//*[local-name()=”Order” and namespace-uri()=”http://namespace”]’).  In order to visualize the result, add a Terminate shape that contains the trigger body as the message.

Trigger the workflow again.  The result is as expected and the namespaces are nicely preserved!

Exception Handling

The advantage of this approach is that every child message immediately starts processing independently from the others. If one message fails during further processing, it does not impact the others and exception handling can be done on the level of the child message. This is comparable to recoverable interchange processing in BizTalk Server. In this way, you can better make use of the resubmit functionality. Read more about it here.

Let’s have a look what happens if the xPath expression is invalid. The following exception is returned: The template language expression evaluation failed: ‘The template language function ‘xpath’ parameters are invalid: the ‘xpath’ parameter must be a supported, well-formed XPath expression. Please see https://aka.ms/logicexpressions#xpath for usage details. This behavior is as desired.

What happens if the splitOn command does not find a match within the incoming trigger message? Just change the xPath for example to @xpath(xml(triggerBody()), ‘//*[local-name()=”XXX” and namespace-uri()=”http://namespace”]’). In this case, no workflow instance gets triggered. The trigger has the Succeeded status, but did not fire. The consumer of the Logic App receives an HTTP 202 Accepted, so assumes everything went fine.

This is important to bear in mind, as you might lose invalid messages in this way. The advice is to perform schema validation before consuming a nested Logic App with the splitOn trigger.

Monitoring

Within the standard overview blade, you cannot see that the three instances relate to each other. However, if you look into the Run Details, you notice that they share the same Correlation ID. It’s good to see that in the backend, these workflow instances can be correlated. Let’s hope that such functionality also makes it to the portal in a user-friendly way!  

For the time being, you can leverage the Logic Apps Management REST API to build your custom monitoring solution.

For Each Command

Another way to achieve a debatching-alike behavior, is by leveraging the forEach command. It’s very straightforward to use.

Debatching JSON Messages

Let’s use the same JSON message as in the splitOn example. Add a forEach command to the Logic App and provide the same expression: @triggerBody()[‘OrderBatch’][‘Orders’].

If we now send the JSON message to this Logic App, we get the following result. Remark that the forEach results in 3 loops, one for each child message.

Debatching XML Messages

Let’s have a look if the same experience applies for XML messages. Modify the Logic App, to perform the looping based on this expression: @xpath(xml(triggerBody()), ‘//*[local-name()=”Order” and namespace-uri()=”http://namespace”]’)

Use now the XML message from the first example to trigger the Logic App. Again, the forEach includes 3 iterations.  Great!

Exception Handling

I want to see what happens if one child message fails processing. Therefore, I take the JSON Logic App and add the Parse JSON action that validates against the schema below. Remark that all fields are required.

Take the JSON message from previous example and remove in the second order a required field. This will cause the Logic App to fail for the second child message, but to succeed for the first and third one.

Trigger the Logic App and investigate the run history. This is a great result! Each iteration processes independent from the other. Quite similar behavior as with the splitOn command, however it’s more difficult to use the resubmit function.

You must understand that by default, the forEach branches are executed in parallel. You can modify this to sequential execution. Dive into the code view and add “operationOptions” : “Sequential” to the forEach.

Redo the test and you will see that this has no influence on the exception behavior. Every loop gets invoked, regardless whether the previous run failed.

Monitoring

The monitoring experience is great! You can easily scroll through all iterations to see which iteration succeeded and which on failed. If one of the actions fails within a forEach, the Logic App gets the Failed status assigned.

What should we use?

In order to have a real debatching experience, I recommend the splitOn command to be used within enterprise integration scenarios. The fact that each child message gets immediately its specific workflow instance assigned, makes the exception handling strategy easier and operational interventions more straightforward.

Do not forget to perform first schema validation and then invoke a nested workflow with the Request trigger, configured with the splitOn command. This will ensure that no invalid message disappears. Calling a nested workflow also offers the opportunity to pass the batch header information via the HTTP headers, so you can preserve header information in the child message. Another way to achieve this, is by executing a Transformation in the first Logic App, that adds header information to every child message.

The nested workflow cannot have a Response action, because it’s decorated with a splitOn trigger.  If you want to invoke such a Logic App, you need to update the consuming Logic App action with the following expression: “operationOptions”: “DisableAsyncPattern”.

If we run the setup, explained above, we get the following debatching experience with header information preserved!

Conclusion

Logic Apps provides all required functionality to debatch XML and JSON messages. As always, it’s highly encouraged to investigate all options in depth and to conclude what approach suites the best for your scenario.

Thanks for reading!
Toon