This blog series consists of:
- Logic Apps connector performance (1/3) – The bad!
- Logic Apps connector performance (2/3) – The ugly!
- Logic Apps connector performance (3/3) – The good!
Load patterns
Integrations often result in bursty load. Some examples are:
- Huge data export from a line of business application
- Splitting of a big batch file into individual transactions
- More and more event-driven integrations
The integration platform has two important responsibilities. On one hand, it has to process the messages as fast as possible, to meet the business requirements. On the other hand, it sometimes has to slow down (throttle) in case the receiving system can’t keep up with the pace. The latter responsibility can be easily configured via the concurrency control feature. However, on the performance side, the Logic Apps connectors have limitations.
Connector throttling
In the connector overview, each connector has documented throttling limits. The throttling happens per API connection. These are the current throttling limits of some important connectors:
| SQL Server | 100 calls / 10 seconds | 10 calls / second |
| Blob Storage | 600 calls / 60 seconds | 10 calls / second |
| Event Grid | 100 calls / 60 seconds | 1,66 calls / second |
| File Storage | 600 calls / 60 seconds | 10 calls / second |
| SFTP | 100 calls / 60 seconds | 1,66 calls / second |
| Custom Connector | 500 calls / 60 seconds | 8,33 calls / second |
Recently, we faced the following business requirement: receive and process 10K messages from an SFTP folder. The business gave us even 1 hour, which was very generous, in my opinion. No big deal we thought! But… for each message, we had at least 3 calls (GET file content, COPY file to archive, DELETE file from folder). This means 30K calls / hour, which equals 8,33 calls / second. Far above the throttling limit of the SFTP connector!
Throttling behavior
Once a connector starts throttling, it returns an HTTP 429 “Too many requests”. Based on this HTTP status code, Logic Apps will kick off its retry mechanism, which is by default 4 retries with a 20 seconds interval. When there is sporadic throttling, this gracefully resolves the issue. However, when hitting the limits really hard, this results in a cascading effect that makes it only worse.
As a test setup, I’ve created a Logic App that gets fired through an HTTP request. The second action is writing a file to blob storage. By using artillery, I put the Logic App under high concurrent load. This causes throttling and cascading retries, as shown below.
Conclusion
In production-grade integration solutions, it’s not desired that the integration platform is the limiting factor in the end-to-end performance. Typically, the integration platform should be throttled, in order not to overwhelm the backend systems. The Logic Apps connectors should have much higher throttling limits, to fulfill their scalability promise. Several escalations to the product team resulted only in minor improvements. Read in the next parts of this blog post series, how we can work around these limitations.
Subscribe to our RSS feed