Rate limit frustrations

1

We use the API to create an invoice with FA, register the payment with GoCardless/Stripe and then update the invoice in FA. Alongside this, each morning we have a background job that runs, finding all unexplained/unapproved transactions in FA, parsing the reference and then hitting the GoCardless/Stripe APIs to try and find the data we need to explain the transaction. Once we’ve gathered the payout data, we then loop through the invoices (placed individually onto a queue), explaining them in the FA API.

With the 2 requests per second rate limit in place, we have to artificially slow down our background jobs from being able to process, which is frustrating and clogs our queues. In order to process our background jobs, we run 3 servers, so the moment we register 1500 invoices to raise or 1500 transactions to explain, we’ll trigger the rate limit exception. To get round this, we’ve put in a sleep(1) second to each of our background jobs to try and not trigger the exception. Without the sleep(1), we’d hit the API limit every single second.

Can I plead with you to increase your rate limit to something that’s more practical? I note that GoCardless have a limit of 16/s and Stripe 1000/s - we’ve never triggered these, but 2/s is very hard to work with…

I’m pretty sure you’ll say that you can’t raise your API rate limit, so assuming that’s the case, please could you give me a suggestion as to how, given the scenario I’ve described, you would work with your APIs and within the rate limits?

2 Likes
2

In DoubleAgent I keep track of the rates in the API layer and auto-throttle the API requests (i.e. sleep until the limit is reset). It means each process goes as fast as it can and then waits for as short a time as possible to comply with the rate limits.

I agree it would be great if the rate limits were increased.

– Andrew

1 Like
3

I’m hitting similar frustrations when trying to explain 1000s of transactions.

If they allowed us to explain transactions in bulk we could do it in one API call and avoid the rate limit issues. I.e. accepting an array of transaction explanations rather than only accepting 1 explanation per request.

4

Annoyingly, I ended up having to change our company’s billing cycle from running on the 1st of the month to running every day (splitting our 2k customers between days 1-28). This was the only realistic way I could see us being able to continue scaling towards 10k customers; that and potentially moving to Xero (which I’d rather not have to do, but these types of API limitation are making me consider the move).