We’re currently helping a client get a custom application up and running that uses the FreeAgent API. Although the application works with our company’s test account, it fails with the client’s account. Specifically, the issue seems to be the fact that despite setting the ‘Accept’ header within our requests to application/json, the FreeAgent API is returning XML-formatted data instead of JSON-formatted data when logged in with the client’s account.
As far as I can tell within the API documentation, the FreeAgent API is to return the data in the format specified within the ‘Accept’ header. Is there some other setting elsewhere within FreeAgent that could be overriding this setting, and forcing responses to come back as XML?
Sean, this is a tricky one to diagnose without seeing more detail about the requests (the one which works and the one which fails). Could you supply more information please, e.g. which endpoint and sample payload?
Have you tried setting the Content Type header to JSON as well? This shouldn’t be necessary, so shouldn’t affect the response, but it would be useful to clarify.
We’re only ever making GET requests, so as far as I understand things, setting Content-Type does not come into the picture. On that point, we are making GET requests to basically any of the endpoints that provide data that shows up in the FreeAgent Data Export. Unfortunately I cannot tell you a specific endpoint that is failing with the exception we’re seeing, which is the following by the way (using the Newtonsoft deserializer in C#):
Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
I’ll have to add some additional logging, and have the client log in with their account tomorrow to say for sure which is the first endpoint we call that fails (it should be the ‘Bank Account’ endpoint). I can say at this point, however, that I was able to reproduce the above exception in our dev environment by setting Accept to ‘application/xml’, as opposed to ‘application/json’ (the setting in production).
So after having the chance to add more logging to our application, we’ve found that a “We’ll be back soon!” HTML maintenance page is being returned upon attempting to fetch invoices.
Is the FreeAgent API becoming overwhelmed with requests from our service? Otherwise, any idea why this page is being returned?
It appears your app may indeed be “overwhelming” the API and hitting our rate limits - without knowing which application is yours it’s hard to say for definite.
There does appear to be a bug with the way the app is responding to these rate limit overages - it should definitely be responding with the correct format. We’ll take a look at that and get that fixed up.
Worth noting at this point that our plans are to lower the rate limits over time to achieve a sensible level which allows everyone to get fair use out of the service. It might be worth you looking at tweaking your code to interact with our API in a more optimised manner, keeping the number of requests as low as possible.
If you’re capturing the User-Agent header in your logs, you should be able to identify our application (we’ve specified a value that should be unique to our application).
Apologies for exceeding the rate limits, however, all I really had to go on when trying to find what they were was this tweet: https://twitter.com/cushionapp/status/632653462719832064. I couldn’t find any rate limits defined on FreeAgent’s website, or within the API documentation.
To ensure that we’re working within FreeAgent API limits, could you please let us know what’s the maximum rate at which we should be sending requests?
I’m afraid there are no hard limits I can give you at the minute as it’s something we’re currently reviewing. The limit you hit is there to prevent DoS type attacks rather than promote fair usage.
Re the user agent string - could you DM me that just to confirm that it is your application we’re looking at in our logs? I’m 99% sure it is, but better to remove any uncertainty.
We’re hoping to have something concrete we can publish about rate limits during the coming quarter, but we expect this to be in line with other SaaS APIs - somewhere up to the low hundreds of requests per minute. I’m sure you can appreciate we’d rather not commit ourselves to anything here which may turn out to be ill advised later on.
Okay - We’ll aim to be no more than 300 requests per minute, unless we hear otherwise. It’s important to note that our application will be run on-demand maybe a few times a week, and when it does run it will only be fetching data for a few minutes.
That’s great, thanks for that. In order to future-proof your integration it might be worth implementing support for 429 “Too many requests” response statuses if possible.
Other optimisations you could look at would be to to cache infrequently changing data (e.g. categories) locally, or requesting more items per page (for paginated endpoints) to cut down on the number of requests.