API limitations
Does Railway have an API call limit per service in the free account? I am running a cron job that involves a set of external API calls in a loop. However, it stops after executing 20 APIs. I have tested it locally, and it works fine. I would like to know the limitations of the Railway server.
Project ID: dd417141-599f-4431-bbf6-538cbc507f1d
27 Replies
Project ID:
dd417141-599f-4431-bbf6-538cbc507f1drailway has no limitations on the amount of calls your app can make to external services, you are likely being rate limited by the external service
Thank you I'll check with them.
what errors do you see on the 21st api call?
No errors, Just stop
I have added logs to monitor the pages.
1 page = 1 API call
you probably aren't logging errors when they occur
would you mind sharing what this external service is?
yes, I have missed to add logs to see errors. I'll add and test again.
CommerceLayer is the external service which I am using.
and why the need for making consecutive api calls?
They have a page limit, and we need to make multiple calls to retrieve all the data.
Ex: "meta": {
"record_count": 598,
"page_count": 24
},
they have that limit for a reason π
To get all 598 records, we have to call 24 times the API
they obviously dont appreciate you scanning through all the pages
and if they don't appreciate that kind of activity, railway wont appreciate that activity either
I my business requirement, I have to gather all the data per 5min. π
This task is already working with Vercel, There I have to use multiple cron jobs to handle the load. Because they are having time limitations for long running services.
That's the reason to move with Railway
I'm not sure this time CommerceLayer side also having any issues, I'll talk with them also.
very good chance they just tell you to stop scraping their api
and thats kinda what im telling you too
Without looping their API, There is no way to get all the data.
That's the problem.
the problem is why do you need to scrape their api for the data every 5 minutes
I have to run 320 times to gather our production data π
it also sounds like you should be on the railway teams plan
Sorry, I can't explain the whole business logic in this thread. By the way, this is an API for extracting product details. When the product stock levels change from CommerceLayer side, we need to update our inventory every 5 minutes.
Because Our inventory DB is the middleman used to expose stock levels to external services.
Can you explain a bit why I need to move for Railway Teams plan?
please find a different way to gather this data without loops, you are abusing CommerceLayer's api and CommerceLayer could decide to ban railway cloud, thus blocking anyone else on railway from using CommerceLayer
the bigger issue is the abuse of CommerceLayer, but you would need to be on the teams plan because it sounds like you have a commercial product
yes, I'll talk with our team, But this is my research to find a suitable service for our problem.
in the interest of not getting external services to block railway, i will be asking the team to look into your use of the railway platform, no hard feelings
Okay, no problem. I'll contact you again further more assistant on this case.
hey @Malindu - whats the purpose? Research, what university?
I'm a kind of R&D developer, So, I'm researching a suitable service for the project.
I would suggest looking at their rate limiting docs. You likely want to use an existing SDK or wrapper that handles the rate limiting headers and backs off accordingly.
https://docs.commercelayer.io/core/rate-limits
Rate limits
Information about the restrictions on the number of times a user or client can access Commerce Layer API endpoints within a specified period of time.