I’m currently working with the Groq Batch API for processing large-scale workloads and had a question regarding the rate limits for batch operations.The documentation mentions that the Batch API has separate rate limits from the synchronous API, but it doesn’t provide any detailed statistics on exact batch rate limits. Are there specific rate limits for batch processing, and if so, what are they?
Hi there! Batch processing rate limits are unlimited, but the speed at which they process is based on many other factors. For example, if you send a large batch workload with a 24h completion window, it may not complete in time and may expire. If you change it to 7d completion window, it will have a greater chance to complete.
Batch processing is unlimited, but does not guarantee completion, based on request size and current load on our services. I’ll update our docs to be more clear about this.
1 Like