Measuring Inference Runtime Over Multiple Hours on Groq LPU?

Forum
1

Hey everyone,

I’m testing a continuous AI workload on a Groq LPU and want to monitor inference stability and latency trends over long sessions, ideally for 6 to 12 hours straight.

Is there a Groq-native way (or external tool) to track inference time consistency and resource usage over hours? I’m currently logging timestamps manually around each inference call, but I imagine there’s a more elegant way.

Any suggestions, scripts, or best practices for tracking runtime performance over extended periods would be greatly appreciated!

Thanks,
Jhonn Mick

15

Hi @jhonnmarie,

We don’t currently have a Groq-native way to track inference usage aside from what we have implemented in console; would using observability tools like Arize help?