feat: add job monitoring endpoint /backends/jobs #8095
Conversation
✅ Deploy Preview for localai ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
| var once sync.Once | ||
|
|
||
| //Return singleton instance of the JobStore | ||
| func GetStore() *JobStore{ |
There was a problem hiding this comment.
This looks basically superseded by https://github.com/mudler/LocalAI/blob/master/core/http/middleware/trace.go which is used to trace instead any requests (useful for dataset extraction).
@pmarini-nc, wouldn't the current tracing feature already satisfy your issue?
There was a problem hiding this comment.
@mudler I have analyzed core/http/middleware/trace.go.
While trace.go is excellent for historical analysis and debugging, it captures the APIExchange after the request handler returns (Lines 100-125).
Why this PR is distinct and necessary:
The goal of issue #7906 is to have a "top-like" monitoring view. The key requirement is to see jobs while they are executing (e.g., long-running inference or downloads).
Trace: Shows what happened (Past tense).
Job Monitor: Shows what is happening (Present tense).
If we use trace.go, the API user won't see the job until it is already finished, which defeats the purpose of monitoring live resource usage.
This PR inserts the job into the store before next(c) is called, allowing real-time visibility into running processes. I believe keeping this lightweight "Active Job Store" separate from the heavy "Historical Trace Buffer" is the cleanest approach.
There was a problem hiding this comment.
While I see your point, rather than having a new scaffolding and code for tracing in-flight jobs, it would be better to extend what we currently already have under a single API/Middleware to avoid duplication and reduce maintenance efforts.
There was a problem hiding this comment.
While I see your point, rather than having a new scaffolding and code for tracing in-flight jobs, it would be better to extend what we currently already have under a single API/Middleware to avoid duplication and reduce maintenance efforts.
@mudler , how can I test it? |
|
Helllo @mudler , I'd be interested in testing what you mention here, but I don't know how to test it. In any case @Divyanshupandey007 makes exactly the point, any chance to have this PR considered for merging? |
To test the current tracing functionality, you need to go to 'Trace' in the menu, enable tracing from the webui, and then you will be able to see the requests that have been processed. While I understand @Divyanshupandey007 point, I'd prefer we extend the current tracing functionality to be able to catch in-flight jobs rather than creating an entirely new endpoint/Middleware. This PR as-is introduces significant code/logic duplication which undermines maintenance on the long run. If @Divyanshupandey007 is willing to frame this on top of the current code I'd be happy to pick it up. |
Thanks! I'm able to see the traces 
I do have some suggestions for improvement, (not sure if anything changed in 3.11.0), but first: is it normal that transcript jobs are not traced? I just see text processing jobs there. |
3 participants
Description
This PR addresses issue #7906 by implementing a job tracking system. It allows users to monitor running AI jobs (like chat completions, transcriptions, etc.) via a new API endpoint.
Changes
core/http/jobs: Implemented a thread-safe in-memory store to track job status, start time, and metadata.JobTrackerMiddlewarewhich intercepts/v1/and/api/requests to automatically register them in the job store.GET /backends/jobswhich returns the list of active and recently finished jobs.Related Issue
Fixes #7906
Testing
curlby sending a POST request to/v1/chat/completions./backends/jobsreturns the job details with statusexecuting(during processing) andfinishedorerror(after completion).Sample Output:
[ { "job_id": "4cb9b834-d190-47dc-87b3-b8cfb70f6ab0", "type": "/v1/chat/completions", "model": "gpt-4", "start_time": "2026-01-18T17:09:18.72755107Z", "status": "executing", "client_ip": "4.240.39.193" } ]