web dev//backend//job queue
A job queue is a list of pending units of work (send this email, process this document, generate this report) that one part of a system writes and separate workers consume in the background, so the request that created the work can answer at once. It is the standard way a web backend keeps slow or unreliable work out of the user's request.
A job queue is a list of pending units of work (send this email, process this document, generate this report) that one part of a system writes and separate workers consume in the background, so the request that created the work can answer at once. It is the standard way a web backend keeps slow or unreliable work out of the user's request.
The picture is a restaurant ticket rail. The waiter clips the order and goes back to the tables; the cooks take tickets as they free up. A user who uploads a PDF gets received in 100 ms while a worker spends the next 40 seconds extracting it, and a spike of a thousand uploads becomes a longer rail instead of a thousand timed-out requests.
It decouples speed and failure. The producer never waits on the consumer, workers can be added when the rail grows, and a worker that crashes leaves its job to be picked up again. Managed services such as Inngest or a Redis-backed queue provide the rail; the event-driven API pattern is the same idea seen from the API side.
Retries make duplicates normal. A job whose worker died halfway, or whose acknowledgement was lost, runs again, so most queues promise at-least-once delivery. A job must therefore be idempotent (charging a card twice is the classic bug), and jobs that keep failing are set aside in a dead-letter queue for a person to inspect instead of retrying forever.
Order and timing are weaker than they look. Jobs can finish out of order, wait minutes under load, and pile up silently if the workers stop; a queue needs monitoring of its depth and age the way a tank needs a level gauge.
A queue turns a slow operation into a fast promise plus a background obligation.
The obligation is real: someone has to watch the rail, and every job has to survive being run twice.
A job queue is often confused with a cron job: the queue holds work created by events, the cron job creates work on a clock, and the two combine when a scheduled trigger fills the queue. Message brokers such as MQTT carry events to subscribers instead of work to workers.