PostgreSQL:Queue

lastmod 2026-04-22created 2026-04-203 edits

Keeping a Postgres queue healthy

• Keeping a Postgres queue healthy | GeekNews
  • [Original] Keeping a Postgres queue healthy — PlanetScale

A summary of the causes of, and fixes for, the chronic problems of running a job queue on Postgres: dead tuple buildup, the resulting table bloat, and performance degradation

• Most rows in a queue table cycle quickly through insert-read-delete, so the size stays constant but the cumulative throughput is very large
• Due to Postgres's MVCC design, deleted rows are not removed immediately; they remain as dead tuples that must be cleaned up, which is autovacuum's job
• When long-running transactions or overlapping analytics queries pin the MVCC horizon, autovacuum cannot clean up dead tuples and queue performance degrades
• PlanetScale's Traffic Control (an Insights extension) feature is presented as a practical way to solve this problem through per-query-class resource limits

Summary:

• The MVCC dead tuple problem in Postgres-based queues is not a relic of 2015
• Modern Postgres offers considerable headroom with B-tree improvements and SKIP LOCKED, but the underlying mechanism is the same
  • If VACUUM cannot clean up dead tuples, they accumulate
  • If long-running or overlapping transactions pin the MVCC horizon, VACUUM cannot clean up
• In environments that follow “Just use Postgres” and keep queues, analytics, and app logic in one DB, this is an everyday operating condition, not a theoretical risk
• The dangerous form is not a dramatic crash but a quietly degrading equilibrium — lock time slowly grows, jobs slow down, and no alarms go off
• Postgres's timeout tools cannot distinguish workload classes or limit concurrency
• If you run a queue alongside other workloads, the most effective measure is to ensure VACUUM can keep up, and Traffic Control simplifies this

See also

• PostgreSQL
• Job Queue
• MVCC
• Dead Tuple
• Autovacuum
• Database Performance
• Traffic Control
• PlanetScale