Nq
lastmod 2025-11-11created 2025-11-112 edits
Unix command-line queue tool
About
- A utility for implementing a lightweight job queue system that works without any configuration or daemon
- Supports Linux/MacOS - runs on POSIX-compatible systems that support flock(2)
- Supports command-line-based ad-hoc queuing
- Suited to sequential jobs such as Makefile builds, file downloads, and running benchmarks
- Can also be used as an alternative to nohup
- Creates a TIMESTAMP.PID for each process and flocks it (file lock)
- ㅤ- Starts the next job once the preceding flocks are unlocked
- Synchronization is done at the file system level, so no polling is needed
- STDOUT/STDERR are redirected to a log file
- The execute bit (+x) on the log file visually distinguishes job status
- Before execution it is nq's PID; after execution it changes to the actual job's PID
- A job can be rerun by executing the log file like a shell script
Differences from at, batch, and task-spooler
at
- A tool that runs jobs at a specified time
- Runs commands at scheduled times based on the system clock; mainly used for one-off scheduled jobs
- batch
- Runs jobs when the system load (load average) is low
- Aims at load-based execution control rather than scheduling, and is managed by a daemon
task-spooler
- Automatically starts a daemon that manages the queue, supporting sequential or parallel job processing
- The maximum number of jobs to run concurrently can be specified
- Different queues can be set up per terminal, and the status of each queue can be queried with a command
nq
- Works without a daemon, managing the queue simply with directory-level file system locks (flock)
- Simply guarantees job order without considering system load or scheduled times
- The queue's logs can be checked in real time with nqtail
- Each directory acts as an independent queue, so multiple queues can be run separately using the $NQDIR environment variable