Node-exporter

lastmod 2025-12-02created 2025-12-027 edits

Exporter for machine metrics

Running with Docker

docker run -d \
  --net="host" \
  --pid="host" \
  -v "/:/host:ro,rslave" \
  quay.io/prometheus/node-exporter:latest \
  --path.rootfs=/host

Compose

---
version: '3.8'

services:
  node_exporter:
    image: quay.io/prometheus/node-exporter:latest
    container_name: node_exporter
    command:
      - '--path.rootfs=/host'
    network_mode: host
    pid: host
    restart: unless-stopped
    volumes:
      - '/:/host:ro,rslave'

Collectors

Enabled by default

Name	Description	OS
arp	Exposes ARP statistics from `/proc/net/arp`.	Linux
bcache	Exposes bcache statistics from `/sys/fs/bcache/`.	Linux
bonding	Exposes the number of configured and active slaves of Linux bonding interfaces.	Linux
btrfs	Exposes btrfs statistics.	Linux
boottime	Exposes system boot time derived from the `kern.boottime` sysctl.	Darwin, Dragonfly, FreeBSD, NetBSD, OpenBSD, Solaris
conntrack	Shows conntrack statistics (does nothing if no `/proc/sys/net/netfilter/` present).	Linux
cpu	Exposes CPU statistics.	Darwin, Dragonfly, FreeBSD, Linux, Solaris, OpenBSD
cpufreq	Exposes CPU frequency statistics.	Linux, Solaris
diskstats	Exposes disk I/O statistics.	Darwin, Linux, OpenBSD
dmi	Exposes Desktop Management Interface (DMI) info from `/sys/class/dmi/id/`.	Linux
edac	Exposes error detection and correction statistics.	Linux
entropy	Exposes available entropy.	Linux
exec	Exposes execution statistics.	Dragonfly, FreeBSD
fibrechannel	Exposes Fibre Channel information and statistics from `/sys/class/fc_host/`.	Linux
filefd	Exposes file descriptor statistics from `/proc/sys/fs/file-nr`.	Linux
filesystem	Exposes file system statistics, such as disk space used.	Darwin, Dragonfly, FreeBSD, Linux, OpenBSD
hwmon	Exposes hardware monitoring and sensor data from `/sys/class/hwmon/`.	Linux
infiniband	Exposes network statistics specific to InfiniBand and Intel OmniPath configurations.	Linux
ipvs	Exposes IPVS status from `/proc/net/ip_vs` and stats from `/proc/net/ip_vs_stats`.	Linux
loadavg	Exposes load average.	Darwin, Dragonfly, FreeBSD, Linux, NetBSD, OpenBSD, Solaris
mdadm	Exposes statistics about devices in `/proc/mdstat` (does nothing if no `/proc/mdstat` present).	Linux
meminfo	Exposes memory statistics.	Darwin, Dragonfly, FreeBSD, Linux, OpenBSD
netclass	Exposes network interface info from `/sys/class/net/`.	Linux
netdev	Exposes network interface statistics such as bytes transferred.	Darwin, Dragonfly, FreeBSD, Linux, OpenBSD
netisr	Exposes netisr statistics.	FreeBSD
netstat	Exposes network statistics from `/proc/net/netstat`. This is the same information as `netstat -s`.	Linux
nfs	Exposes NFS client statistics from `/proc/net/rpc/nfs`. This is the same information as `nfsstat -c`.	Linux
nfsd	Exposes NFS kernel server statistics from `/proc/net/rpc/nfsd`. This is the same information as `nfsstat -s`.	Linux
nvme	Exposes NVMe info from `/sys/class/nvme/`.	Linux
os	Exposes OS release info from `/etc/os-release` or `/usr/lib/os-release`.	any
powersupplyclass	Exposes power supply statistics from `/sys/class/power_supply`.	Linux
pressure	Exposes pressure stall statistics from `/proc/pressure/`.	Linux (kernel 4.20+ and/or CONFIG_PSI)
rapl	Exposes various statistics from `/sys/class/powercap`.	Linux
schedstat	Exposes task scheduler statistics from `/proc/schedstat`.	Linux
selinux	Exposes SELinux statistics.	Linux
sockstat	Exposes various statistics from `/proc/net/sockstat`.	Linux
softnet	Exposes statistics from `/proc/net/softnet_stat`.	Linux
stat	Exposes various statistics from `/proc/stat`. This includes boot time, forks and interrupts.	Linux
tapestats	Exposes statistics from `/sys/class/scsi_tape`.	Linux
textfile	Exposes statistics read from local disk. The `--collector.textfile.directory` flag must be set.	any
thermal	Exposes thermal statistics like `pmset -g therm`.	Darwin
thermal_zone	Exposes thermal zone & cooling device statistics from `/sys/class/thermal`.	Linux
time	Exposes the current system time.	any
timex	Exposes selected `adjtimex(2)` system call stats.	Linux
udp_queues	Exposes UDP total lengths of the `rx_queue` and `tx_queue` from `/proc/net/udp` and `/proc/net/udp6`.	Linux
uname	Exposes system information as provided by the `uname` system call.	Darwin, FreeBSD, Linux, OpenBSD
vmstat	Exposes statistics from `/proc/vmstat`.	Linux
watchdog	Exposes statistics from `/sys/class/watchdog`.	Linux
xfs	Exposes XFS runtime statistics.	Linux (kernel 4.4+)
zfs	Exposes ZFS performance statistics.	FreeBSD, Linux, Solaris

Disabled by default

Name	Description	OS
buddyinfo	Exposes statistics of memory fragments as reported by `/proc/buddyinfo`.	Linux
cgroups	Provides a summary of the number of active and enabled cgroups.	Linux
cpu_vulnerabilities	Exposes CPU vulnerability information from sysfs.	Linux
devstat	Exposes device statistics.	Dragonfly, FreeBSD
drm	Exposes GPU metrics using sysfs / DRM. amdgpu is the only driver which exposes this information through DRM.	Linux
drbd	Exposes Distributed Replicated Block Device statistics (up to version 8.4).	Linux
ethtool	Exposes network interface information and network driver statistics equivalent to `ethtool`, `ethtool -S`, and `ethtool -i`.	Linux
interrupts	Exposes detailed interrupts statistics.	Linux, OpenBSD
kernel_hung	Exposes the number of tasks that have been detected as hung from `/proc/sys/kernel/hung_task_detect_count`.	Linux
ksmd	Exposes kernel and system statistics from `/sys/kernel/mm/ksm`.	Linux
lnstat	Exposes stats from `/proc/net/stat/`.	Linux
logind	Exposes session counts from logind.	Linux
meminfo_numa	Exposes memory statistics from `/sys/devices/system/node/node[0-9]/meminfo`, `/sys/devices/system/node/node[0-9]/numastat`.	Linux
mountstats	Exposes file system statistics from `/proc/self/mountstats`. Exposes detailed NFS client statistics.	Linux
network_route	Exposes the routing table as metrics.	Linux
pcidevice	Exposes PCI device information, including link status and parent devices.	Linux
perf	Exposes perf-based metrics (Warning: metrics depend on kernel configuration and settings).	Linux
processes	Exposes aggregate process statistics from `/proc`.	Linux
qdisc	Exposes queuing discipline statistics.	Linux
slabinfo	Exposes slab statistics from `/proc/slabinfo`. Note that permission of `/proc/slabinfo` is usually 0400, so set it appropriately.	Linux
softirqs	Exposes detailed softirq statistics from `/proc/softirqs`.	Linux
sysctl	Expose sysctl values from `/proc/sys`. Use `--collector.sysctl.include(-info)` to configure.	Linux
swap	Expose swap information from `/proc/swaps`.	Linux
systemd	Exposes service and system status from systemd.	Linux
tcpstat	Exposes TCP connection status information from `/proc/net/tcp` and `/proc/net/tcp6` (Warning: the current version has potential performance issues in high load situations).	Linux
wifi	Exposes WiFi device and station statistics.	Linux
xfrm	Exposes statistics from `/proc/net/xfrm_stat`.	Linux
zoneinfo	Exposes NUMA memory zone metrics.	Linux

PromQL Examples

CPU

# 1. CPU 사용률 (%) - 전체 평균
100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

# 2. CPU 사용률 - 코어별
100 - (rate(node_cpu_seconds_total{mode="idle"}[5m]) * 100)

# 3. CPU 사용률 - mode별 분류
sum by (mode, instance) (rate(node_cpu_seconds_total[5m])) * 100

# 4. System CPU 사용률
rate(node_cpu_seconds_total{mode="system"}[5m]) * 100

# 5. User CPU 사용률
rate(node_cpu_seconds_total{mode="user"}[5m]) * 100

# 6. I/O Wait 시간
rate(node_cpu_seconds_total{mode="iowait"}[5m]) * 100

# 7. CPU 코어 개수
count(node_cpu_seconds_total{mode="idle"}) by (instance)

Memory

# 1. 메모리 사용률 (%)
(1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100

# 2. 메모리 사용량 (bytes)
node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes

# 3. 메모리 사용량 (GB)
(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / 1024 / 1024 / 1024

# 4. Swap 사용률 (%)
(1 - (node_memory_SwapFree_bytes / node_memory_SwapTotal_bytes)) * 100

# 5. Swap 사용량 (bytes)
node_memory_SwapTotal_bytes - node_memory_SwapFree_bytes

# 6. 캐시 메모리 (bytes)
node_memory_Cached_bytes + node_memory_Buffers_bytes

# 7. 사용 가능한 메모리 (%)
(node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100

# 8. Free 메모리 (GB)
node_memory_MemFree_bytes / 1024 / 1024 / 1024

# 9. 메모리 사용률 - 인스턴스별
avg by (instance) ((1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100)

# 10. 메모리 압박 상태 (Available memory < 10%)
(node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100 < 10

Network

# 1. 네트워크 수신 속도 (bytes/sec)
rate(node_network_receive_bytes_total[5m])

# 2. 네트워크 송신 속도 (bytes/sec)
rate(node_network_transmit_bytes_total[5m])

# 3. 네트워크 수신 속도 (MB/sec)
rate(node_network_receive_bytes_total[5m]) / 1024 / 1024

# 4. 네트워크 송신 속도 (MB/sec)
rate(node_network_transmit_bytes_total[5m]) / 1024 / 1024

# 5. 네트워크 총 트래픽 (bytes/sec)
rate(node_network_receive_bytes_total[5m]) + rate(node_network_transmit_bytes_total[5m])

# 6. 네트워크 에러 수신
rate(node_network_receive_errs_total[5m])

# 7. 네트워크 에러 송신
rate(node_network_transmit_errs_total[5m])

# 8. 네트워크 패킷 수신 속도 (packets/sec)
rate(node_network_receive_packets_total[5m])

# 9. 네트워크 패킷 송신 속도 (packets/sec)
rate(node_network_transmit_packets_total[5m])

# 10. 네트워크 드롭 패킷 수신
rate(node_network_receive_drop_total[5m])

# 11. 네트워크 드롭 패킷 송신
rate(node_network_transmit_drop_total[5m])

# 12. 특정 인터페이스 수신 속도 (eth0)
rate(node_network_receive_bytes_total{device="eth0"}[5m])

# 13. 특정 인터페이스 송신 속도 (eth0)
rate(node_network_transmit_bytes_total{device="eth0"}[5m])

# 14. loopback 제외 네트워크 수신 속도
rate(node_network_receive_bytes_total{device!="lo"}[5m])

# 15. 인터페이스별 네트워크 사용량 (MB/sec)
sum by (device) (rate(node_network_receive_bytes_total[5m]) + rate(node_network_transmit_bytes_total[5m])) / 1024 / 1024

# 16. 네트워크 에러율 (%)
(rate(node_network_receive_errs_total[5m]) / rate(node_network_receive_packets_total[5m])) * 100

# 17. 네트워크 대역폭 사용률 (1Gbps 기준)
(rate(node_network_transmit_bytes_total[5m]) * 8 / 1000000000) * 100

Filesystem

# 1. 디스크 사용률 (%)
(1 - (node_filesystem_avail_bytes / node_filesystem_size_bytes)) * 100

# 2. 디스크 사용률 - 마운트 포인트별
(1 - (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"})) * 100

# 3. 디스크 사용 공간 (bytes)
node_filesystem_size_bytes - node_filesystem_avail_bytes

# 4. 디스크 사용 공간 (GB)
(node_filesystem_size_bytes - node_filesystem_avail_bytes) / 1024 / 1024 / 1024

# 5. 사용 가능한 디스크 공간 (GB)
node_filesystem_avail_bytes / 1024 / 1024 / 1024

# 6. 전체 디스크 크기 (GB)
node_filesystem_size_bytes / 1024 / 1024 / 1024

# 7. tmpfs, devtmpfs 제외한 디스크 사용률
(1 - (node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"} / node_filesystem_size_bytes{fstype!~"tmpfs|devtmpfs"})) * 100

# 8. 특정 파일시스템 타입만 (ext4, xfs)
(1 - (node_filesystem_avail_bytes{fstype=~"ext4|xfs"} / node_filesystem_size_bytes{fstype=~"ext4|xfs"})) * 100

# 9. 디스크 여유 공간 10GB 미만인 마운트 포인트
node_filesystem_avail_bytes / 1024 / 1024 / 1024 < 10

# 10. 디스크 사용률 90% 이상인 파일시스템
(1 - (node_filesystem_avail_bytes / node_filesystem_size_bytes)) * 100 > 90

# 11. inode 사용률 (%)
(1 - (node_filesystem_files_free / node_filesystem_files)) * 100

# 12. 사용 가능한 inode 개수
node_filesystem_files_free

# 13. 전체 inode 개수
node_filesystem_files

# 14. inode 사용률 80% 이상
(1 - (node_filesystem_files_free / node_filesystem_files)) * 100 > 80

# 15. Read-only 파일시스템 감지
node_filesystem_readonly == 1

# 16. 인스턴스별 디스크 사용률 평균
avg by (instance) ((1 - (node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"} / node_filesystem_size_bytes{fstype!~"tmpfs|devtmpfs"})) * 100)

# 17. 마운트 포인트별 디스크 사용률 정렬 (높은 순)
sort_desc((1 - (node_filesystem_avail_bytes / node_filesystem_size_bytes)) * 100)

# 18. /var 디렉토리 사용률
(1 - (node_filesystem_avail_bytes{mountpoint="/var"} / node_filesystem_size_bytes{mountpoint="/var"})) * 100

# 19. 디스크 여유 공간 퍼센트
(node_filesystem_avail_bytes / node_filesystem_size_bytes) * 100

# 20. 특정 디바이스 사용률 (예: /dev/sda1)
(1 - (node_filesystem_avail_bytes{device="/dev/sda1"} / node_filesystem_size_bytes{device="/dev/sda1"})) * 100

# 21. 로컬 파일시스템만 (NFS, CIFS 제외)
(1 - (node_filesystem_avail_bytes{fstype!~"nfs|cifs|tmpfs|devtmpfs"} / node_filesystem_size_bytes{fstype!~"nfs|cifs|tmpfs|devtmpfs"})) * 100

# 22. 마운트 포인트 개수
count(node_filesystem_size_bytes)

# 23. 디스크 사용량 총합 (GB)
sum(node_filesystem_size_bytes - node_filesystem_avail_bytes) / 1024 / 1024 / 1024

# 24. 디스크 여유 공간이 가장 적은 파일시스템 (GB)
bottomk(1, node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"} / 1024 / 1024 / 1024)

# 25. 디스크 사용률이 가장 높은 파일시스템
topk(5, (1 - (node_filesystem_avail_bytes / node_filesystem_size_bytes)) * 100)

# 26. 디스크 사용률 예측 (4시간 후)
predict_linear(node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"}[1h], 4 * 3600) / node_filesystem_size_bytes * 100

# 27. 디스크 사용률 증가율 (1시간 기준)
rate(node_filesystem_size_bytes{fstype!~"tmpfs|devtmpfs"}[1h]) - rate(node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"}[1h])

# 28. 마운트 포인트가 루트(/)인 디스크 사용률
(1 - (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"})) * 100

# 29. 모든 파일시스템의 총 크기 (TB)
sum(node_filesystem_size_bytes{fstype!~"tmpfs|devtmpfs"}) / 1024 / 1024 / 1024 / 1024

# 30. 특정 임계값 이하의 여유 공간 (예: 5GB)
node_filesystem_avail_bytes{fstype!~"tmpfs|devtmpfs"} / 1024 / 1024 / 1024 < 5

Alertmanager example

# prometheus_rules.yml
groups:
  - name: cpu_alerts
    interval: 30s
    rules:
      # High CPU usage alert
      - alert: HighCPUUsage
        expr: 100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High CPU usage detected on {{ $labels.instance }}"
          description: "CPU usage is above 80% (current value: {{ $value }}%)"

      # Critical CPU usage alert  
      - alert: CriticalCPUUsage
        expr: 100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 95
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Critical CPU usage on {{ $labels.instance }}"
          description: "CPU usage is above 95% (current value: {{ $value }}%)"

      # High I/O wait alert
      - alert: HighIOWait
        expr: rate(node_cpu_seconds_total{mode="iowait"}[5m]) * 100 > 20
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High I/O wait on {{ $labels.instance }}"
          description: "I/O wait is above 20% (current value: {{ $value }}%)"

      # High system CPU usage
      - alert: HighSystemCPU
        expr: rate(node_cpu_seconds_total{mode="system"}[5m]) * 100 > 30
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High system CPU on {{ $labels.instance }}"
          description: "System CPU usage is above 30% (current value: {{ $value }}%)"

Grafana Pannel example

{
  "targets": [
    {
      "expr": "100 - (avg by (instance) (rate(node_cpu_seconds_total{mode=\"idle\"}[5m])) * 100)",
      "legendFormat": "{{instance}}"
    }
  ],
  "title": "CPU Usage (%)",
  "type": "graph"
}

Grafana Dashboard example

# Python으로 Prometheus 쿼리 실행 예시
import requests
from datetime import datetime

def query_prometheus(query, prometheus_url="http://localhost:9090"):
    """
    Prometheus에서 CPU 메트릭 조회
    Query CPU metrics from Prometheus
    """
    params = {
        'query': query,
        'time': datetime.now().isoformat()
    }

    response = requests.get(
        f"{prometheus_url}/api/v1/query",
        params=params
    )

    return response.json()

# 사용 예시 / Usage example
if __name__ == "__main__":
    # CPU 사용률 조회 / Query CPU usage
    cpu_usage_query = '100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)'
    result = query_prometheus(cpu_usage_query)

    # 결과 출력 / Print results
    for item in result['data']['result']:
        instance = item['metric']['instance']
        cpu_usage = float(item['value'][1])
        print(f"Instance: {instance}, CPU Usage: {cpu_usage:.2f}%")

prometheus.yml example

scrape_configs:
  - job_name: 'node-exporter'
    scrape_interval: 15s  # 수집 간격 / Scrape interval
    scrape_timeout: 10s   # 타임아웃 / Timeout
    static_configs:
      - targets:
          - 'localhost:9100'
          - 'server1:9100'
          - 'server2:9100'
        labels:
          env: 'production'

Favorite site

author Zer0 sitemap

Node-exporter

Running with Docker

Compose

Collectors

Enabled by default

Disabled by default

PromQL Examples

CPU

Memory

Network

Filesystem

Alertmanager example

Grafana Pannel example

Grafana Dashboard example

prometheus.yml example

See also

Favorite site