Non-uniform memory access

lastmod 2024-08-19created 2017-01-2210 edits

Non-Uniform Memory Access (NUMA) is one of the computer memory design methods used in multiprocessor systems, in which the memory access time depends on the relative location of the memory and the processor. In a NUMA architecture, a processor accesses its own local memory faster than remote memory. Remote memory refers to memory attached to another processor, and local memory refers to memory attached to the processor itself.

The NUMA architecture is theoretically a concept extended from the SMP architecture. It was first commercialized by the Burroughs large systems, and later commercialized by Unisys, Convex Computer (later Hewlett-Packard), Silicon Graphics, Sequent Computer Systems, Data General, and DEC. The technologies developed by these companies were adopted in Unix-like operating systems, and some were also adopted in Windows NT.

Basic concept

Modern CPUs run much faster than main memory. However, in the early days of computing, CPUs generally ran slower than memory. This was reversed in the 1960s with the development of the first supercomputers and high-speed computers. Since then, CPUs, which needed to use ever more data, had to wait until all the data was fetched from memory. Many supercomputers developed in the 1980s and 1990s provided ways to access memory quickly, which allowed them to process large data sets faster than any other systems.

Reducing the number of memory accesses is one way to increase performance in modern computers. Examples include steadily increasing high-speed cache memory and using sophisticated algorithms to reduce cache misses. However, the dramatic growth in the size of operating systems and of the applications running on them overwhelms the performance gains from caches. The problem is especially severe in multiprocessor systems, because only one processor can access memory at a time, so many other processors have to wait.

NUMA attempts to solve this problem by providing each processor with its own separate memory. This avoids the performance hit when multiple processors try to access the same memory address. For problems involving spread-out data, which are very common in servers and applications, NUMA can improve performance over a single shared memory roughly by the number of processors or the number of memory banks. However, when more than one processor needs the same data, a NUMA system may have to move data between memory banks. This operation degrades NUMA performance. Therefore, overall performance depends on the nature of the tasks running at a given time. Getting processes to use local memory instead of remote memory is a matter of program optimization.

Automatic numa balancing

• (Linux) Improving multi-core CPU performance through OS parameter NUMA settings

Shows multiple nodes:

$ numactl --hardware

Shows NUMA in the flags:

$ cat /sys/kernel/debug/sched_features

To disable automatic NUMA balancing, use the following command:

$ echo 0 > /proc/sys/kernel/numa_balancing

To enable automatic NUMA balancing, use the following command:

$ echo 1 > /proc/sys/kernel/numa_balancing

Manual configuration

Using numactl

numactl is a utility for setting NUMA policies.

numactl --cpunodebind=1 --membind=1 python your_script.py

• --cpunodebind=1 - pins the process to the CPUs of NUMA node 1.
• --membind=1 - makes the process allocate the memory it uses only from NUMA node 1.

Using taskset

taskset is used to set CPU affinity.

taskset -c 1 python your_script.py

• -c 1 - makes it run only on CPU 1.

Both methods can pin a Python script to specific CPUs when running it. numactl allows more fine-grained configuration in that it can also pin memory.

Setting environment variables

You can also set environment variables before running Python to optimize performance in a NUMA environment.

export NUMEXPR_MAX_THREADS=1
export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1

These settings can improve performance when multithreading libraries are used.

See also

• CPU

Favorite stie

• Wikipedia (en) Non-uniform memory access
• Red Hat Enterprise Linux - 6.8 - Performance Tuning Guide - Chapter 4. CPU
• Namuwiki - NUMA
• [https://technet.microsoft.com/ko-kr/library/ms178144(v=sql.105.aspx MSDN - Understanding NUMA (Non-Uniform Memory Access)]

MS Windows

• [https://msdn.microsoft.com/ko-kr/library/windows/desktop/aa363804(v=vs.85.aspx MSDN - NUMA Support]
• [https://msdn.microsoft.com/ko-kr/library/windows/desktop/aa965223(v=vs.85.aspx MSDN - Allocating Memory from a NUMA Node]
• [https://msdn.microsoft.com/ko-kr/library/windows/desktop/ms684251(v=vs.85.aspx MSDN - Multiple Processors]

Linux

• NUMA with Linux ¹
• Work for running experiments on NUMA (uses numactl, numastat, taskset)
• [Recommended] NUMA, a new world of memory management ²

References