CUDA:IPC

lastmod 2024-07-22created 2024-07-135 edits

Introduction - Interprocess Communication

• CUDA C++ Programming Guide # Interprocess Communication

Any device memory pointer or event handle created by a host thread can be directly referenced by any other thread within the same process. It is not valid outside this process however, and therefore cannot be directly referenced by threads belonging to a different process.

To share device memory pointers and events across processes, an application must use the Inter Process Communication API, which is described in detail in the reference manual. The IPC API is only supported for 64-bit processes on Linux and for devices of compute capability 2.0 and higher. Note that the IPC API is not supported for cudaMallocManaged allocations.

Using this API, an application can get the IPC handle for a given device memory pointer using cudaIpcGetMemHandle(), pass it to another process using standard IPC mechanisms (for example, interprocess shared memory or files), and use cudaIpcOpenMemHandle() to retrieve a device pointer from the IPC handle that is a valid pointer within this other process. Event handles can be shared using similar entry points.

Note that allocations made by cudaMalloc() may be sub-allocated from a larger block of memory for performance reasons. In such case, CUDA IPC APIs will share the entire underlying memory block which may cause other sub-allocations to be shared, which can potentially lead to information disclosure between processes. To prevent this behavior, it is recommended to only share allocations with a 2MiB aligned size.

An example of using the IPC API is where a single primary process generates a batch of input data, making the data available to multiple secondary processes without requiring regeneration or copying.

Applications using CUDA IPC to communicate with each other should be compiled, linked, and run with the same CUDA driver and runtime.

Korean translation

Every device memory pointer or event handle created by a host thread can be referenced directly from other threads within the same process. However, it is not valid outside this process, so it cannot be referenced directly from threads belonging to another process.

To share device memory pointers and events across processes, an application must use the interprocess communication API, which is described in detail in the reference manual. The IPC API is supported only for 64-bit processes on Linux and devices with compute capability 2.0 or higher. The IPC API is not supported for cudaMallocManaged allocations.

With this API, an application can obtain the IPC handle for a given device memory pointer using cudaIpcGetMemHandle(), can pass it to another process using standard IPC mechanisms (e.g. interprocess shared memory or files), and, using cudaIpcOpenMemHandle(), is the device pointer of the IPC handle that is a valid pointer within this other process. Event handles can be shared using similar entry points.

Allocations made by cudaMalloc() may be sub-allocated from a larger memory block for performance reasons. In such cases, the CUDA IPC API shares the entire underlying memory block, so other sub-allocations may be shared, which can potentially lead to information disclosure between processes. To prevent this behavior, it is recommended to share only allocations with a 2MiB-aligned size.

An example of using the IPC API is when a single primary process generates a batch of input data and makes the data available to multiple secondary processes without regeneration or copying.

Applications that communicate with each other using CUDA IPC must be compiled, linked, and run with the same CUDA driver and runtime.

APIs

• CUDA Runtime API :: CUDA Toolkit Documentation

__host__ ​cudaError_t cudaIpcCloseMemHandle ( void* devPtr )

Attempts to close memory mapped with cudaIpcOpenMemHandle.

__host__ ​cudaError_t cudaIpcGetEventHandle ( cudaIpcEventHandle_t* handle, cudaEvent_t event )

Gets an interprocess handle for a previously allocated event.

__host__ ​cudaError_t cudaIpcGetMemHandle ( cudaIpcMemHandle_t* handle, void* devPtr )

Gets an interprocess memory handle for an existing device memory allocation.

__host__ ​cudaError_t cudaIpcOpenEventHandle ( cudaEvent_t* event, cudaIpcEventHandle_t handle )

Opens an interprocess event handle for use in the current process.

__host__ ​cudaError_t cudaIpcOpenMemHandle ( void** devPtr, cudaIpcMemHandle_t handle, unsigned int flags )

Opens an interprocess memory handle exported from another process and returns a device pointer usable in the local process.

Example

• cuda-samples/Samples/0_Introduction/simpleIPC/simpleIPC.cu at master · NVIDIA/cuda-samples

See also

• CUDA

Favorite site

• CUDA C++ Programming Guide # Interprocess Communication