Class CudaGraphSupport

Namespace

DotCompute.Backends.CUDA.Execution

Assembly

DotCompute.Backends.CUDA.dll

Advanced CUDA Graph support for kernel fusion and optimization on RTX 2000 Ada architecture. Provides comprehensive graph management, execution, and optimization capabilities.

public sealed class CudaGraphSupport : IDisposable

Inheritance

object

CudaGraphSupport

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

BufferExtensions.IsSafeToDispose(IDisposable)

BufferExtensions.SafeDispose(IDisposable)

Remarks

This class manages the complete lifecycle of CUDA graphs including creation, instantiation, execution, and optimization. It provides advanced features like kernel fusion, Ada Lovelace architecture optimizations, and performance monitoring for high-throughput computing scenarios.

Constructors

CudaGraphSupport(CudaContext, CudaStreamManager, CudaEventManager, ILogger<CudaGraphSupport>)

Initializes a new instance of the CudaGraphSupport class.

public CudaGraphSupport(CudaContext context, CudaStreamManager streamManager, CudaEventManager eventManager, ILogger<CudaGraphSupport> logger)

Parameters

context CudaContext

The CUDA context for graph operations.

streamManager CudaStreamManager

The stream manager for execution scheduling.

eventManager CudaEventManager

The event manager for timing and synchronization.

logger ILogger<CudaGraphSupport>

The logger for diagnostic information.

Exceptions

ArgumentNullException

Thrown when any parameter is null.

Methods

CaptureGraphAsync(string, Func<nint, Task>, CudaGraphCaptureMode, CancellationToken)

Captures a sequence of operations into a graph.

public Task<string> CaptureGraphAsync(string graphId, Func<nint, Task> operations, CudaGraphCaptureMode mode = CudaGraphCaptureMode.Global, CancellationToken cancellationToken = default)

Parameters

graphId string

The unique identifier for the captured graph.

operations Func<nint, Task>

A function that executes operations to be captured.

mode CudaGraphCaptureMode

The capture mode for the operation.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<string>

A task that represents the asynchronous capture operation, returning the graph ID.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.

CleanupUnusedInstances(TimeSpan)

Cleans up unused graph instances older than the specified age.

public int CleanupUnusedInstances(TimeSpan maxAge)

Parameters

maxAge TimeSpan

The maximum age for keeping unused instances.

Returns

int

The number of instances that were cleaned up.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.

CreateGraphAsync(string, IEnumerable<CudaKernelOperation>, CudaGraphOptimizationOptions?, CancellationToken)

Creates a CUDA graph from a sequence of kernel operations.

public Task<string> CreateGraphAsync(string graphId, IEnumerable<CudaKernelOperation> operations, CudaGraphOptimizationOptions? options = null, CancellationToken cancellationToken = default)

Parameters

graphId string

The unique identifier for the graph.

operations IEnumerable<CudaKernelOperation>

The sequence of kernel operations to include in the graph.

options CudaGraphOptimizationOptions

Optional optimization settings for the graph.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<string>

A task that represents the asynchronous graph creation operation, returning the graph ID.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.

Dispose()

Disposes of the graph support resources and cleans up all managed graphs and instances.

public void Dispose()

ExecuteGraphAsync(CudaGraphInstance, nint, CancellationToken)

Executes a graph instance.

public Task<CudaGraphExecutionResult> ExecuteGraphAsync(CudaGraphInstance instance, nint stream = 0, CancellationToken cancellationToken = default)

Parameters

instance CudaGraphInstance

The graph instance to execute.

stream nint

The CUDA stream for execution (default stream if not specified).

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<CudaGraphExecutionResult>

A task that represents the asynchronous execution operation, returning the execution results.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.

InvalidOperationException

Thrown if the graph instance is not valid.

FuseKernelsAsync(string, IEnumerable<CudaCompiledKernel>, CudaKernelFusionOptions, CancellationToken)

Fuses multiple kernels into a single optimized graph.

public Task<string> FuseKernelsAsync(string graphId, IEnumerable<CudaCompiledKernel> kernels, CudaKernelFusionOptions fusionOptions, CancellationToken cancellationToken = default)

Parameters

graphId string

The unique identifier for the fused graph.

kernels IEnumerable<CudaCompiledKernel>

The kernels to fuse together.

fusionOptions CudaKernelFusionOptions

Configuration options for the fusion operation.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<string>

A task that represents the asynchronous fusion operation, returning the graph ID.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.

GetGraphStatistics(string)

Gets performance statistics for a graph.

public CudaGraphStatistics GetGraphStatistics(string graphId)

Parameters

graphId string

The identifier of the graph for which to retrieve statistics.

Returns

CudaGraphStatistics

A CudaGraphStatistics object containing performance metrics.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.

InstantiateGraphAsync(string, CancellationToken)

Instantiates a graph for execution.

public Task<CudaGraphInstance> InstantiateGraphAsync(string graphId, CancellationToken cancellationToken = default)

Parameters

graphId string

The identifier of the graph to instantiate.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<CudaGraphInstance>

A task that represents the asynchronous instantiation operation, returning the graph instance.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.

ArgumentException

Thrown if the specified graph is not found.

UpdateGraphAsync(CudaGraphInstance, CudaGraphUpdateParameters, CancellationToken)

Updates a graph instance with new parameters.

public Task<bool> UpdateGraphAsync(CudaGraphInstance instance, CudaGraphUpdateParameters updateParams, CancellationToken cancellationToken = default)

Parameters

instance CudaGraphInstance

The graph instance to update.

updateParams CudaGraphUpdateParameters

The parameters for the update operation.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<bool>

A task that represents the asynchronous update operation, returning true if successful.

Exceptions

ObjectDisposedException

Thrown if the object has been disposed.