Class CudaMemoryPrefetcher

Namespace

DotCompute.Backends.CUDA.Memory

Assembly

DotCompute.Backends.CUDA.dll

Manages memory prefetching for unified memory to optimize data movement. Uses cudaMemPrefetchAsync to proactively move data between host and device.

public sealed class CudaMemoryPrefetcher : IDisposable

Inheritance

object

CudaMemoryPrefetcher

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

BufferExtensions.IsSafeToDispose(IDisposable)

BufferExtensions.SafeDispose(IDisposable)

Constructors

CudaMemoryPrefetcher(CudaContext, CudaDevice, ILogger)

Initializes a new instance of the CudaMemoryPrefetcher class.

public CudaMemoryPrefetcher(CudaContext context, CudaDevice device, ILogger logger)

Parameters

context CudaContext

The context.

device CudaDevice

The device.

logger ILogger

The logger.

Properties

PrefetchCount

Gets the total number of prefetch operations.

public long PrefetchCount { get; }

Property Value

long

PrefetchHitRate

Gets the prefetch hit rate.

public double PrefetchHitRate { get; }

Property Value

double

Statistics

Gets prefetch statistics.

public PrefetchStatistics Statistics { get; }

Property Value

PrefetchStatistics

SupportsPrefetch

Gets whether the device supports memory prefetching.

public bool SupportsPrefetch { get; }

Property Value

bool

TotalPrefetchedBytes

Gets the total number of bytes prefetched.

public long TotalPrefetchedBytes { get; }

Property Value

long

Methods

AdviseMemoryAsync(nint, long, CudaMemoryAdvise, int, CancellationToken)

Advises CUDA about the expected access pattern for memory.

public Task<bool> AdviseMemoryAsync(nint devicePointer, long sizeInBytes, CudaMemoryAdvise advice, int deviceId = -1, CancellationToken cancellationToken = default)

Parameters

devicePointer nint

sizeInBytes long

advice CudaMemoryAdvise

deviceId int

cancellationToken CancellationToken

Returns

Task<bool>

BatchPrefetchAsync(PrefetchRequest[], CancellationToken)

Prefetches multiple memory regions in batch.

public Task<int> BatchPrefetchAsync(PrefetchRequest[] requests, CancellationToken cancellationToken = default)

Parameters

requests PrefetchRequest[]

cancellationToken CancellationToken

Returns

Task<int>

Dispose()

Performs dispose.

public void Dispose()

PrefetchToDeviceAsync(nint, long, int, nint, CancellationToken)

Prefetches memory to the specified device asynchronously.

public Task<bool> PrefetchToDeviceAsync(nint ptr, long sizeInBytes, int deviceId = -1, nint stream = 0, CancellationToken cancellationToken = default)

Parameters

ptr nint

sizeInBytes long

deviceId int

stream nint

cancellationToken CancellationToken

Returns

Task<bool>

PrefetchToHostAsync(nint, long, nint, CancellationToken)

Prefetches memory to the host CPU asynchronously.

public Task<bool> PrefetchToHostAsync(nint ptr, long sizeInBytes, nint stream = 0, CancellationToken cancellationToken = default)

Parameters

ptr nint

sizeInBytes long

stream nint

cancellationToken CancellationToken

Returns

Task<bool>

RecordPrefetchHit(nint)

Records a prefetch hit (data was used as expected).

public void RecordPrefetchHit(nint devicePointer)

Parameters

devicePointer nint

RecordPrefetchMiss(nint)

Records a prefetch miss (data was not where expected).

public void RecordPrefetchMiss(nint devicePointer)

Parameters

devicePointer nint

WaitForPrefetchesAsync(CancellationToken)

Waits for all pending prefetch operations to complete.

public Task WaitForPrefetchesAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Returns

Task