Class CudaMemoryBuffer<T>

Namespace

DotCompute.Backends.CUDA.Memory

Assembly

DotCompute.Backends.CUDA.dll

Represents a generic CUDA memory buffer allocated on the GPU device.

public sealed class CudaMemoryBuffer<T> : IUnifiedMemoryBuffer<T>, IUnifiedMemoryBuffer, IAsyncDisposable, IDisposable where T : unmanaged

Type Parameters

T

The element type stored in the buffer.

Inheritance

object

CudaMemoryBuffer<T>

Implements

IUnifiedMemoryBuffer<T>

IUnifiedMemoryBuffer

IAsyncDisposable

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

BufferExtensions.IsSafeToDispose(IDisposable)

BufferExtensions.SafeDispose(IDisposable)

Remarks

Initializes a new instance of the CudaMemoryBuffer<T> class.

Constructors

CudaMemoryBuffer(nint, long, CudaContext, MemoryOptions, IAccelerator?)

Represents a generic CUDA memory buffer allocated on the GPU device.

public CudaMemoryBuffer(nint devicePointer, long count, CudaContext context, MemoryOptions options = MemoryOptions.None, IAccelerator? accelerator = null)

Parameters

devicePointer nint

The device memory pointer.

count long

The number of elements.

context CudaContext

The CUDA context.

options MemoryOptions

Memory allocation options.

accelerator IAccelerator

Optional accelerator reference for advanced features.

Remarks

Initializes a new instance of the CudaMemoryBuffer<T> class.

Properties

Accelerator

Gets the accelerator this buffer is associated with.

public IAccelerator Accelerator { get; }

Property Value

IAccelerator

Count

public long Count { get; }

Property Value

long

DevicePointer

Gets the device memory pointer.

public nint DevicePointer { get; }

Property Value

nint

IsDirty

Gets whether the buffer has been modified and needs synchronization.

public bool IsDirty { get; }

Property Value

bool

IsDisposed

Gets whether the buffer has been disposed.

public bool IsDisposed { get; }

Property Value

bool

IsOnDevice

Gets whether the buffer is currently available on the device.

public bool IsOnDevice { get; }

Property Value

bool

IsOnHost

Gets whether the buffer is currently available on the host.

public bool IsOnHost { get; }

Property Value

bool

Length

Gets the number of elements in the buffer.

public int Length { get; }

Property Value

int

Options

Gets the memory allocation options.

public MemoryOptions Options { get; }

Property Value

MemoryOptions

SizeInBytes

Gets the size of the buffer in bytes.

public long SizeInBytes { get; }

Property Value

long

State

Gets the current state of the buffer.

public BufferState State { get; }

Property Value

BufferState

Methods

AsMemory()

Gets a memory handle to the host memory. Will trigger transfer if needed.

public Memory<T> AsMemory()

Returns

Memory<T>

AsReadOnlyMemory()

Gets a read-only memory handle to the host memory. Will trigger transfer if needed.

public ReadOnlyMemory<T> AsReadOnlyMemory()

Returns

ReadOnlyMemory<T>

AsReadOnlySpan()

Gets a read-only span to the host memory. Will trigger transfer if needed.

public ReadOnlySpan<T> AsReadOnlySpan()

Returns

ReadOnlySpan<T>

AsSpan()

Gets a span to the host memory. Will trigger transfer if needed.

public Span<T> AsSpan()

Returns

Span<T>

AsType<TNew>()

Creates a view of this buffer with a different element type.

public IUnifiedMemoryBuffer<TNew> AsType<TNew>() where TNew : unmanaged

Returns

IUnifiedMemoryBuffer<TNew>

A view of this buffer as the new type.

Type Parameters

TNew

The new element type.

ClearDirty()

Clears the dirty flag and all dirty ranges.

public void ClearDirty()

CopyFromAsync(ReadOnlyMemory<T>, CancellationToken)

Copies data from a source memory into this buffer.

public ValueTask CopyFromAsync(ReadOnlyMemory<T> source, CancellationToken cancellationToken = default)

Parameters

source ReadOnlyMemory<T>

cancellationToken CancellationToken

Returns

ValueTask

CopyFromAsync<TSource>(ReadOnlyMemory<TSource>, long, CancellationToken)

Asynchronously copies data from host memory to this buffer.

public ValueTask CopyFromAsync<TSource>(ReadOnlyMemory<TSource> source, long offset = 0, CancellationToken cancellationToken = default) where TSource : unmanaged

Parameters

source ReadOnlyMemory<TSource>

offset long

cancellationToken CancellationToken

Returns

ValueTask

Type Parameters

TSource

CopyFromHost(ReadOnlySpan<T>)

Copies data from host memory to this device buffer.

public void CopyFromHost(ReadOnlySpan<T> source)

Parameters

source ReadOnlySpan<T>

CopyRangeTo(CudaMemoryBuffer<T>, long, long, long)

Copies a range of data from this buffer to another device buffer.

public void CopyRangeTo(CudaMemoryBuffer<T> destination, long sourceOffset, long destinationOffset, long count)

Parameters

destination CudaMemoryBuffer<T>

The destination buffer.

sourceOffset long

The offset in the source buffer (in elements).

destinationOffset long

The offset in the destination buffer (in elements).

count long

The number of elements to copy.

CopyRangeToAsync(CudaMemoryBuffer<T>, long, long, long, nint, CancellationToken)

Asynchronously copies a range of data from this buffer to another device buffer.

public ValueTask CopyRangeToAsync(CudaMemoryBuffer<T> destination, long sourceOffset, long destinationOffset, long count, nint stream = 0, CancellationToken cancellationToken = default)

Parameters

destination CudaMemoryBuffer<T>

sourceOffset long

destinationOffset long

count long

stream nint

cancellationToken CancellationToken

Returns

ValueTask

CopyTo(CudaMemoryBuffer<T>)

Copies data from this buffer to another buffer.

public void CopyTo(CudaMemoryBuffer<T> destination)

Parameters

destination CudaMemoryBuffer<T>

CopyToAsync(IUnifiedMemoryBuffer<T>, CancellationToken)

Copies data from this buffer to another buffer.

public ValueTask CopyToAsync(IUnifiedMemoryBuffer<T> destination, CancellationToken cancellationToken = default)

Parameters

destination IUnifiedMemoryBuffer<T>

cancellationToken CancellationToken

Returns

ValueTask

CopyToAsync(int, IUnifiedMemoryBuffer<T>, int, int, CancellationToken)

Copies data from this buffer to another buffer with specified ranges.

public ValueTask CopyToAsync(int sourceOffset, IUnifiedMemoryBuffer<T> destination, int destinationOffset, int count, CancellationToken cancellationToken = default)

Parameters

sourceOffset int

destination IUnifiedMemoryBuffer<T>

destinationOffset int

count int

cancellationToken CancellationToken

Returns

ValueTask

CopyToAsync(Memory<T>, CancellationToken)

Copies data from this buffer to a destination memory.

public ValueTask CopyToAsync(Memory<T> destination, CancellationToken cancellationToken = default)

Parameters

destination Memory<T>

cancellationToken CancellationToken

Returns

ValueTask

CopyToAsync<TDest>(Memory<TDest>, long, CancellationToken)

Asynchronously copies data from this buffer to host memory.

public ValueTask CopyToAsync<TDest>(Memory<TDest> destination, long offset = 0, CancellationToken cancellationToken = default) where TDest : unmanaged

Parameters

destination Memory<TDest>

offset long

cancellationToken CancellationToken

Returns

ValueTask

Type Parameters

TDest

CopyToHost(Span<T>)

Copies data from this device buffer to host memory.

public void CopyToHost(Span<T> destination)

Parameters

destination Span<T>

Dispose()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

public void Dispose()

DisposeAsync()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources asynchronously.

public ValueTask DisposeAsync()

Returns

ValueTask

A task that represents the asynchronous dispose operation.

EnsureOnDevice()

Ensures the buffer is available on the device. Triggers transfer if needed.

public void EnsureOnDevice()

EnsureOnDeviceAsync(AcceleratorContext, CancellationToken)

Asynchronously ensures the buffer is available on the device.

public ValueTask EnsureOnDeviceAsync(AcceleratorContext context = default, CancellationToken cancellationToken = default)

Parameters

context AcceleratorContext

cancellationToken CancellationToken

Returns

ValueTask

EnsureOnHost()

Ensures the buffer is available on the host. Triggers transfer if needed.

public void EnsureOnHost()

EnsureOnHostAsync(AcceleratorContext, CancellationToken)

Asynchronously ensures the buffer is available on the host.

public ValueTask EnsureOnHostAsync(AcceleratorContext context = default, CancellationToken cancellationToken = default)

Parameters

context AcceleratorContext

cancellationToken CancellationToken

Returns

ValueTask

FillAsync(T, int, int, CancellationToken)

Fills a portion of this buffer with a specified value.

public ValueTask FillAsync(T value, int offset, int count, CancellationToken cancellationToken = default)

Parameters

value T

offset int

count int

cancellationToken CancellationToken

Returns

ValueTask

FillAsync(T, CancellationToken)

Fills this buffer with a specified value.

public ValueTask FillAsync(T value, CancellationToken cancellationToken = default)

Parameters

value T

cancellationToken CancellationToken

Returns

ValueTask

GetDeviceMemory()

Gets the device memory handle. Will trigger transfer if needed.

public DeviceMemory GetDeviceMemory()

Returns

DeviceMemory

GetDirtyRanges()

Gets the dirty ranges in the buffer.

public (long start, long end)[] GetDirtyRanges()

Returns

(long start, long end)[]

An array of dirty ranges (start, end) where end is exclusive.

Map(MapMode)

Maps this buffer to host memory for direct access.

public MappedMemory<T> Map(MapMode mode = MapMode.ReadWrite)

Parameters

mode MapMode

The mapping mode.

Returns

MappedMemory<T>

A mapped memory region.

MapAsync(MapMode, CancellationToken)

Asynchronously maps this buffer to host memory.

public ValueTask<MappedMemory<T>> MapAsync(MapMode mode = MapMode.ReadWrite, CancellationToken cancellationToken = default)

Parameters

mode MapMode

The mapping mode.

cancellationToken CancellationToken

Cancellation token.

Returns

ValueTask<MappedMemory<T>>

A task that returns the mapped memory region.

MapRange(int, int, MapMode)

Maps a portion of this buffer to host memory for direct access.

public MappedMemory<T> MapRange(int offset, int length, MapMode mode = MapMode.ReadWrite)

Parameters

offset int

The offset to start mapping at.

length int

The number of elements to map.

mode MapMode

The mapping mode.

Returns

MappedMemory<T>

A mapped memory region.

MarkDeviceDirty()

Marks the buffer as modified on the device.

public void MarkDeviceDirty()

MarkDirty()

Marks the entire buffer as dirty.

public void MarkDirty()

MarkHostDirty()

Marks the buffer as modified on the host.

public void MarkHostDirty()

MarkRangeDirty(long, long)

Marks a specific range of the buffer as dirty.

public void MarkRangeDirty(long start, long end)

Parameters

start long

The start index (inclusive).

end long

The end index (exclusive).

Slice(int, int)

Creates a slice of this buffer.

public IUnifiedMemoryBuffer<T> Slice(int offset, int length)

Parameters

offset int

The offset in elements.

length int

The length of the slice in elements.

Returns

IUnifiedMemoryBuffer<T>

A slice of this buffer.

SyncDirtyToHostAsync<TDest>(Memory<TDest>, CancellationToken)

Synchronizes only the dirty ranges from device to host.

public ValueTask SyncDirtyToHostAsync<TDest>(Memory<TDest> destination, CancellationToken cancellationToken = default) where TDest : unmanaged

Parameters

destination Memory<TDest>

cancellationToken CancellationToken

Returns

ValueTask

Type Parameters

TDest

Synchronize()

Synchronizes the buffer state between host and device.

public void Synchronize()

SynchronizeAsync(AcceleratorContext, CancellationToken)

Asynchronously synchronizes the buffer state between host and device.

public ValueTask SynchronizeAsync(AcceleratorContext context = default, CancellationToken cancellationToken = default)

Parameters

context AcceleratorContext

cancellationToken CancellationToken

Returns

ValueTask