Class UnifiedMemoryManager

Namespace

DotCompute.Memory

Assembly

DotCompute.Memory.dll

Production-ready unified memory manager that consolidates all memory management functionality. This is the SINGLE source of truth for memory management in DotCompute.

Features:

• Memory pooling with 90% allocation reduction
• Automatic cleanup and defragmentation
• Cross-backend compatibility (CPU, CUDA, Metal, etc.)
• Production-grade error handling
• Comprehensive statistics and monitoring
• Thread-safe operations

public class UnifiedMemoryManager : BaseMemoryManager, IUnifiedMemoryManager, IAsyncDisposable, IDisposable

Inheritance

object

BaseMemoryManager

UnifiedMemoryManager

Implements

IUnifiedMemoryManager

IAsyncDisposable

IDisposable

Inherited Members

BaseMemoryManager.Logger

BaseMemoryManager.CurrentStatistics

BaseMemoryManager.TotalAllocatedBytes

BaseMemoryManager.PeakAllocatedBytes

BaseMemoryManager.AllocationCount

BaseMemoryManager.IsDisposed

BaseMemoryManager.AllocateAsync<T>(int, MemoryOptions, CancellationToken)

BaseMemoryManager.AllocateAndCopyAsync<T>(ReadOnlyMemory<T>, MemoryOptions, CancellationToken)

BaseMemoryManager.AllocateRawAsync(long, MemoryOptions, CancellationToken)

BaseMemoryManager.AllocateAsync(long, MemoryOptions, CancellationToken)

BaseMemoryManager.CreateView(IUnifiedMemoryBuffer, long, long)

BaseMemoryManager.AllocateAsync<T>(int)

BaseMemoryManager.CopyToDevice<T>(IUnifiedMemoryBuffer, ReadOnlySpan<T>)

BaseMemoryManager.CopyFromDevice<T>(Span<T>, IUnifiedMemoryBuffer)

BaseMemoryManager.Free(IUnifiedMemoryBuffer)

BaseMemoryManager.AllocateDevice(long)

BaseMemoryManager.FreeDevice(DeviceMemory)

BaseMemoryManager.MemsetDevice(DeviceMemory, byte, long)

BaseMemoryManager.MemsetDeviceAsync(DeviceMemory, byte, long, CancellationToken)

BaseMemoryManager.CopyHostToDevice(nint, DeviceMemory, long)

BaseMemoryManager.CopyDeviceToHost(DeviceMemory, nint, long)

BaseMemoryManager.CopyHostToDeviceAsync(nint, DeviceMemory, long, CancellationToken)

BaseMemoryManager.CopyDeviceToHostAsync(DeviceMemory, nint, long, CancellationToken)

BaseMemoryManager.CopyDeviceToDevice(DeviceMemory, DeviceMemory, long)

BaseMemoryManager.AllocateBufferCoreAsync(long, MemoryOptions, CancellationToken)

BaseMemoryManager.TrackBuffer(IUnifiedMemoryBuffer, long)

BaseMemoryManager.CleanupUnusedBuffers()

BaseMemoryManager.GetRefreshedStatistics()

BaseMemoryManager.ThrowIfDisposed()

BaseMemoryManager.Dispose()

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

UnifiedMemoryManagerExtensions.AllocatePinnedAsync<T>(IUnifiedMemoryManager, int, CancellationToken)

UnifiedMemoryManagerExtensions.AllocateUnifiedAsync<T>(IUnifiedMemoryManager, int, CancellationToken)

UnifiedMemoryManagerExtensions.GetAvailableMemory(IUnifiedMemoryManager)

UnifiedMemoryManagerExtensions.GetMemorySummary(IUnifiedMemoryManager)

UnifiedMemoryManagerExtensions.GetMemoryUtilizationPercentage(IUnifiedMemoryManager)

UnifiedMemoryManagerExtensions.GetTotalMemory(IUnifiedMemoryManager)

UnifiedMemoryManagerExtensions.GetUsedMemory(IUnifiedMemoryManager)

UnifiedMemoryManagerExtensions.IsUnderMemoryPressure(IUnifiedMemoryManager, double)

BufferExtensions.IsSafeToDispose(IDisposable)

BufferExtensions.SafeDispose(IDisposable)

Constructors

UnifiedMemoryManager(IAccelerator, ILogger?)

Initializes a new instance of the UnifiedMemoryManager class.

public UnifiedMemoryManager(IAccelerator accelerator, ILogger? logger = null)

Parameters

accelerator IAccelerator

The accelerator this memory manager is associated with.

logger ILogger

Logger instance for diagnostics.

UnifiedMemoryManager(ILogger?)

Creates a CPU-only memory manager for scenarios without GPU acceleration.

public UnifiedMemoryManager(ILogger? logger = null)

Parameters

logger ILogger

Logger instance for diagnostics.

Properties

Accelerator

Gets the accelerator this memory manager is associated with.

public override IAccelerator Accelerator { get; }

Property Value

IAccelerator

CurrentAllocatedMemory

Gets the current allocated memory in bytes.

public override long CurrentAllocatedMemory { get; }

Property Value

long

MaxAllocationSize

Gets the maximum memory allocation size in bytes.

public override long MaxAllocationSize { get; }

Property Value

long

Statistics

Gets memory usage statistics.

public override MemoryStatistics Statistics { get; }

Property Value

MemoryStatistics

TotalAvailableMemory

Gets the total available memory in bytes.

public override long TotalAvailableMemory { get; }

Property Value

long

Methods

AllocateInternalAsync(long, MemoryOptions, CancellationToken)

Allocates memory using backend-specific implementation.

protected override ValueTask<IUnifiedMemoryBuffer> AllocateInternalAsync(long sizeInBytes, MemoryOptions options, CancellationToken cancellationToken)

Parameters

sizeInBytes long

options MemoryOptions

cancellationToken CancellationToken

Returns

ValueTask<IUnifiedMemoryBuffer>

Clear()

Clears all allocated memory and resets the manager.

public override void Clear()

CopyAsync<T>(IUnifiedMemoryBuffer<T>, IUnifiedMemoryBuffer<T>, CancellationToken)

Copies data between buffers.

public override ValueTask CopyAsync<T>(IUnifiedMemoryBuffer<T> source, IUnifiedMemoryBuffer<T> destination, CancellationToken cancellationToken = default) where T : unmanaged

Parameters

source IUnifiedMemoryBuffer<T>

The source buffer.

destination IUnifiedMemoryBuffer<T>

The destination buffer.

cancellationToken CancellationToken

Cancellation token.

Returns

ValueTask

A task representing the copy operation.

Type Parameters

T

The element type.

CopyAsync<T>(IUnifiedMemoryBuffer<T>, int, IUnifiedMemoryBuffer<T>, int, int, CancellationToken)

Copies data between buffers with specified ranges.

public override ValueTask CopyAsync<T>(IUnifiedMemoryBuffer<T> source, int sourceOffset, IUnifiedMemoryBuffer<T> destination, int destinationOffset, int length, CancellationToken cancellationToken = default) where T : unmanaged

Parameters

source IUnifiedMemoryBuffer<T>

The source buffer.

sourceOffset int

The offset in the source buffer.

destination IUnifiedMemoryBuffer<T>

The destination buffer.

destinationOffset int

The offset in the destination buffer.

length int

cancellationToken CancellationToken

Cancellation token.

Returns

ValueTask

A task representing the copy operation.

Type Parameters

T

The element type.

CopyFromDeviceAsync<T>(IUnifiedMemoryBuffer<T>, Memory<T>, CancellationToken)

Copies data from a device buffer to host memory.

public override ValueTask CopyFromDeviceAsync<T>(IUnifiedMemoryBuffer<T> source, Memory<T> destination, CancellationToken cancellationToken = default) where T : unmanaged

Parameters

source IUnifiedMemoryBuffer<T>

The source buffer.

destination Memory<T>

The destination memory.

cancellationToken CancellationToken

Cancellation token.

Returns

ValueTask

A task representing the copy operation.

Type Parameters

T

The element type.

CopyToDeviceAsync<T>(ReadOnlyMemory<T>, IUnifiedMemoryBuffer<T>, CancellationToken)

Copies data from host memory to a device buffer.

public override ValueTask CopyToDeviceAsync<T>(ReadOnlyMemory<T> source, IUnifiedMemoryBuffer<T> destination, CancellationToken cancellationToken = default) where T : unmanaged

Parameters

source ReadOnlyMemory<T>

The source data.

destination IUnifiedMemoryBuffer<T>

The destination buffer.

cancellationToken CancellationToken

Cancellation token.

Returns

ValueTask

A task representing the copy operation.

Type Parameters

T

The element type.

CreateViewCore(IUnifiedMemoryBuffer, long, long)

Backend-specific view creation implementation.

protected override IUnifiedMemoryBuffer CreateViewCore(IUnifiedMemoryBuffer buffer, long offset, long length)

Parameters

buffer IUnifiedMemoryBuffer

offset long

length long

Returns

IUnifiedMemoryBuffer

CreateView<T>(IUnifiedMemoryBuffer<T>, int, int)

Creates a view over existing memory.

public override IUnifiedMemoryBuffer<T> CreateView<T>(IUnifiedMemoryBuffer<T> buffer, int offset, int length) where T : unmanaged

Parameters

buffer IUnifiedMemoryBuffer<T>

The source buffer.

offset int

The offset in elements.

length int

The length of the view in elements.

Returns

IUnifiedMemoryBuffer<T>

A view over the existing buffer.

Type Parameters

T

The element type.

Dispose(bool)

protected override void Dispose(bool disposing)

Parameters

disposing bool

DisposeAsync()

Gets dispose asynchronously.

public override ValueTask DisposeAsync()

Returns

ValueTask

The result of the operation.

FreeAsync(IUnifiedMemoryBuffer, CancellationToken)

Asynchronously frees a memory buffer.

public override ValueTask FreeAsync(IUnifiedMemoryBuffer buffer, CancellationToken cancellationToken = default)

Parameters

buffer IUnifiedMemoryBuffer

The buffer to free.

cancellationToken CancellationToken

Cancellation token.

Returns

ValueTask

A task representing the free operation.

GetStatisticsAsync(CancellationToken)

public virtual ValueTask<MemoryStatistics> GetStatisticsAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Returns

ValueTask<MemoryStatistics>

OptimizeAsync(CancellationToken)

Optimizes memory by defragmenting and releasing unused memory.

public override ValueTask OptimizeAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Cancellation token.

Returns

ValueTask

A task representing the optimization operation.