Modernize unmanaged allocation: Marshal.AllocHGlobal → NativeMemory

[Nucs]

Overview

Replace legacy Marshal.AllocHGlobal/FreeHGlobal with the modern NativeMemory API (.NET 6+) across all unmanaged allocation sites, enabling aligned allocation for future SIMD vectorization and zero-initialized allocation for np.zeros.

Problem

All unmanaged memory allocation in NumSharp goes through Marshal.AllocHGlobal/FreeHGlobal at 5 call sites in 2 files:

File	Call	Purpose
UnmanagedMemoryBlock1.cs:31`	Marshal.AllocHGlobal(new IntPtr(bytes))	Primary array allocation
UnmanagedMemoryBlock1.cs:995`	Marshal.FreeHGlobal(Address)	Deallocation in Disposer
StackedMemoryPool.cs:90	Marshal.AllocHGlobal(SingleSize)	Pool overflow allocation
StackedMemoryPool.cs:169	Marshal.FreeHGlobal(addr)	Pool cleanup
StackedMemoryPool.cs:238	individualyAllocated.ForEach(Marshal.FreeHGlobal)	Pool disposal

Marshal.AllocHGlobal wraps LocalAlloc on Windows (Win32 legacy) and malloc on Unix — no alignment guarantees beyond platform default (8 or 16 bytes), no zero-init option, and IntPtr return type requires casting.

Proposal

Replace with equivalent NativeMemory calls:

// Drop-in replacement:
var ptr = (IntPtr)NativeMemory.Alloc((nuint)bytes);
NativeMemory.Free((void*)ptr);

// Aligned (enables future SIMD):
var ptr = (IntPtr)NativeMemory.AlignedAlloc((nuint)bytes, alignment: 32);
NativeMemory.AlignedFree((void*)ptr);

// Zero-initialized (optimized np.zeros):
var ptr = (IntPtr)NativeMemory.AllocZeroed((nuint)bytes);

• Replace Marshal.AllocHGlobal → NativeMemory.Alloc in UnmanagedMemoryBlock1.cs`
• Replace Marshal.FreeHGlobal → NativeMemory.Free in Disposer
• Replace alloc/free in StackedMemoryPool.cs (3 sites)
• Update AllocationType enum if needed (new variant or replace AllocHGlobal wholesale)
• Add AllocZeroed fast path for np.zeros / np.zeros_like
• Add allocation benchmarks to NumSharp.Benchmark
• Verify all tests pass

Evidence

• NativeMemory.AlignedAlloc allows 32-byte (AVX2) or 64-byte (AVX-512) alignment — prerequisite for SIMD vectorization of arithmetic loops
• NativeMemory.AllocZeroed delegates to calloc / OS zero-page mapping — potentially faster than Alloc + manual Unsafe.InitBlock
• NativeMemory.Alloc returns void* directly, avoiding IntPtr round-trip in a codebase that immediately casts to T*
• The Disposer class already dispatches on AllocationType enum — clean extension point

Scope / Non-goals

• In scope: Replace 5 allocation sites, add benchmarks, optional AllocZeroed fast path
• Not in scope: SIMD vectorization of arithmetic loops (separate effort), changing StackedMemoryPool pooling strategy, NativeMemory.AlignedRealloc

Benchmark / Performance

Must benchmark before merging. The allocation hot path affects every NDArray creation.

Benchmark	What to measure
Allocation throughput	NativeMemory.Alloc vs Marshal.AllocHGlobal at small (<1KB), medium (1KB-1MB), large (>1MB) sizes
Aligned overhead	AlignedAlloc(32) vs Alloc for the same sizes
Zero-init	AllocZeroed vs Alloc + Unsafe.InitBlock / Span.Clear
Pool interaction	StackedMemoryPool.Take/Return with both APIs
End-to-end	np.arange(N), np.zeros(N), a + b for representative array sizes

Breaking changes

None — internal implementation detail, no public API changes.

Related issues

• Upgrade target frameworks from netstandard2.0 to net8.0;net10.0 #531 — .NET 8/10 TFM upgrade (prerequisite — NativeMemory requires net6.0+)

[Nucs]

Related NEPs: See docs/neps/NEP49.md - NEP 49 covers NumPy's data allocation strategies and custom allocator API patterns.

[Nucs]

Benchmark Results: NativeMemory Allocation Modernization

Ran comprehensive benchmarks comparing allocation strategies on Ubuntu 24.04, .NET 10.0, Intel i9-13900K.

RoundTrip Performance (alloc + free)

Size	Marshal.AllocHGlobal	NativeMemory.Alloc	AlignedAlloc(32)	AllocZeroed
64B	9.5ns	10ns	10ns	24ns
1KB	9.6ns	9.8ns	10ns	40ns
100KB	29ns	34ns	35ns	1,170ns
10MB	25ns	25ns	33ns	196,000ns

Key Findings

1. NativeMemory.Alloc ≈ Marshal.AllocHGlobal — identical performance, cleaner API
2. AlignedAlloc(32) — only ~8ns overhead, enables SIMD alignment
3. AllocZeroed is a trap — 1000x slower for large arrays (actually writes zeros vs lazy page allocation)

Recommendation

Simple replacement, no threshold strategy needed:

// Replace:
Marshal.AllocHGlobal(bytes)
Marshal.FreeHGlobal(ptr)

// With:
NativeMemory.Alloc((nuint)bytes)
NativeMemory.Free(ptr)

Do NOT use:

• AlignedAlloc — requires AlignedFree (easy to mismatch, causes crashes)
• AllocZeroed — deceptively slow for large arrays

Affected Sites (from issue description)

File	Line	Change
UnmanagedMemoryBlock\1.cs`	31	Marshal.AllocHGlobal → NativeMemory.Alloc
UnmanagedMemoryBlock\1.cs`	995	Marshal.FreeHGlobal → NativeMemory.Free
StackedMemoryPool.cs	90	Marshal.AllocHGlobal → NativeMemory.Alloc
StackedMemoryPool.cs	169	Marshal.FreeHGlobal → NativeMemory.Free
StackedMemoryPool.cs	238	ForEach(Marshal.FreeHGlobal) → ForEach(NativeMemory.Free)

Benchmark code: benchmark/NumSharp.Benchmark.GraphEngine/Benchmarks/Allocation/

[Nucs]

Implementation Complete ✅

NativeMemory modernization has been implemented in commit c8ddfd65.

Changes

File	Change
UnmanagedMemoryBlock\1.cs`	Marshal.AllocHGlobal → NativeMemory.Alloc
UnmanagedMemoryBlock\1.cs`	Marshal.FreeHGlobal → NativeMemory.Free
StackedMemoryPool.cs	Marshal.AllocHGlobal → NativeMemory.Alloc
StackedMemoryPool.cs	Marshal.FreeHGlobal → NativeMemory.Free (3 sites)

Benchmark Results (from earlier analysis)

Size	Marshal	NativeMemory
64B	9.5ns	10ns
10MB	25ns	25ns

Performance is identical within noise margin. This modernizes the codebase to use the cleaner .NET 6+ API.

Included in PR #573.

[Nucs]

NEP 49 Analysis: What Applies to NumSharp

After reviewing NEP 49 (Data Allocation Strategies) and NumSharp's architecture, here's what's relevant:

✅ Already Done

Feature	Status
Modern allocation API	NativeMemory.Alloc/Free (commit c8ddfd65)
Memory pooling	ScalarMemoryPool, StackedMemoryPool exist
View/ownership tracking	_baseStorage, OWNDATA flag
Custom dispose callbacks	External allocation type in Disposer

⚪ Worth Considering

Feature	Rationale
NativeMemory.AllocZeroed	May be faster than Alloc + fill for np.zeros. Worth benchmarking.
Memory statistics	Simple allocated/freed counters for debugging. Low priority.

❌ Not Applicable to NumSharp

Aligned Allocation (NativeMemory.AlignedAlloc)

The issue mentions 32/64-byte alignment for SIMD. In practice:

• Default alignment (16 bytes) is sufficient for Vector128
• Vector256 unaligned loads have negligible penalty on modern CPUs (Haswell+)
• .NET's Vector256.Load() uses vmovups (unaligned) — it just works
• The complexity isn't justified by measurable gains

Pluggable Allocators (NEP 49's main feature)

NEP 49 allows NumPy users to swap allocators for:

• GPU memory — NumSharp uses raw T* pointers throughout. GPU memory (CUDA/OpenCL) lives in a separate address space and is not dereferenceable from CPU code. You can't just swap an allocator — you'd need a completely different backend architecture (like TensorFlow.NET).
• NUMA pinning — .NET doesn't expose NUMA APIs in a useful way
• Huge pages — Linux-specific, requires OS configuration, extremely niche
• Memory profiling — .NET has its own profiling ecosystem

The fundamental problem: NumSharp's architecture assumes CPU-accessible memory via pointer arithmetic. NEP 49's allocator abstraction doesn't help because the operations themselves would need to change, not just allocation.

Recommendation

Close this issue once AllocZeroed is benchmarked (if beneficial). The NEP 49 "pluggable allocator" pattern is architecturally incompatible with NumSharp's pointer-based design and provides no practical value.

---

Related: docs/neps/NEP49.md already marked this as "LOW" impact. This analysis confirms that assessment.