PATENT

Patent Abstract

Systems and Methods for Deterministic Gather of Hierarchically Managed Key-Value State for Neural Network Inference

Patent application 202641037509 in the Manish KL patent portfolio, covering kv cache systems and related technical systems.

202641037509 2026-03-27 KV Cache Systems
Full Patent Record India Filings Table

Summary

Overview

The core invention is deterministic gather of logically managed key-value state into execution-ready artifacts, supported by a residency-first hierarchical memory substrate.

Abstract

Technical Abstract

Key-value objects are given logical identities independent of physical placement and tracked across SRAM, HBM, host memory, remote memory, and storage-backed tiers. A deterministic gather engine resolves shared lineage, compiles a gather plan, overlaps retrieval across tiers, and emits execution-ready tiles or descriptors without exposing compute kernels to metadata complexity.

Search Context

SEO Keywords

KV cache patent, deterministic gather patent, transformer inference patent, key value state patent, memory hierarchy inference patent

Related Patents

More Patents in memory residency, KV systems, and deterministic inference

These filings sit nearby in the portfolio and strengthen internal linking across related patent topics.

Tiered Weight Memory

System and Method for Predictive Multi-Tier Weight Residency and Precision Orchestration for Neural-Network Inference

Neural-network weights are treated as live runtime state whose placement and precision can change across HBM, lower volatile tiers, and storage-backed tiers according to workload behavior.

 HBM Residency Management

System and Method for Confidence-Gated HBM Residency Management with Thrash-Budgeted Prefetch, Pinning, and Eviction Control

A confidence-gated HBM controller uses telemetry, uncertainty bounds, and thrash budgets to decide when speculative residency actions are safe.

 Deterministic Inference

Deterministic Memory-Orchestrated Neural Network Inference Using Scheduled DMA Transfers and Bounded Buffers

This filing turns DRAM into a deterministic streaming substrate by staging model data into bounded on-chip buffers ahead of compute.