Storage bottlenecks on Oracle Cloud Infrastructure (OCI) can now be mitigated as WEKA rolls out its NeuralMesh offerings on the Oracle Marketplace, offering native deployment, unified billing through Oracle Universal Credits, and specialized architectures designed to accelerate AI training and scale inference token generation.
Deploying NeuralMesh Axon for GPU Training Clusters
The economics of production AI depend on how much useful work an organization extracts from its underlying infrastructure. Training teams constantly face the challenge of keeping expensive GPU clusters saturated with datasets and checkpoints. Storage constraints and slow metadata lookups often throttle these pipelines, forcing accelerators to sit idle while waiting for data.
📡 Telecom networks are generating more data than ever. WEKA NeuralMesh helps get that data to GPUs faster, so teams can run more AI without adding more infrastructure.> 95% GPU utilization. Up to 86% lower infrastructure costs. <1ms latency.📄 https://t.co/MLluZqKSGk pic.twitter.com/WVhXiyGmRd
NeuralMesh Axon addresses this by running directly on OCI GPU instances and utilizing local NVMe capacity. This design places high-performance storage physically close to the compute layer rather than isolating it in a separate silo. The Center for AI Safety (CAIS) deployed NeuralMesh Axon on its OCI compute environment after facing severe metadata and storage bottlenecks that inflated dataset copy costs and constrained its growing research community.
CAIS reported a 90% reduction in cloud storage costs and an estimated fivefold improvement in storage performance compared to its prior solution. These efficiency gains allowed its active researcher count to expand from roughly 30 to more than 200. These metrics reflect specific results from the CAIS deployment rather than guaranteed outcomes for every distinct workflow.
Scaling Inference Context with Augmented Memory Grid
Inference workloads present an entirely different architectural hurdle. Modern applications handling long conversations or agentic workflows must retain prior context within a key-value (KV) cache to avoid repeating expensive computations. When that cache exceeds available memory, the system evicts useful context and forces recomputation, which subsequently caps throughput as user demand scales.
WEKA designed NeuralMesh with Augmented Memory Grid to bypass this limitation by establishing a persistent, shared cache backed by NeuralMesh-managed NVMe storage. Joint testing conducted on a nine-node OCI cluster equipped with 72 H100 GPUs and 100,000-token inputs demonstrated distinct performance thresholds. The Augmented Memory Grid configuration supported more than 5,000 concurrent users under those parameters. By comparison, a DRAM-only setup on the exact same GPU footprint topped out at about 600 concurrent users.
Token throughput scaled similarly under the test conditions. The OCI and WEKA joint testing reached roughly two million tokens per second, while the DRAM-only baseline hovered below 200,000 tokens per second. In a separate one-hour test involving 2,400 active users, the shared cache architecture served five billion tokens, contrasted with 700 million tokens for the baseline configuration.
Streamlining Procurement Through the Oracle Marketplace
Beyond raw performance benchmarks, Oracle and WEKA are streamlining the commercial integration between the two platforms. Oracle customers can now apply eligible Oracle Universal Credits toward WEKA offerings deployed directly within their OCI tenancies. This integration removes procurement friction, providing a clearer pathway from software selection to unified billing.
Engineering teams can evaluate these offerings based on their specific technical requirements. Organizations focusing on training data ingestion, checkpoint frequency, and raw storage efficiency can assess NeuralMesh Axon against local NVMe availability. Meanwhile, engineering groups architecting long-context LLM applications can measure KV-cache behavior, token throughput, and concurrency targets against the published OCI benchmarks.
To begin the deployment process, technical stakeholders can consult with their Oracle account teams or WEKA representatives. Discussions typically center on data shape and size, the underlying GPU environment, expected user growth, and concrete performance metrics such as delivered tokens per GPU and overall cost per token.
