Skip to content

Marvell Photonic Fabric Memory: How 32TB Could Be Shared Across AI Servers

Marvell's Photonic Fabric architecture aims to let up to 16 AI hosts share 32TB of DDR5 and more than 1TB of HBM3e. Here's how its optical memory design works, why the KV cache matters, and what remains unproven before customer sampling in 2027.

Marvell Photonic Fabric Memory: How 32TB Could Be Shared Across AI Servers

On this page

Marvell has shown a memory module designed to let artificial-intelligence servers share up to 32 terabytes of conventional memory without putting all of it beside a graphics processor. The Photonic Fabric Memory Module uses optical connections, which carry data as light, to link memory resources across multiple systems. The distinction matters: the 32TB figure describes a proposed appliance assembled from 16 modules, not a single module or a product customers can order today. Marvell is targeting customer samples in 2027, leaving performance, pricing and production readiness to be proven.

Marvell's 32TB figure belongs to an entire memory appliance

Each Photonic Fabric Memory Module combines up to eight double data rate 5 (DDR5) memory modules, 72GB of high-bandwidth memory (HBM3e), a custom application-specific integrated circuit and optical input/output rated at 7.2 terabits per second. HBM3e is a high-speed memory technology commonly placed close to AI accelerators, while DDR5 offers a way to provide much larger capacity at a different performance and cost balance. Marvell's proposed appliance connects 16 of its modules, creating a pool of up to 32TB of DDR5 and 1,152GB of HBM3e that can be accessed by as many as 16 hosts, meaning connected servers or accelerator systems. Those are architectural figures for the proposed configuration, not independently measured results from a shipping system.

The capacity distinction is easy to miss in headline numbers. One module does not contain 32TB of DDR5: the full appliance reaches that total by combining 16 modules, each populated with eight 256GB memory modules. The HBM3e total is 1,152GB, or roughly 1.1TB, rather than a precise 1TB. This arrangement puts a smaller, faster memory tier in front of a much larger pool, aiming to give AI workloads more room without requiring every extra unit of memory to be installed beside another expensive accelerator.

Optical links could loosen the connection between memory and processors

In conventional accelerator systems, HBM sits extremely close to the processor because moving data at very high speeds over electrical connections becomes difficult as distances increase. Marvell has described the gap between a processor and its attached HBM as roughly 1.5 millimetres. That tight physical arrangement delivers the bandwidth modern accelerators need, but it also ties the amount of local HBM to the accelerator package and its design. If a workload needs more compute and more memory, a buyer can end up paying for additional processors even when memory capacity is the more immediate constraint.

Photonic Fabric is intended to change that trade-off by using light to carry data between memory modules and connected hosts. Optical links can move high-speed signals over longer distances than conventional electrical connections without the same signal-loss limitations. Marvell's technical material describes a shared-memory architecture spanning multiple racks, with its published distance figures varying across materials: one company blog says up to 30 metres, while a later announcement refers to links reaching 50 metres. Those numbers should not be treated as interchangeable guarantees for every configuration; actual reach depends on the system design and optical components used.

The immediate target is an AI model's working memory, not ordinary PC RAM

The technology is aimed at a particular problem in AI inference, the stage when a trained model generates responses. Large language models maintain a key-value cache, usually shortened to KV cache, that stores intermediate information from earlier tokens so the model does not have to recalculate the same context repeatedly. Longer conversations, larger batches of users and more complex reasoning can make that cache grow substantially. When the available high-speed memory fills up, systems must move data elsewhere or limit how much context they can keep readily available, potentially reducing efficiency.

A shared pool could give a cluster access to more of this working data without requiring every accelerator to carry all the memory itself. Marvell argues that its architecture could improve accelerator utilisation and increase token throughput by two to three times in suitable deployments. That is a company performance claim, not a result established by independent testing of a commercially available appliance. The size of any benefit will depend on the model, the cache workload, how data is distributed and whether the optical memory tier can serve requests quickly enough to keep accelerators busy.

The 72GB HBM cache is meant to keep the larger pool useful

Putting terabytes of memory in a separate appliance would have limited value if every access were too slow for an accelerator to use effectively. Marvell's design therefore pairs the larger DDR5 capacity with 72GB of HBM3e on each module. The HBM acts as a faster tier for frequently needed data, while the larger DDR5 pool provides additional capacity for information that must remain accessible but does not always need to sit in the fastest memory. This resembles the broader principle behind a computer's cache and main memory, although the scale, connection method and access patterns are different.

Marvell claims memory access latency below 350 nanoseconds for its near-NUMA architecture. Non-uniform memory access (NUMA) describes systems in which the time needed to reach memory depends on where that memory sits relative to a processor. The claim matters because a shared memory tier must offer both capacity and predictable access times to be useful for AI workloads. However, a specification or company estimate is not the same as a measured result under production conditions, and buyers will need independent tests to understand how latency behaves when multiple hosts request data at once.

Sampling in 2027 leaves cost and real-world performance unanswered

Marvell's October demonstration brought fresh attention to technology it had already described in August 2026. During its October investor presentation, company executives said the design had been taped out earlier in the year and was being brought up, with samples for multiple customers expected in 2027. That is a development milestone, not a retail launch date. Marvell has not announced a public price for the full appliance, and it has not provided independently verified production benchmarks that establish how much the architecture will save customers.

The economics are central to the pitch. Separating memory from compute could let cloud providers add capacity without purchasing an equivalent amount of accelerator hardware, potentially improving how much useful work they get from each server. But the system also needs optical components, custom silicon, memory modules, compatible host connections and software capable of using the shared pool. Those costs and integration requirements could offset some of the savings, especially if a workload gains little from keeping its cache in remote memory. Without transparent system pricing and workload-level tests, the claim of lower overall costs remains a possibility rather than a demonstrated outcome.

Marvell faces a wider race to make AI memory more flexible

Marvell is not the only company trying to separate memory capacity from the processor that uses it. Compute Express Link (CXL), an industry-standard connection for sharing and expanding memory, is already used in designs that pool memory across servers. Marvell also sells its Structera family of CXL memory-expansion products, while other hardware vendors are pursuing their own ways to manage AI context data and expand memory capacity. Photonic Fabric's differentiator is its use of optical connectivity to reach across multiple racks while targeting the bandwidth and latency demands of AI inference.

For now, the most useful way to read the 32TB announcement is as a proposed infrastructure design rather than a ready-made solution to AI memory costs. The capacity numbers describe what the assembled appliance is designed to hold; the throughput improvements and efficiency benefits remain claims that need to survive real workloads. The next evidence worth watching is working silicon, customer sampling in 2027, and independent measurements that compare latency, power use and total system cost against conventional accelerator servers. Until those results arrive, the architecture is promising, but the practical advantage is still unproven.

Copy this post

0 Comments

No comments yet. Be the first to share your thoughts.

Join the conversation

Log in or create a free account to leave a comment. You can edit or delete your own comments any time.