DeepSeek-V4.1-Flash is built around a practical problem: agents spend enormous amounts of time reading. Tool outputs, documents, code, and conversation history keep growing, making prompt processing and context storage increasingly expensive.

V4.1 reduces that cost from several directions: it processes most input tokens through fewer layers, shares memory across layers, searches that memory selectively, and stores it at lower precision. Alongside these changes, it adds learned lookup memory, faster speculative decoding, and training that emphasizes verifiable agent tasks.

The model has 552B backbone parameters plus 196B Engram parameters. DeepSeek reports 8B parameters activated per token during prefill and 16B during decode, with a global KV cache of 890 bytes per token. Keep those three numbers separate, total capacity, active computation, and cached context: the architecture choices below all trade among them.

1. Causal Encoder–Decoder: make reading cheaper

The Causal Encoder–Decoder (CED) splits the backbone into a 20-layer causal encoder and a 20-layer decoder. The encoder builds contextual representations, and the decoder projects its global key-value memory directly from those representations.

In a conventional Transformer, each layer creates its KV cache from its own hidden states. Building the full cache therefore requires processing the prompt through the entire stack. CED allows most prompt tokens to stop after the encoder.

In simpler terms

Imagine reading a large codebase before answering a question. The encoder prepares a detailed set of notes. The decoder can consult those notes without first repeating all the processing for every line of code.

For a decoder layer that creates global memory:

\[C_l = E W_l^{KV},\]

where $E$ is the encoder’s final representation and $W_l^{KV}$ projects it into global memory. The encoder is causal: its representation at a position only uses information available up to that position.

Local sliding-window attention still needs each decoder layer’s own states. The decoder therefore processes the final 128 prompt tokens to initialize this local memory. Each token generated after that traverses both halves.

Why it matters

For a long prompt, the approximate layer-computation cost changes from:

\[NL \quad\text{to}\quad N\frac{L}{2}+W\frac{L}{2},\]

with $N$ prompt tokens, $L=40$ layers, and window $W=128$. This approaches half the original work when $N$ is large. Actual latency also depends on attention, projections, and hardware.

CED builds on YOCO’s separation of memory construction from consumption, while retaining layer-specific local attention.

2. CSA2: share memory without copying the answer

Compressed Sparse Attention 2 (CSA2) combines local sliding-window attention with selectively accessed global memory. Its central change is separating three operations: creating memory, selecting positions, and attending to those positions.

In simpler terms

Think of global memory as a collection of records:

• Main KV holds the information the model reads.
• Indexer K provides compact keys for searching those records.
• Top-K indices identify which records to read.

Layers can share the records, the selected addresses, or both. They still use their own queries to interpret the selected information.

Mode	Global KV and indexer keys	Selection of positions	Attention output
Full	Create new memory and keys	Compute fresh Top-K	Compute fresh output
Reindex	Reuse memory and keys	Compute fresh Top-K	Compute fresh output
Reuse	Reuse memory; no indexer needed	Reuse existing Top-K	Compute fresh output

Every mode computes a new main query and local SWA memory. Full means the complete CSA2 path, not dense attention over every historical token.

Two Reuse layers can read the same 512 records and assign them different attention weights. Sharing a selection does not mean sharing an answer.

Why it matters

Sharing KV reduces storage; sharing Top-K indices reduces repeated search. Reindex layers preserve flexibility by making new selections against the same memory.

The encoder uses three global banks, each compressing two tokens into one entry. The decoder shares one global bank with no sequence compression. Across the network, only four layers produce global KV, while 30 CSA2 layers reuse selections.

Compared with earlier CSA, CSA2 also removes overlapping compression groups and derives indexer keys from main KV. The trade-off is reduced layer-specific global memory and the risk of reusing a selection that misses something a later layer needs.

3. Hierarchical indexing: search broadly once

The decoder’s Hierarchical Sparse Indexer makes one broad search, then limits later searches to a candidate pool. Later layers can choose different entries from that pool without rescanning the entire context.

In simpler terms

A researcher first identifies promising sections of a large library. Other researchers can examine those sections differently instead of each searching every shelf.

The first decoder indexer scores all visible global positions and selects its own Top-512. It also groups positions into blocks of eight, scores each block by its highest-scoring position, and retains up to 2,048 blocks:

\[2{,}048\times8=16{,}384\text{ candidate positions}.\]

Later Reindex layers select their own Top-512 from those candidates. Keeping whole blocks preserves nearby context around promising positions.

Why it matters

For one million positions, five full-range searches would score five million positions. One broad search plus four candidate searches scores approximately:

\[1{,}000{,}000+4(16{,}384)=1{,}065{,}536.\]

That is about 4.69× fewer scored positions in this indexing comparison, not a whole-model speedup. The first search remains linear in context length. If it excludes an important block, later decoder indexers cannot recover that block for the current query.

4. SWA bounded replay: rebuild only the recent local state

SWA Bounded Replay reconstructs missing sliding-window attention states by processing only the most recent 128 tokens. It makes both decoder initialization and local-cache misses cheaper.

In simpler terms

After pausing a long reading session, you reread the last paragraph while keeping your notes from the earlier pages.

Although each layer has a short attention window, information propagates through successive layers. Exact reconstruction can therefore require replaying a segment roughly proportional to layers × window size.

Bounded replay truncates that dependency chain by design. It replays one window and accepts approximate local states, while global context remains available through cached or projected KV.

Why it matters

The decoder can initialize its local memory without processing the full prompt. The serving system can also keep local states in host memory for a short window, rather than persisting many snapshots for far longer.

DeepSeek reports roughly 8× less persistent-cache storage from combining this redesign with global-cache compression. The saving concerns context storage, not model weights. Approximate replay can change behavior at cache-resumption boundaries, even when average quality remains similar.

5. Single-Pass mHC: move less data between blocks

Manifold-Constrained Hyper-Connections (mHC) maintain multiple residual streams and learn how to mix them around Transformer blocks. V4.1’s Single-Pass mHC shifts the input-mixing coefficients by one block, enabling more efficient kernel fusion.

In simpler terms

The original implementation must inspect the current residual state before deciding how to mix it, then read that state again to perform the mixing.

Single-Pass mHC uses coefficients prepared by the previous block. It mixes the current state without waiting, while preparing coefficients for the next block.

The block input changes from:

\[A_lX_l \quad\text{to}\quad A_{l-1}X_l,\]

where $X_l$ contains the residual streams and $A$ determines how to combine them.

Why it matters

The Mega-mHC deployment kernel fuses residual updates, input mixing, coefficient prediction, normalization, and FP8 conversion. With four residual streams, the reported activation traffic for this operation falls from $20d$ to $10d$ scalar reads and writes.

That is half the traffic for the residual operation. The architectural compromise is using mixing coefficients from one block earlier; DeepSeek reports negligible quality loss from the change.

6. Engram: give the model a learned lookup memory

Engram retrieves learned embeddings using short token sequences and merges them into the model’s contextual representation. V4.1 integrates 196B Engram parameters, distributed across two modules.

In simpler terms

A neural network spends computation recognizing familiar patterns. Engram gives it a fast lookup mechanism for some of that work.

For 2-, 3-, and 4-token patterns, multiple hash functions select entries from learned embedding tables. A context-dependent gate then decides how much of the retrieved information to use.

The lookup address depends on token identities, so it can be calculated before deeper computation finishes. This enables memory prefetching.

Why it matters

MoE scales capacity through selective computation. Engram adds a complementary form of capacity through selective memory access. Enlarging its tables does not increase per-token matrix computation in proportion to their total size.

The capacity still requires substantial storage and lookup bandwidth. Engram is learned model memory, rather than a document database or the conversation KV cache.

V4.1 adapts the earlier Engram design by removing a short causal convolution and changing the table optimizer. Its FP8 modules sit at zero-based layers 1 and 14.

7. FP4 global KV: use fewer bits for stored context

V4.1 stores its main global KV cache in a four-bit floating-point format, then dequantizes values before attention. Quantization-aware training helps the model adapt to the reduced precision.

In simpler terms

Compress the notes while storing them, then reconstruct their numeric values when reading them. Training with that compression helps the model produce representations that tolerate the rounding.

Each group of 16 channels contains four-bit E2M1 values and one eight-bit E4M3 scale:

\[16\times4+8=72\text{ bits}=4.5\text{ bits per channel}.\]

A 512-channel latent therefore needs 288 bytes for values and scales, before external alignment or metadata.

Cache component	Precision
Main global KV	E2M1 values with an E4M3 scale per 16 channels
Indexer queries and keys	OCP MXFP4
Local SWA KV	FP8

Why it matters

Compared with V4’s FP8 main cache, the new representation nearly halves storage. Combined with CSA2’s sharing, DeepSeek reports 890 bytes per token for global KV, about 890 MB for one million tokens.

That figure excludes model weights and other runtime memory. SWA stays at FP8 because it is more sensitive to quantization. Lower-precision global storage can still perturb attention, which is why the model is adapted to it during post-training.

8. DSpark: draft several tokens, verify selectively

DSpark combines parallel token proposals with lightweight dependency modeling and confidence-based verification scheduling. V4.1 incorporates this earlier speculative-decoding method to accelerate generation.

In simpler terms

A small assistant drafts the next few words. The full model checks them. If later words are unlikely to survive checking, the system can avoid spending expensive verification capacity on them.

Three draft Transformer blocks produce base logits for five positions in parallel. A lightweight Markov head models dependencies among the proposed tokens, and a confidence head estimates their acceptance probabilities.

If each token has conditional acceptance probability $p_i$, the chance that an entire prefix survives is approximately:

\[P(\text{first }j\text{ tokens accepted})\approx\prod_{i=1}^{j}p_i.\]

A few uncertain tokens near the end are enough to collapse that probability.

Why it matters

Parallel proposal computation reduces drafting latency. Dependency modeling improves later proposals. The scheduler combines confidence with current serving conditions to choose useful verification lengths.

Compared with verifying a fixed-size draft every time, this can waste less full-model work. The gain depends on acceptance rates and server load; the target model still verifies the output.

9. Modality-specific balancing: keep image and text routing healthy

V4.1 maintains separate expert-balancing biases for image and text tokens. Both modalities use the same expert collection, but their load statistics are controlled independently.

In simpler terms

Suppose two experts each process 100 tokens. That looks balanced. But one might receive almost every image token while the other receives almost every text token. Aggregate totals hide what happens within each modality.

Routing uses the token’s modality-specific bias when selecting experts:

\[S_t=\operatorname{TopK}_e\big(s_e(h_t)+b_{e,m(t)}\big).\]

Here $s_e$ is the original expert score and $b_{e,m(t)}$ is the correction for that modality. The selected expert outputs are still weighted using the original scores.

Why it matters

This extends the earlier auxiliary-loss-free balancing method to multimodal training. It can prevent modality-specific imbalance without creating separate image-only and text-only expert pools.

10. Optimizer revisions: adapt to heads and enormous tables

V4.1 uses head-wise Muon for attention Q/K matrices and momentum with Sinkhorn balancing for Engram tables, token embeddings, and the prediction head.

In simpler terms

Different attention heads may benefit from different update transformations. Head-wise Muon processes them separately instead of treating the entire matrix as one unit.

For huge embedding tables, Adam’s extra optimizer-state tensors are costly. The alternative maintains momentum and alternates row and column normalization of the update. Rows represent token or N-gram identities; columns represent hidden features.

Why it matters

Head-wise updates accommodate differences between attention heads. Sinkhorn-balanced updates avoid a full Adam second-moment buffer for the affected tables, reducing optimizer-state memory.

These are targeted applications of existing optimizer ideas. The Sinkhorn operation here balances update magnitudes; the one in mHC constrains residual mixing.

11. Verifiable task synthesis: improve what the model practices

The post-training recipe emphasizes automatically constructed agent tasks with executable environments and reliable verification. The core unit is:

\[\text{task}=(\text{problem},\text{environment},\text{verification system}).\]

In simpler terms

Give the model a working practice lab: a clear goal, usable tools, and tests that separate a correct solution from a convincing-looking failure.

The pipeline draws on real workflow failures, voluntarily shared interactions, and public code repositories. Models help construct tasks, prepare environments, attempt solutions, inspect quality, and repair defects. Later training attempts provide more evidence about whether a task is useful.

Why it matters

Compared with a fixed dataset, this approach can continually target weaknesses and expand the variety of situations the model practices. Its value depends on verifier quality: scaling flawed tests would reinforce the wrong behavior.

12. Reasoning effort: learn when extra thinking is worth it

During reinforcement learning, V4.1 is conditioned on an effort value $b$ from 1 to 100. Higher effort reduces the penalty for longer reasoning, allowing one checkpoint to learn different cost–quality settings.

In simpler terms

The model learns to spend more time exploring and checking when the requested effort makes additional reasoning less expensive in its reward.

The length penalty is:

\[r_{len}=-\min\left(C_{max},\;k(b)\frac{\ell}{L_{norm}}\right), \qquad k(b)=k_0\exp\left(-\frac{b-b_{min}}{\tau}\right).\]

Here $\ell$ is reasoning length, $L_{norm}$ is a reference length, and $C_{max}$ caps the deduction. As $b$ rises, $k(b)$ falls.

Effort preset	Value
Low	50
High	75
Max	100

Why it matters

The setting changes learned behavior before generation reaches a token limit. Applications can trade additional exploration and verification against latency and token cost using the same weights.

Higher effort has diminishing returns and does not guarantee a better answer on every task.

13. Multi-teacher OPD: combine domain strengths

The final stage uses full-vocabulary on-policy distillation (OPD) with more than 40 teacher models. Different domains can draw on teachers from different checkpoints and architectures.

In simpler terms

The student attempts the work, then receives guidance at the points it reaches, including its own mistakes. Different specialists can provide that guidance for different subjects.

“On-policy” refers to learning from student-generated trajectories. “Full-vocabulary” means supervision uses a teacher’s distribution over possible next tokens, rather than only one chosen token.

Why it matters

Compared with training only on expert-written completions, OPD addresses situations the student encounters itself. Multiple teachers can consolidate domain strengths into one model, with the teacher computation paid during training.

Distillation itself is established; the distinctive feature here is its scale and heterogeneous-teacher integration. The final deployed model does not require a 40-model voting ensemble.