Reason Through the Latent! Making Latent Visual Reasoning Necessary
CVRR revisits the same visual evidence as the question state evolves. The final state carries visual information into the answer decoder.
Figure 2 from Park, Jung, and Kang (2026), resized and re-encoded as WebP, CC BY 4.0. Source ↗ · License ↗
Abstract
Causal Visual Recurrent Reasoning (CVRR) tests whether a vision-language model can retain visual reasoning ability when it has to answer through a recurrent hidden state. The model starts with a question representation formed by the pretrained backbone, updates that representation against the image several times, and decodes through the final state. Across four benchmarks, CVRR outperformed retrained latent reasoning baselines subject to the same decoder constraint. Paper
When does a model use its latent reasoning?
A model can encode the right visual evidence in a hidden state while its answer still depends on the original image context. In preliminary experiments, replacing a latent state with noise or a state from a mismatched example barely changed accuracy for some methods. The resulting question was whether an answer could be made to depend on the latent state while preserving the image understanding learned during pretraining. Preliminary experiments
How CVRR works
- Start from the pretrained model’s question state. Swapping visual activations between paired images helps locate the layer where the question tokens already carry the visual evidence needed for the task.
- Revisit fixed visual evidence. A shared decoder layer updates the question state repeatedly. At each step, its existing attention mechanism can consult the same visual tokens with the updated question state.
- Decode from the final state. The answer decoder receives the last recurrent question state and a text-only cache. The original visual tokens and multimodal cache are removed before decoding.
The backbone stays frozen, with low-rank adapters trained only on the shared transition layer. The training signal is answer-token cross-entropy. Method and training
Main results
The comparison uses a decoder that can receive image-conditioned information only through the latent state. On MMVP, a pair counts as correct when the model answers both image questions correctly. Benchmark protocol and results
| Benchmark | CVRR | Best retrained latent baseline |
|---|---|---|
| $V^*$ overall accuracy | 81.2% | 39.8% |
| MMVP pair accuracy | 52.7% | 2.7% |
| BLINK overall accuracy | 55.2% | 38.8% |
| MME-RealWorld-Lite overall accuracy | 46.7% | 33.1% |
What the recurrent state carries
Image access during recurrence
Removing visual access during the recurrent steps and retraining reduced MMVP pair accuracy from 52.7% to 18.0%. This test starts from the same kind of multimodal question state and isolates the contribution of rereading the image during the updates. Component analysis
A fixed question with different evidence
For MMVP image pairs that shared the same question, swapping in the competing image’s final state reduced accuracy in the intervention experiment from 73.3% to 26.7%. Holding the question fixed isolates the effect of changing the visual evidence carried by the state. State replacement experiment
Scope
The visual-read boundary is selected separately for each backbone. The mechanistic tests mainly cover multiple-choice tasks, and the model reuses the visual evidence available at initialization. Limitations
Resources and citation
BibTeX
@misc{park2026reasonlatentmakinglatent,
title = {Reason Through the Latent! Making Latent Visual Reasoning Necessary},
author = {Park, Suhyeong and Jung, Junha and Kang, Jaewoo},
year = {2026},
eprint = {2609.06746},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2609.06746}
}