LectureRechercheRadarCadre d'investissement
Connexion / Inscription
← Retour à la lecture
ActualitésSujet · Voltic记忆架构Prise rapide1 min de lectureRédaction DisconfirmAI

Voltic Separates Volatility from Noise in Memory

New preprint introduces Voltic, distinguishing volatility from stochasticity to optimize recurrent memory, outperforming baselines in small model reasoning.

2026-10-06 12:00 GMT+8

The Voltic architecture outperforms gated baselines in average performance across eight reasoning tasks and long-context retrieval accuracy in 45M-parameter language models.

Traditional recurrent memories like Gated Delta-Rule treat uncertainty as isotropic, failing to distinguish between 'volatility' (how fast associations change) and 'stochasticity' (observation noise). This rigidity limits adaptation to dynamic environments.

Voltic maintains anisotropic covariance and makes noise variances input-dependent, allowing write operations to carry accumulated uncertainty. It uses diagonal and quasi-diagonal approximations to preserve parallel training efficiency by reusing chunked kernels.

These results are based on author-run controlled recall tasks and small-scale experiments, with no independent reproduction or large-scale deployment verification yet.

Sources

  • arxiv.org↗
Sujet

Voltic记忆架构

Voltic是一种新型循环记忆架构,旨在通过区分“波动性”(关联变化的速度)与“随机性”(观测噪声的大小)来优化序列模型的内存写入机制。该架构由Parsa Hejabi等人提出,核心在于将门控delta规则记忆的更新视为各向同性不确定性下的…

2 étapes1 pièces

Connexion / Inscription
LectureRechercheRadarCadre d'investissement
Mis à jour 2026-10-08 20:19

Comment on en est arrivé là

  1. 2026-10-05arXiv预印本《Voltic: Distinguishing Volatility from Stochasticity in Recurrent Memory》提交,提出Voltic架构。arxiv.org ↗
  2. 2026-10-06新闻摘要指出Voltic通过区分波动性与随机性优化循环记忆,在小模型推理中领先基线。arxiv.org ↗

Ce qui reste non résolu

  • Voltic架构在更大规模参数模型(如数十亿参数)中的扩展性和性能表现如何?
  • 对角和准对角近似方法在极端长序列场景下是否会累积显著误差?
  • 该架构是否已被集成到主流开源框架或实际生产环境中?
Lire le sujet complet →
Continuer la recherche →