MOREH TIDE Framework
4 steps1 pieces5 Sources
South Korean AI infrastructure firm MOREH has introduced TIDE (Temporal Incremental Draft Engine), a framework designed to optimize Large Language Model (LLM) inference by addressing the limitations of speculative decoding. Speculative decoding accelerates generation by using a small draft model to predict tokens for verification by a larger target model, but its efficiency drops when user workloads shift and the draft model becomes misaligned. TIDE solves this by reusing intermediate hidden states from the target model during live inference to continuously retrain the draft model online, eliminating the need for separate data generation passes.
The technology was presented at the Supercomputing 2026 (SC26) conference in Chicago. According to MOREH’s research, TIDE achieves up to 1.66x higher throughput compared to non-speculative baselines and reduces draft model training time by up to 3.02x. In heterogeneous GPU environments, such as pairing NVIDIA H100s for inference with AMD MI250s for training, system-wide throughput improved by up to 1.22x. This approach allows AI service providers to maintain high performance even as request patterns evolve, without manual intervention or significant computational overhead.
How it got here
November 10, 2025: MOREH published a blog post detailing the technical architecture of TIDE, explaining how it uses EAGLE3 and SGLang to enable runtime draft model training.
moreh.io ↗February 5, 2026: The full technical report 'TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference' was released on arXiv and MOREH's website, outlining the methodology for zero-overhead training signal collection.
moreh.io ↗