ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
TopicDisconfirmAI EditorialUpdated 2026-11-23 04:44

MOREH TIDE Framework

4 steps1 pieces5 Sources

South Korean AI infrastructure firm MOREH has introduced TIDE (Temporal Incremental Draft Engine), a framework designed to optimize Large Language Model (LLM) inference by addressing the limitations of speculative decoding. Speculative decoding accelerates generation by using a small draft model to predict tokens for verification by a larger target model, but its efficiency drops when user workloads shift and the draft model becomes misaligned. TIDE solves this by reusing intermediate hidden states from the target model during live inference to continuously retrain the draft model online, eliminating the need for separate data generation passes.

The technology was presented at the Supercomputing 2026 (SC26) conference in Chicago. According to MOREH’s research, TIDE achieves up to 1.66x higher throughput compared to non-speculative baselines and reduces draft model training time by up to 3.02x. In heterogeneous GPU environments, such as pairing NVIDIA H100s for inference with AMD MI250s for training, system-wide throughput improved by up to 1.22x. This approach allows AI service providers to maintain high performance even as request patterns evolve, without manual intervention or significant computational overhead.

How it got here

  1. 2025-11-10

    November 10, 2025: MOREH published a blog post detailing the technical architecture of TIDE, explaining how it uses EAGLE3 and SGLang to enable runtime draft model training.

    moreh.io ↗
  2. 2026-02-05

    February 5, 2026: The full technical report 'TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference' was released on arXiv and MOREH's website, outlining the methodology for zero-overhead training signal collection.

    moreh.io ↗
Sign in / Sign up
ReadingResearchRadarInvestment framework
  • 2026-11-18

    November 18, 2026: MOREH researchers presented the TIDE paper at the SC26 conference in Chicago, demonstrating performance gains on real-world workloads.

    sc26.conference-program.com ↗
  • 2026-11-20

    November 20, 2026: South Korean media outlets, including Platum and Financial News, reported on the SC26 presentation, highlighting the 1.66x throughput improvement and heterogeneous GPU support.

    platum.kr ↗
  • What is still unsettled

    • How does TIDE's performance scale across different model sizes beyond those tested in the initial benchmarks?
    • What are the specific latency trade-offs when integrating TIDE into existing production serving stacks like vLLM or TensorRT-LLM?
    • Will major cloud service providers adopt similar self-improving inference frameworks, or will they rely on static optimization techniques?

    Sources

    • platum.krPlatum↗
    • sc26.conference-program.comSC26 Conference Program↗
    • moreh.ioMOREH Technical Report↗
    • arxiv.orgarXiv Paper↗
    • en.fnnews.comFinancial News↗

    Coverage in this topic

    • Korean AI Infrastructure Firm MOREH Unveils TIDE Framework at SC26, Claiming 1.66x Throughput Gain2026-11-20News

      MOREH introduced TIDE at SC26, a framework that uses target model hidden states to retrain draft models online, addressing speculative decoding pattern mismatches.