ReadingResearchRadarInvestment framework
Sign in / Sign up
AllTechnologyProductsInvestingPublic marketsOther

Reading

2026-11-0647 posts

Refresh

Databricks Application Scaling Guide: Separating Soft and Hard State to Decide Between Adding Machines or Instances

Material
2026-11-06 02:00 GMT+8

Bruno Tafur and Tushar Madan of Databricks published an extension guide for application builders in November 2026, discussing how to maintain fast response times and high availability as a Databricks App grows from a few users to hundreds or thousands. The core judgment of the article is that the intuitive answer 'add more compute resources' is only effective once you know where requests spend their time and whether the application itself is built to scale.

The authors divide the scaling process into four stages: design, build, test deployment, and operations. In the architecture design stage, the most critical decision is dividing application state into two categories. Soft state is temporary working data, such as in-memory context, caches, and ongoing work, which can be rebuilt from hard state if lost; hard state consists of business records, checkpoints, and user preferences, which must be persisted in Lakebase or Unity Catalog tables so that any instance can recover the session. This classification determines whether horizontal scaling is truly feasible—if session state exists only in memory, users will lose their work when instances restart or migrate.

The article provides a specific retailer scenario to demonstrate load estimation: during the busiest hour of Black Friday, 200 out of 2,000 store managers are using an inventory query app simultaneously, with each performing two queries per minute, each executing three SQL statements. The result is 1,200 statements per minute, averaging 20 per second. The authors emphasize that this estimates query rate rather than concurrency; testing needs to validate burst traffic and growth, and database connections should be budgeted separately.

Regarding connection budgeting, the article provides an actionable formula: when each worker maintains a connection pool, the upper limit of connections is the number of instances multiplied by the number of workers per instance multiplied by the maximum pool size. For example, with five instances, eight workers each, and a pool limit of ten, up to 400 connections can be opened. After adding other clients and connections opened on behalf of users, this total must be compared against database documentation limits, leaving headroom for deployments. For scenarios requiring queries to Lakebase on behalf of users, each user requires an independent Postgres role, and Lakebase limits branches to 500 roles; adding instances does not change this upper limit.

The core recommendation for the build phase is to use asynchronous code. Most applications spend time waiting for external services—SQL queries, model calls, and external APIs. Blocking calls occupy an execution slot for the duration of the wait, meaning a few slow calls can exhaust capacity. When handlers await model and database calls, one worker can continue serving other requests while waiting. The article also recommends using asyncio.gather or Promise.all to launch independent queries in parallel, making request latency close to that of the slowest query rather than the sum of all queries, while limiting parallelism to avoid overwhelming the backend.

Sources:Databricks

Research

Airbnb Plans to Launch Custom AI Travel Agent in First Half of 2027, Raises Revenue Guidance

Material
2026-11-06 06:40 GMT+8

Online travel platform Airbnb announced in its third-quarter shareholder letter and earnings call that its custom-built AI travel agent is scheduled to launch for consumers in the first half of 2027. This timeline is more specific than previous vague references to "sometime next year," marking a key step in Airbnb's AI productization strategy.

CEO Brian Chesky emphasized that the agent is not a traditional text-heavy chatbot but is built around the concept of "generative UI." Chesky noted that travel planning is aspirational and visual-dependent, with 80% of travelers enjoying the research process. Consequently, the agent will compose hundreds of travel-specific interface components—such as maps, photos, comparisons, calendars, and itineraries—in real time based on user queries, rather than simply responding with text.

Beyond booking, Airbnb stated the agent will assist with other aspects of the trip, including changing reservations, resolving payment issues, handling refunds, and providing 24-hour safety support. The company argues that these deeply integrated services create a barrier that external general-purpose AI agents cannot easily replicate.

The product roadmap update coincided with strong financial results. Airbnb reported third-quarter revenue up 19% year-over-year to $4.87 billion, beating Wall Street estimates. Based on this performance, the company raised its full-year 2026 revenue growth forecast from "mid-teens" to "high teens." Chesky stated, "Airbnb is back in growth mode," noting that Q3 was the company's fastest revenue growth in more than three years.

For investors, the explicit 2027 launch timeline and unique "generative UI" architecture provide concrete anchors for evaluating Airbnb's AI strategy execution. However, actual market reaction and user adoption remain unverified until the product launches, and the specific impact on conversion rates or long-term competitive dynamics cannot yet be quantified.

Sources:TechCrunch

Research

Memrise Co-founder Ed Cooke’s AI Tutor Startup Mindmax Raises $3.3M Pre-Seed

Topic · MindmaxQuick take
2026-11-05 17:00 GMT+8

AI edtech startup Mindmax has raised $3.3 million in a pre-seed round led by Concept Ventures, with participation from Redrice Ventures, Daybreak Ventures, and Lerer Hippeau. Angel investors include Maria Raga, former CEO of Depoplo, and Torsten Reil, co-founder of defense AI company Helsing.

Founded by Memrise co-founders Ed Cooke and Daniel Zohar, Mindmax offers an AI tutor for professional exam preparation and upskilling. The app is currently live globally on the App Store and Play Store. The company positions its product as a solution to 'cognitive surrender,' a term describing the tendency to rely too heavily on AI at the expense of active thinking.

Unlike one-way video lessons, Mindmax emphasizes real-time interaction, allowing users to interrupt, ask questions, or steer the lesson. The founders claim that combining Memrise's memory techniques with large language models improves information retention. The eight-person team is based in Paris, Latvia, London, and India. While the company asserts its interactive approach outperforms traditional teaching, it has not yet published error rates or independent validation results.

Sources:Tech Funding News

Research

Atlassian Shares Fall Over 17% After-Hours Despite Earnings Beat, as Subscription Growth Guidance Slows

Topic · AtlassianMaterial
2026-11-06 06:26 GMT+8

Australian collaboration software company Atlassian saw its shares fall more than 17% in after-hours trading following the release of its first-quarter fiscal 2027 results, despite beating analyst expectations across key financial metrics. The sharp decline reflects investor anxiety over a downward revision in future subscription annual recurring revenue (ARR) growth, which overshadowed strong short-term profitability and backlog figures.

For the quarter ended September 30, Atlassian reported adjusted earnings per share of $1.65, up from $1.04 in the same period last year and exceeding the consensus estimate of $1.43. Revenue rose 24% year-over-year to $1.772 billion, surpassing the expected $1.71 billion. This growth was primarily driven by its cloud segment, where revenue increased 31% to $1.302 billion. Conversely, the Data Center product line, which is scheduled for retirement in March 2029, saw growth slow to approximately 5%, with revenue at $391 million, as customers planning their migration to the cloud expanded less rapidly.

The pivotal factor triggering the sell-off was the guidance update. Atlassian now expects subscription ARR growth to decelerate to about 18.5% by the end of the fiscal year, down from the 22.5% rate just reported. Although remaining performance obligations surged 44% to $4.746 billion—supported by larger, multi-year enterprise deals—investors focused heavily on the immediate trajectory of subscription revenue growth. The company also posted net income of $91 million for the second consecutive quarter on an unadjusted basis, with free cash flow rising to $463 million and $581 million returned to shareholders through buybacks.

Sources:SiliconANGLE

Research

NVIDIA Opens Source DSX OS: Why GPU Cluster Readiness Cannot Be a One-Time Event

Topic · 英伟达DSX OSMaterial
2026-11-06 01:00 GMT+8

A GPU performing slightly below target specifications may not trigger errors, Xid events, or abnormal utilization metrics on monitoring dashboards. However, it can slow down an entire distributed training job to the speed of its slowest rank, with operations staff potentially discovering the issue only hours later through user complaints. In an article published on November 5, 2026, NVIDIA Technical Blog authors Sowmyan Soman and Kai Wombacher use this problem as a starting point to explain why GPU cluster readiness validation cannot be performed just once at delivery.

Their argument is that while a cluster passes readiness checks at handover, the original certification no longer reflects the cluster's actual state as nodes are replaced, drivers are patched, and capacity is expanded. This visibility gap wastes compute power on failing nodes and leaves healthy nodes idle due to conservative policies. The authors propose measuring cluster efficiency using goodput (the volume of successfully completed work) rather than utilization, and argue that maintaining readiness requires continuous validation throughout the operational lifecycle—after configuration changes, during capacity growth, and before repaired nodes return to service.

The article maps the three open-source components of NVIDIA DSX OS to these three verification moments. AICR (AI Cluster Runtime) provides version-locked configuration recipes, recording compatible versions and dependencies for components such as networking, storage, and drivers as auditable files, rendered as deployment artifacts usable by Argo CD, Flux, or Helm. NVCRE (Cluster Readiness Engine) runs NCCL communication tests and training workloads prior to cluster delivery, determining whether nodes meet standards via configurable performance thresholds, and bisecting suspect node ranges upon failure. NVSentinel continuously monitors GPU, node, and network health signals during cluster operation, publishes node statuses, and isolates, drains, and remediates faulty nodes when enabled.

The true incremental value lies in the automatic integration mechanism between these three components. When a node joins the cluster or returns after remediation, NVSentinel’s lifecycle controller automatically creates a ValidationRequest, gates scheduling using cordon or taint, and then invokes NVCRE to run tests. If the tests pass, the controller removes the gate; if they fail after retries, the node remains isolated. This means repaired nodes do not silently rejoin the pool. The authors specifically note that for situations such as driver upgrades, operational procedures, or fabric changes that do not trigger lifecycle events, AICR’s drift detection (aicr diff --fail-on-drift) can fail within a CI pipeline, thereby triggering a certification run within the same change process.

The article also clearly delineates the responsibility boundaries of each component. NVCRE is responsible only for testing and attribution—recording which nodes failed and why, without altering node status; NVSentinel is responsible for isolation and repair. The authors explain that this separation is intended to prevent two systems from cordoning the same node simultaneously, which would lead to double isolation and desynchronized cleanup. NVSentinel is installed in monitoring mode by default, reporting faults merely as node conditions without executing cordon, eviction, or repair actions; operations teams enable automated handling only after trusting the quality of the reported signals.

Sources:NVIDIA Technical Blog

Research

Microsoft Foundry Extension for Azure Developer CLI Reaches General Availability

Quick take
2026-11-06 04:55 GMT+8

On November 5, 2026, Microsoft announced that the Microsoft Foundry extension bundle (microsoft.foundry) for Azure Developer CLI (azd) has reached version 1.0 and is generally available. The extension integrates AI agent configuration, local testing via Agent Inspector, evaluation, and deployment into the azd workflow, allowing developers to manage the full lifecycle of hosted agents without switching tools. Requires azd 1.37.0 or later and an Azure subscription.

Sources:Microsoft Foundry Blog

Research

Qualcomm Publicly Exhibits World's First 3GPP-Compliant 6G Terminal Prototype at CIIE, Achieving ~14Gbps Peak Rate

Topic · 高通6G终端原型机Material
2026-11-05 05:02 GMT+8

US chipmaker Qualcomm publicly exhibited the world's first 6G terminal prototype compliant with the latest 3GPP definitions at the ninth China International Import Expo (CIIE), which opened on November 5. This marks the first time the prototype has been displayed physically to the public outside of a laboratory setting.

According to reports from Zhidongxi and official CIIE news, the prototype serves as a key test and verification platform for moving towards 6G commercialization. It has already been used in joint 6G technology tests and Interoperability Device Testing (IoDT) with Chinese industry partners including China Mobile and ZTE. In September, Qualcomm and China Mobile completed IoDT based on the latest 3GPP-defined radio frequency specifications, achieving uplink and downlink link interconnection between devices from different vendors.

Regarding performance parameters, tests conducted by Qualcomm and ZTE in their Xi'an laboratory late last year showed that the prototype utilized 400 MHz of downlink bandwidth and 400 MHz of uplink bandwidth. Its downlink bandwidth is four times that of current 5G spectrum, with a peak downlink rate of approximately 14 Gbps. Beyond wired radio frequency calibration tests, Qualcomm stated it will gradually move into Over-The-Air (OTA) testing phases.

Currently, the 3GPP 6G standard is still evolving, with the first version, Release 21, expected to be finalized by 2029. At the Mobile World Congress (MWC) in March, Qualcomm reached a consensus with nearly 60 global partners, including about 20 Chinese ones, to push for 6G commercialization starting from 2029. Qualcomm plans to continuously configure and upgrade this prototype based on standard developments to accumulate practical experience.

Sources:zhidx.com

Research

SemiAnalysis: TPUv7 Delivers 1.2 to 2.7 Times More Tokens Per Dollar Than NVIDIA/AMD on GLM-5.2 AgentX

Topic · TPUv7推理性能Material
2026-11-06 03:03 GMT+8

SemiAnalysis released the first cross-model results for TPUv7 (Ironwood) on the InferenceX/AgentX agent inference benchmark on November 5, 2026. AgentX simulates real-world agent workloads characterized by multi-turn interactions, long contexts, and high prefix reuse, which differ significantly from single-turn chat scenarios. This release covers four models: Meta's Muse Spark Open (816B parameters), Zhipu AI's GLM-5.2/5.3 (744B parameter MoE), Alibaba's Qwen3.5 397B, and Moonshot AI's Kimi K3 (2.8T parameters).

At FP8 precision, the results for each model must be viewed separately. On GLM-5.2, within an interaction range of 60 to 300 tokens per second per user, TPUv7 delivers 1.2 to 2.7 times more tokens per dollar than the B200, B300, and AMD MI355X. On Qwen3.5 397B, SemiAnalysis reports that TPUv7 FP8 performance per dollar is 1.67 times that of the GB300 NVL72 FP4—this marks the first AgentX model result submitted by Google itself. On Kimi K3, results submitted by Inferact, founded by the vLLM founding team, show that TPUv7 delivers 1.15 to 2.5 times more tokens per dollar than the GB200, B200, B300, and MI355X. Muse Spark Open was submitted by RadixArk, maintained by SGLang, achieving competitive decode cost-effectiveness even on a GPU-optimized architecture theoretically unfriendly to TPUs.

However, SemiAnalysis notes that NVIDIA FP4 still maintains a lead in pure performance per dollar. On GLM-5.2, at 100 tok/s/user, the GB300 and GB200 FP4 deliver 3.6x and 3.5x more tokens per dollar than TPUv7, respectively. This is because FP4 halves the weight bytes, and small-batch decode is memory-bandwidth bound. TPUv7 currently lacks native FP4 support, with this feature expected to arrive with TPUv8 in Q1 2027. NVIDIA submitted FP4 counter-results on Qwen3.5; while these exceeded TPUv7 FP8 in decode performance, they came at the expense of prefill, resulting in lower end-to-end performance than TPUv7 FP8.

SemiAnalysis's core argument is that TPUv7's competitiveness stems not just from hardware specifications but from ecosystem collaboration with upstream maintainers such as vLLM (Inferact, RedHat), SGLang (RadixArk), llm-d, PyTorch, and Ray. Meta has already deployed TPUv7 in production using SGLang-TorchTPU. Kimi K3.1 Pro received TPUv7 support on its release day. SemiAnalysis argues that NVIDIA's moat is being eroded from its software ecosystem, citing bureaucratic processes—such as a three-month legal review required for driver feature approvals—as a structural risk.

It is important to emphasize that the submitters of the above results vary: Qwen3.5 was submitted directly by Google, Muse Spark Open by RadixArk, and Kimi K3 by Inferact. All data are first-party results analyzed and published by SemiAnalysis and have not yet been independently reproduced. Regarding NVIDIA, preview submissions for the Rubin NVL72 suggest its performance per dollar could reach 6 times that of TPUv7, but SemiAnalysis points out this is an unfair comparison; Rubin should be benchmarked against TPUv8i rather than TPUv7. For readers making compute procurement decisions, these figures provide directional reference, but independent verification should be requested before making commitments.

Sources:newsletter.semianalysis.com

Research

AI Coding Startup Factory Acquires HumanLayer, Completing Third Acquisition in Under a Month

Topic · Factory收购潮Material
2026-11-06 04:02 GMT+8

US-based AI coding tool company Factory announced on Wednesday that it has acquired HumanLayer, marking its third acquisition of an AI coding startup in less than a month. The deal aims to fold more of the software engineering lifecycle into a single autonomous platform. This move comes as demand for agentic coding tools climbs at a pace few enterprise products ever see, pushing companies building them to consolidate rapidly.

According to announcements from both parties, HumanLayer founders Dex Horthy and Kyle Mistele will join Factory, bringing along technology for coordinating multiple coding agents and collaborating on their work. Factory co-founder and CEO Matan Grinberg welcomed the pair on X, describing them as two of the most ambitious polymaths he has met.

For existing users, Factory stated it will refund paying customers for the last several months, though neither company has set a shutdown date for HumanLayer’s hosted service or specified which features will carry over. As an escape hatch for users who prefer not to migrate, HumanLayer open-sourced its core software under Apache 2.0 on October 23, although some processes still require third-party API keys.

This acquisition caps a quick run of deals mapping Factory’s priorities. On October 9, Factory acquired Mentlio, a Y Combinator startup focused on making coding agents more cost-efficient through model routing, context compression, and spending analysis. Three weeks later, on October 30, it acquired Typedef, which builds tools for understanding relationships across data models, pipelines, and dashboards, technology Factory plans to use for enterprise data migrations and refactoring.

Factory’s coding agents, known as Droids, handle everything from planning and writing code to testing and maintenance. HumanLayer’s founders have focused on what happens when teams run multiple coding agents at once, including managing context, coordinating tasks, and reviewing results. Horthy and Mistele also helped popularize the Research/Plan/Implement workflow, which breaks coding tasks into separate stages.

Sources:The New Stack

Research

US Investment Firm Belvedere Trading Discloses $170.8 Billion Stock Portfolio, Heavy on AI Leaders Like Nvidia and AMD

Material
Verified 2026-11-06 05:39 GMT+8

US investment firm Belvedere Trading LLC filed its Form 13F-HR with the U.S. Securities and Exchange Commission (SEC) on November 5, 2026, for the quarter ended September 30, 2026. The filing discloses that the firm held approximately $170.77 billion in reportable US equity securities. It is important to note that this figure represents only the market value of specific stocks held in US markets as of the quarter-end snapshot; it does not reflect the firm's total Assets Under Management (AUM), which remains undisclosed in this form.

The portfolio reveals substantial institutional exposure to the artificial intelligence sector. Key holdings include significant positions in NVIDIA Corporation, Advanced Micro Devices Inc, and Micron Technology Inc. For instance, call options on NVIDIA alone correspond to nearly $3.5 billion in underlying share value, while positions in Micron and AMD also reach into the billions. Additionally, the firm holds positions in Amazon, Apple, Meta Platforms, and Broadcom Inc.

13F filings are quarterly snapshots required from large institutional investors, reflecting holdings at quarter-end rather than real-time trading activity or complete asset portfolios (excluding shorts, detailed derivatives, and non-US assets). Despite the lag, Belvedere Trading's disclosure of such a massive portfolio provides concrete quantitative evidence of how top-tier institutional capital is positioned within the AI infrastructure ecosystem.

Sources:sec.gov

Research

AI Inference Infrastructure Company Rebellions and Hanyang University Find FP4 Reasoning Failure Originates from Low-Entropy Position Sampling Errors; ReQAT Enables 4-bit Full Quantization to Match BF16 Precision

Topic · FP4推理精度失效Structural
2026-11-05 06:49 GMT+8

Hardware support for 4-bit floating-point formats has outpaced model tolerance. FP4 compute cores are now shipping, and microscaled formats like MXFP4 and NVFP4 make 4-bit floating-point activations feasible, yet it is precisely in reasoning models—those that generate ultra-long chains of thought to solve math problems—that precision collapses first. In an oral paper presented at ICML 2026, Rebellions and Hanyang University propose a counterintuitive explanation: FP4 damage to reasoning occurs primarily at the model's most confident positions, rather than at its most hesitant branching points.

The research team initially hypothesized that insufficient fine-tuning budget was the bottleneck. On R1-Llama-8B, they increased the number of fine-tuning epochs from one to five, yet the post-PTQ accuracy drop remained stuck between 11 and 15 points, indicating the problem lay not in training volume. They then turned to analyzing where quantization errors appeared within reasoning traces, tracking the entropy of the next-token distribution at each decoding step. Grouping over 1.5 million generated tokens by entropy revealed two populations: low-entropy tokens, comprising roughly 80% of the trace, consisting mainly of digits, variable names, and operators that the model predicts almost deterministically; and high-entropy tokens, such as discourse markers like 'Then,' 'Let,' and 'Hmm,' which appear at reasoning path branches.

Intuition suggests high-entropy positions are the sensitive ones—a small perturbation there could change the top-1 token. However, two controlled experiments yielded the opposite answer. The first experiment routed each prediction to either a BF16 or an FP4 model based on entropy: sending only low-entropy tokens to BF16 recovered most of the lost accuracy, while sending only high-entropy tokens provided little help. The second experiment injected Gaussian noise into logits as a proxy for quantization error: noise applied to the lowest-entropy 25% of positions caused a steep drop in AIME-2025 accuracy, whereas noise on the highest-entropy 25% had a much smaller effect. This pattern held across R1-Qwen-7B, R1-Qwen-14B, and the newer Qwen3-4B and Qwen3-8B.

The mechanism lies in the distribution tail. In low-entropy buckets, FP4 rarely changes the top-1 token, but the probability mass outside it grows significantly relative to BF16. Under greedy decoding this has little impact, but production services use temperature sampling, where every position carries a tiny probability of jumping to the wrong word. A reasoning trace spans thousands of positions; a single incorrect digit or operator typically invalidates the entire subsequent derivation. Perturbations at high-entropy positions, by contrast, mostly alter wording without changing the final answer. The team summarizes this finding: the dominant cause of FP4 reasoning failure is increased sampling error at the positions where the model is most confident, and standard QAT applies uniform loss across all tokens, failing to target this specific failure mode.

The ReQAT framework designs three components around this insight. Trace-Aligned QAT (TAQ) operates in two stages: first, fine-tune the model in BF16 on a reasoning dataset; second, enable quantization noise for QAT on a subset of the same traces used in Stage 1. Tracking token entropy during training revealed that standalone fine-tuning or standalone QAT primarily altered the high-entropy interval, whereas fine-tuning followed by QAT on the identical traces also changed the low-entropy interval—the model having already learned each trace in Stage 1 meant the Stage 2 gradients acted upon the low-entropy predictions made fragile by quantization. On R1-Qwen-14B under MXFP4 W4A16 with a 350M-token budget, two-stage training without trace alignment raised AIME accuracy from 62.29% to 62.60%, while trace alignment brought it to 67.29%.

Sources:Rebellions

Research

OpenAI says its autonomous vulnerability pipeline found and patched 948 Linux kernel LPEs for $1.2M

Material
2026-11-04 16:00 GMT+8

US-based artificial intelligence company OpenAI published a report on November 4, 2026, stating that it developed an autonomous vulnerability discovery, exploitation, and patching pipeline. Using an upcoming cyber model, the pipeline dynamically reproduced 1,724 vulnerabilities in the Linux kernel and generated 948 local privilege escalation (LPE) exploits and candidate patches. The total API cost for the run was $1.21343 million, averaging approximately $1,279 per LPE exploit.

OpenAI described the pipeline as consisting of five stages: vulnerability discovery, dynamic validation, exploitability assessment, patch generation, and human review. In the discovery stage, agents identified 5,034 high-value audit targets, covering about 7% of the kernel codebase. During the analysis phase, agents produced 2,308 candidate findings within 12 hours. Subsequently, validation agents reproduced 1,724 of these vulnerabilities in isolated environments, achieving a success rate of approximately 74.7%.

To assess economic impact, OpenAI compared the results with historical bug bounty programs. Cybersecurity competitions like Pwn2Own and TyphoonPWN have historically offered $20,000 to $70,000 for Linux kernel LPE exploits. Valuing the 948 exploits at these rates yields a hypothetical total between $18.9 million and $66.3 million, roughly 16 to 55 times the pipeline's cost. Additionally, in tests against Google's kernelCTF program, the pipeline discovered 21 LPE exploits for about $32,936, which would yield around $1.5 million if accepted, approximately 45 times the cost.

OpenAI also noted that the pipeline used at least 10 novel Linux exploitation techniques, which were considered new by third-party kernel exploitation experts. The company emphasized that automated patch generation is essential for securing software as model capabilities improve, urging defenders to proactively analyze their software to make exploitation cost-prohibitive for attackers.

Sources:openai.com

Research
Next →
Sign in / Sign up
ReadingResearchRadarInvestment framework

Regarding graceful shutdown, Databricks sends SIGTERM upon redeployment, followed by SIGKILL if the application has not exited within 15 seconds. This window is intended for saving final checkpoints, but the authors emphasize that important work should already be saved during normal operation. For FastAPI applications, it is recommended to set Uvicorn's --timeout-graceful-shutdown to below 15 seconds (e.g., 10 seconds) to prevent long-running requests from consuming checkpoint time.

In the test deployment phase, the authors recommend using tools like Locust to perform load tests with real user journeys at expected peak levels, gradually increasing concurrent users until throughput no longer increases or latency and error rates exceed targets. The article lists a decision table to help select instance specifications: if requests are slow but application CPU usage is low, it indicates waiting on the backend, so queries or connection pools should be fixed before adding compute; if CPU is near the limit and the server runs multiple workers, upgrading to Large is appropriate; for single-process applications (such as Streamlit or default-configured AppKit), when CPU is near the limit, adding instances rather than upgrading specifications is better, because larger instances are only useful if the application can utilize the additional cores.

The operations phase suggests keeping telemetry enabled, wrapping SQL, model, and API calls with custom spans to locate time consumption, and tracking individual user requests across instances via traces and request IDs. When slow requests occur, the article provides a symptom-to-cause comparison table: slow first requests after idle periods point to cold starts; CPU or memory near the limit points to the application itself being the bottleneck; database connection timeouts point to pool contention or long transactions.

The key premise of the entire method is that application compute is responsible only for serving the UI and coordinating calls, while heavy lifting is delegated to specialized services. Long-running work should be placed in background tasks, allowing the UI to poll for status or receive streaming updates, rather than keeping a request hanging indefinitely. If the application needs to run code written by users or generated by agents, it should be placed in the Databricks Sandbox isolated environment, ensuring its CPU and memory consumption do not affect the instances serving users.

Boundaries that must be preserved include: this is an explanation from NVIDIA regarding its own platform design intent and component interaction mechanisms, not an independently verified performance report. The article does not provide statistics on how often this closed loop actually captures performance degradation in production environments, nor false positive rates or comparisons with other cluster management solutions. Thresholds are not configured by default, meaning the effectiveness of performance degradation detection depends on whether operations teams proactively set appropriate standards such as busBandwidthGBps. The described level of automation requires enabling both the lifecycle manager and fault-quarantine validation independent switches, and the current open-source license is Apache 2.0, awaiting external adoption for further validation.

Selective Entropy Minimization (SEM) adds an entropy minimization term to the SFT loss, applied only to the lowest-entropy 75% of tokens in each batch, with weight increasing as entropy decreases; the highest-entropy 25% are excluded to preserve the original distribution at branch points. TAQ restored most accuracy under W4A4, but accuracy dropped again when the KV cache was quantized to 4 bits, pointing to initialization issues in QAT. Q-FIT jointly calibrates two known transformations per layer—channel-pair scaling before RoPE and per-channel shifting after RoPE—to minimize the difference between BF16 and 4-bit KV attention outputs. Neither transformation alone yielded reliable gains in ablation studies, but joint calibration raised NVFP4 W4A4KV4 accuracy from 63.13% to 65.94%.

On R1-Qwen-14B, fine-tuned on the OpenThoughts3 math subset and evaluated on AIME-120 (AIME 2022–2025, averaged over 8 seeds), results under the same total budget were: BF16 baseline 56.83%, BF16 full fine-tuning 65.46%, direct PTQ under NVFP4 W4A4KV4 only 50.13%, standard QAT 58.86%, and ReQAT reaching 65.94%—slightly higher than BF16 full fine-tuning. Both BF16 fine-tuning and QAT saturated at approximately 280M tokens, whereas ReQAT continued to improve with budget, achieving the highest accuracy in all test settings, including comparisons against AWQ, QuaRot, FlatQuant, and QAD.

Matching BF16 accuracy means the throughput advantage of FP4 can be realized directly. Measuring end-to-end output-token throughput on a single NVIDIA B200 GPU with 1,000 requests of 512-token prompts, a batch limit of 256, and KV cache allocated to memory limits: at 2K output length, NVFP4 showed about 1.8x to 1.9x speedup over BF16, reflecting the raw compute advantage of 4-bit GEMM; at 8K output length, BF16 could no longer maintain the same batch size due to KV cache growth while NVFP4 could, yielding a speedup exceeding 3x. Even accounting for the 4% to 5% overhead of Q-FIT, ReQAT reached 3.05x. For long reasoning outputs, most of the FP4 gain comes from the larger batch sizes permitted by KV cache compression, making accuracy retention with a 4-bit KV cache a necessary condition for realizing FP4 efficiency on inference workloads.

The core insight of this work can be understood independently of specific models and formats: where the training signal is directed matters more than how much is thrown at it—increasing token count or epochs within the budget did not close the FP4 gap, whereas targeting low-entropy positions did. Code and the training recipe are publicly available at github.com/aiha-lab/ReQAT.