LectureRechercheRadarCadre d'investissement
Connexion / Inscription
ToutTechnologieProduitsInvestissementBourseAutres

Lecture

2026-11-18133 publications

Actualiser

Decision Models Cannot Be Selected by Benchmark Scores: Microsoft Engineering VP Says 'Benchmaxed' Models Fail Rapidly in Enterprise Use

Sujet · 决策模型选型之争Matériel
2026-11-18 02:10 GMT+8

Decision models are a new product category that emerged only in September 2026: they do not generate text but use the general intelligence of large language models to make quick judgments between preset options while outputting confidence levels. Data from Vercel’s AI model router shows that in October 2026, more than one-quarter of requests flowed to decision models rather than traditional large language models. One reason for this category's rapid growth is its low barrier to entry—TechCrunch reported that an Amazon engineer built a decision model over a single weekend, prompting major cloud providers to follow suit. On November 17, Microsoft CEO Satya Nadella announced that Microsoft-Decision-1.1 would be immediately rolled out across Azure.

As the number of models rapidly expanded, the selection problem became thorny. Achint Srivastava, Vice President of Engineering at Microsoft, wrote in an email: "What we see is that 'benchmaxed' models fall flat under real enterprise usage very quickly." He further pointed out that it is extremely challenging to deliver accuracy, reliability, speed, and cost efficiency within a single model; Microsoft-Decision-1.1 and TypeSafe’s Jev are among the few models that maintain strong performance across four dimensions even on "held-out" or unseen benchmarks.

Ushnish Sengupta, co-founder of UK-based AI startup Sqwish, offered another perspective: benchmark tests are more like "an invitation to try the model," while continuous tuning is the reason customers stay. Sqwish recently announced completing a $3 million pre-seed funding round. The differentiation of its decision model, "kettei fat," does not lie in initial benchmark scores but in built-in post-training tools—customers can continuously upload their own production data, allowing the model to constantly improve judgment calibration for specific tasks.

This logic aligns with the views of Diogo Almedia, CEO of TypeSafe: the most difficult part of building Jev was finely calibrating the model's judgment using statistically adjusted synthetic data. Sqwish also uses synthetic data but adds the capability for continuous fine-tuning with customer-owned data. Sqwish co-founder Federica Freddi positions this track as "System 1 intelligence"—working in concert with the "System 2 intelligence" of frontier large language models. She believes the market size for this layer is in the tens of billions of dollars, and for frontier labs like OpenAI, decision models are merely side projects.

Sources :TechCrunch

Recherche

AWS Makes Open-Source Time-Series Language Model OpenTSLM Available in SageMaker JumpStart

Sujet · OpenTSLMPrise rapide
2026-11-18 02:06 GMT+8

Amazon Web Services announced that OpenTSLM, an open-source family of time-series language models developed by Aionic Labs, is now available in Amazon SageMaker JumpStart. The model combines numerical time-series data with text context such as maintenance records and market news to interpret operational measurements like equipment sensor data and supply-demand signals, aiming to reduce the high token costs of using general-purpose LLMs for long-horizon industrial metrics. The OpenTSLM-Flamingo variant in JumpStart is built on Gemma 3 4B with 5.24 billion parameters stored in bfloat16, requiring approximately 10.5 GB of GPU memory.

Sources :Amazon Web Services

Recherche

Google to Separate Gemini Chat History and AI Training Settings

Matériel
2026-11-18 02:01 GMT+8

U.S. tech giant Google has announced that it will update the privacy settings for its AI assistant, Gemini, in the coming weeks, addressing a long-standing user pain point: previously, the only way to prevent conversation data from being used to train AI models was to delete all chat history.

According to ZDNET, Gemini currently features a setting called "Keep Activity" that bundles two functions: controlling how long chat records are saved and deciding whether those records are used for AI training. Users cannot disable one without affecting the other. This design forced privacy-conscious users to choose between retaining their data and protecting it from being used for training.

To correct this issue, Google will replace "Keep Activity" with two independent settings. The first allows users to choose whether to keep past chats and for how long; the second specifically controls whether chat content, including text and attachments, is used for model training. In an email to users, Google stated, "With this change, you will be able to keep your history while independently controlling generative AI model training."

This adjustment brings Google's practices closer to those of other major AI providers. For instance, OpenAI's ChatGPT offers a toggle for "Improve the model for everyone," Microsoft Copilot provides options for training on conversation and voice activity, and Anthropic's Claude disables AI training by default. On these platforms, users can typically opt out of training without clearing their conversation history. Google was previously the only major AI player to enforce this bundling, making this change a significant concession in its privacy policy.

Sources :ZDNET

Recherche

Anthropic Launches 'Linked Billing' Beta on Claude Platform to Securely Bill Enterprise Customers via Third-Party Tools

Sujet · AnthropicMatériel
2026-11-17

US-based artificial intelligence company Anthropic announced on November 17, 2026, the launch of "linked billing" in beta on the Claude Platform. This new feature aims to address a core security and compliance pain point for enterprises using Claude APIs through third-party AI agents and automation tools: the traditional model requires users to share API keys with external platforms, effectively handing over a password, which poses leakage risks and makes usage tracking difficult.

Under the linked billing mechanism, end customers simply link their existing Claude Platform organization account to authorize third-party platforms to initiate requests on their behalf. Costs are billed directly to the customer's Anthropic invoice, where they can set monthly spend limits and unlink at any time. For platforms, this eliminates the need to store, encrypt, or rotate customer keys, and removes the burden of building custom metering and invoicing systems, thereby lowering integration barriers and security overhead.

Partners currently supporting this feature include major productivity and development tools such as Asana, ClickHouse, Factory, Gumloop, Harness, Higgsfield AI, Kilo Code, Linear, Major, Mintlify, Omni, OpenArt, Niteshift, Railway, Ramp, Runlayer, Tessera, Tines, and Warp. Anthropic states that platforms can view usage for each linked customer via the Console or programmatically through the Usage and Cost Analytics API, enabling more granular cost auditing.

This move reflects a shift in AI infrastructure competition from pure model capability to enterprise-grade governance and integration convenience. By eliminating the complexity of API key management, Anthropic seeks to accelerate the scaled deployment of AI agents within business workflows while maintaining strict permissions over data security and budget control.

Sources :Claude

Recherche

Replit Launches Missions: Budget-Constrained AI Agent Swarms for Complex Development Tasks

Sujet · Replit智能体群任务Matériel
2026-11-18 01:37 GMT+8

US-based AI coding platform Replit launched a new product feature called Missions on November 17, 2026. The feature allows users to set specific objectives, establish budget limits and approval requirements, and deploy "swarms" of artificial intelligence agents to work collaboratively to pursue these goals over hours, days, or even weeks.

The core mechanism of Missions is the "Mission Agreement," where users define success criteria, spending limits, and actions requiring human approval before starting. Users simply describe their goal in natural language (e.g., "make every page on a website load in under 500 milliseconds") without needing to devise a detailed plan upfront; the agents determine the best approach themselves. Replit President Michele Catasta stated that this design aims to prevent humans from worrying about implementation details, thereby preserving the flexibility AI systems need to solve complex problems.

To control costs, Replit checks spending against the preset budget at every step, blocking further use of paid services once the limit is reached. Catasta noted that most autonomous or multi-agent products lead to runaway token costs, making strict budget controls crucial. Additionally, each agent operates in an isolated cloud environment with access to development tools and infrastructure, coordinated by a dedicated conflict-resolution agent for code merging.

Replit reported using Missions internally to build its application monitoring platform, Radar. According to company self-reported data, agents completed the system's development in two days with a $3,000 budget, identifying database faults within 38 seconds in tests. Missions is now available to customers, while the Radar platform built upon it is planned for open-source release after further reliability improvements.

Sources :The New Stack

Recherche

AMD ROCm Team Proposes Configuration Pinning by Kernel Text as Key, Control Experiments Show Glue Acceleration Comes from Configuration Rather Than Kernel Body Replacement

Sujet · AMD ROCmMatériel
2026-11-17 08:00 GMT+8

When compiling inference graphs, TorchInductor generates a list of candidate configurations for each glue kernel (pointwise, reduction, persistent-reduction). After autotune timing, it writes the winner into a cache keyed by fusion number. This mechanism has two critical flaws: first, it is a closed choice, timing only the candidates listed by the compiler; if a faster tile is not on the list, it will never be selected. Second, the cache key is the fusion number rather than the kernel content; when an upstream template changes, the numbers rearrange, invalidating the cache. Additionally, another process on the same GPU can make the fastest configuration appear slower. In a ROCm blog post dated November 17, 2026, Yuchen Lin, Huan Li, Chunhung Wang, and Chao Chen proposed moving the search outside the compilation phase: offline scanning of candidate configurations, retaining only those that pass numerical checks and are confirmed to be faster via tracing. The final selection uses the SHA-256 fingerprint of the kernel text (with numeric suffixes removed and truncated to 20 hexadecimal characters) as the key, writing both the selected configuration and the kernel text into the AOTInductor package. At startup, there is no timing of candidates; the pre-verified configuration is loaded directly.

The core validation comes from a control experiment: placing the configuration pin file from the Tuned package back onto the Inductor compiler kernel body (without replacing any kernel source code). On GPU 6 running Model A, the glue time measured was 1567.2 us, essentially consistent with the Tuned result of 1598.7 us. On Model B, the compiler kernel body combined with the same configuration pin file yielded 391.2 us, compared to 350.88 us for Tuned. Latency figures also aligned closely. This demonstrates that the 1.388x and 2.846x speedups are almost entirely due to the configuration search; the AITER kernel body replacement (which involved only three SiLU kernels on Model A) is not a necessary condition.

Applicability boundaries include: fixed shapes, ensuring that one measured configuration corresponds to the production launch; this method is ineffective when the hot path remains an untuned GEMM or attention operation; pinning loses meaning when shapes change at every step; and it offers no help when the time bottleneck is launch overhead that the kernel body cannot fuse away. All data represents self-reported results from AMD in an environment featuring GFX942 (80 CUs), ROCm 7.2.2, and PyTorch 2.9.1. For Model A, the variance among three open-list compilation results reached 245 us, which exceeds the advantage Tuned holds over the fastest compilation; therefore, this advantage is not listed as a speedup ratio.

Sources :ROCm Blogs

Recherche

Google DeepMind Launches Gemini 4 Hydrogen and Antigravity Robotics Plugin, Claiming Hours-Level Development Cycle

Matériel
2026-11-17

Google DeepMind has launched Gemini 4 Hydrogen, a new model variant optimized for speed and spatial reasoning, alongside the Antigravity Robotics plugin. Announced on November 17, 2026, this combination aims to address the primary bottleneck in robotics: the high time and technical cost required to move from concept to deployment. The company states that developers can now go from unboxing a robot to running a working application in a single afternoon.

Gemini 4 Hydrogen follows the recent releases of Gemini 4 Argon and Carbon, focusing specifically on embodied reasoning while maintaining lower inference costs. On the Artificial Analysis Robotics 0.1 benchmark, Hydrogen scored 58, performing similarly to Argon but at a lower cost per token. The model demonstrates the ability to plan tasks, write control code, and adjust actions in real-time using camera inputs, directly controlling six-axis robotic arms.

The Antigravity Robotics plugin integrates hardware bring-up, MuJoCo simulation, and perception/control logic into Google's Antigravity development platform. By connecting via USB, developers can use natural language prompts to automate driver installation, calibration, and simulation testing. In one demo, a snack-sorting conveyor system was developed from a whiteboard sketch to a functioning physical prototype in approximately one hour.

While Google showcased support for various hardware platforms like I2RT YAM, SO-ARM101, and FR3 Duo, claiming a seamless 'Real-to-Sim-to-Real' workflow, these results rely on vendor self-testing and specific demonstrations. The feature is currently available to Google AI Pro and Ultra subscribers.

Sources :developers.googleblog.com

Recherche

Ray Train Breaks Down GPU Starvation into Six Timed Stages to Help Engineers Identify Where to Fix

Sujet · Ray TrainMatériel
Vérifié 2026-11-18 02:02 GMT+8

In large-scale model training, fluctuating GPU utilization is the most common signal of performance loss. In the third post of the Ray Train blog series, Anyscale engineer Jason Li points out that a typical cause of this fluctuation is GPU starvation: the training step completes, but the next batch of data is not yet ready, leaving the GPU idle. The problem lies in the fact that there are multiple reasons for data delays—a slow upstream data pipeline, data transmission from other nodes over the network, heavy preprocessing on the training worker's collate_fn, or simply the host-to-device copy taking too long. Each reason requires a completely different fix, yet the symptoms look identical.

Starting with Ray 2.58, Ray Train and Ray Data begin recording how much time each batch spends in each of the six data loading stages. These six stages are sequentially: production wait (waiting for the upstream Ray Data pipeline to produce the next block), data transfer (fetching the block to the training worker, potentially across nodes), batching (slicing and merging blocks into batches, including local shuffle buffers), format conversion (converting to requested formats like NumPy or pandas), collate (executing user-defined per-batch preprocessing on the training worker, such as tokenization or sequence padding), and final copy (moving the batch to the GPU, i.e., host-to-device transfer).

These stages do not execute serially. A background thread pool concurrently prepares the next batch while the current training step runs. As long as the next batch is ready before the training step ends, all loading work is hidden and does not consume wall-clock time. If preparation falls behind, the training loop must wait, causing the GPU to idle. Ray records this wait time (referred to as exposed data loading time) as well as the time spent in each stage on every training card.

The diagnostic path involves three steps. First, check Max Exposed Data Loading Time: if it is zero, data loading keeps up, and tuning the pipeline will not help; if non-zero, this duration represents the GPU idle time per step. Second, examine Percentage Data Loading Breakdown by Stage: the time share of each stage directly points to the bottleneck. A large share of production wait indicates insufficient output from the upstream pipeline; a large share of collate indicates excessive preprocessing on the training worker. Third, split the stage metrics by rank: if one rank is significantly slower than others, it is a straggler. Since all ranks synchronize at every step, a single slow rank can drag down the entire training run.

The article provides specific operational advice for each bottleneck stage. Slow production wait: check the Ray Data dashboard for the slowest operator in the upstream pipeline and tune its parameters, or add CPU-only nodes to allow Ray Data to scale out preprocessing, or use materialize() to cache preprocessed results in memory. Slow data transfer: increase prefetch_batches to start transferring blocks earlier, but be mindful of training node memory usage. Slow batching: reduce local_shuffle_buffer_size or switch to upstream shuffling. Slow format conversion: similarly increase prefetch_batches. Slow collate: move heavy preprocessing from collate_fn to map_batches in the Ray Data pipeline, allowing it to scale across the cluster rather than executing serially on the training worker. Straggling single rank: check if that node has Ray Data tasks competing with the training worker for CPU or memory, or use label_selector to pin the dataset to CPU-only nodes.

Sources :Anyscale

Recherche

NVIDIA Engineer Proposes User-Space Control for Preferred CPU Cores in Linux Kernel

Prise rapide
2026-11-17 19:16 GMT+8

NVIDIA engineer Yury Norov has submitted a Request for Comments (RFC) patch series to the Linux kernel mailing list, proposing a new sysfs control that allows user-space applications to express preferences for specific CPU cores. If adopted, this mechanism would enable user-space daemons like Thermald to schedule tasks on particular cores based on factors such as CPU temperature or other external conditions.

The proposal builds upon the 'steal governor' merged in Linux 7.4, which helps reduce vCPU contention in virtual machine environments. Norov's patches extend this control to user space via a writable interface at /sys/devices/system/cpu/cpu[0-9]*/preferred. The documentation cites a use case where AI training and inference VMs run on the same server: since inference is more latency-sensitive, a user-space manager could dynamically ensure the inference VM retains more 'preferred' cores to optimize performance. The patch series is currently under review and has not yet been merged into the mainline kernel.

Sources :phoronix.com

Recherche

Lightspeed Leads $45M Seed Round for Compute Desk to Build AI Compute Clearing Platform

Matériel
2026-11-18 00:01 GMT+8

Venture capital firm Lightspeed announced it is leading a $45 million seed round for startup Compute Desk. The company is building a clearing platform for AI compute, addressing the core issue where buyers cannot lock in prices, sellers cannot lock in earnings, and lenders cannot assess asset value in current compute transactions.

In its investment statement, Lightspeed noted that while compute is a critical input for the AI economy, trading still relies on outdated contracts and guesswork. With the four largest hyperscalers expected to spend approximately $700 billion on capex this year alone, and private credit directing an estimated $800 billion into compute and data centers over the next two years, the existing model of long-term offtake agreements is struggling to match rapid changes in supply, demand, and risk pricing.

Compute Desk's solution involves three layers: trusted indices built from real transactions rather than scraped list prices, hardware-level testing and monitoring to guarantee delivery, and a software platform connecting buyers and sellers to exchanges. The company has partnered with venues like Nodal and Architect to license its price indices for contract trading.

Founder Andy Bahou previously built GPU data centers at Meta and worked as an exotic derivatives quant at Credit Suisse. Co-founder David Lopez Mateos holds a PhD in particle physics from Caltech, contributed to the Higgs boson discovery, and worked as a commodities quant at Winton. Lightspeed argues this combination of infrastructure-building and capital-markets expertise positions the team to define how the compute market clears.

Sources :Lightspeed Venture Partners

Recherche

Irish Data Protection Commission Forces X to Halt Grok Training on Third-Party Data

Sujet · GrokMatériel
2026-11-18 00:59 GMT+8

According to Politico, the Irish Data Protection Commission (DPC) has pressured social media company X to stop using "third-party data" to train its artificial intelligence system, Grok. X confirmed to the authority on Monday that it had suspended the relevant data processing activities. This move is considered a significant test case for the privacy compliance of AI agents.

Previously, AI agents like Grok-Bot began automatically scraping personal information shared by other users (such as photos or data involving non-users) from the internet starting in October, using it for model training. DPC Commissioner Dale Sunderland noted that regulatory attention is shifting from content actively shared by users to personal data automatically collected by AI agents involving uninvolved parties. Sunderland emphasized that the DPC does not object to AI agents themselves but must scrutinize whether companies operate within legal boundaries when harvesting and using such data.

This intervention marks a new phase in global privacy enforcement: as AI agents increasingly browse and collect information on behalf of users, legal boundaries are being redrawn. The DPC stated it is currently in discussions with another unnamed tech giant regarding similar plans to use agent-collected data for AI training.

Sources :heise online

Recherche

NVIDIA Engineer: The Cost of Sparse Attention Has Not Disappeared, It Has Just Moved into the Indexer

Sujet · 稀疏注意力索引器成本Matériel
2026-11-18 01:30 GMT+8

Sparse attention is often described as the cure for long-context inference: each query attends only to the k most relevant keys, so the compute cost of core attention no longer grows with context length. However, in a co-design guide published by the NVIDIA Technical Blog team in November 2026, they point out that this description obscures a key fact—the selected k keys must be found, and the indexer responsible for finding them still scans all N keys. The cost has not disappeared; it has simply moved from core attention into the indexer. In DeepSeek V3.2’s 128K prefill, the indexer and Top-k selection accounted for 47% of the time spent on an entire layer.

To understand why the indexer becomes a bottleneck, it needs to be reduced to hardware operations. An indexer is essentially a batched matrix multiplication followed by a fused post-processing step (ReLU and cross-head weighted sum). The matrix multiplication runs on Tensor Cores, while the post-processing runs on CUDA cores, finally writing one scalar logit per key back to HBM. What determines whether this operation is compute-bound or bandwidth-bound is its arithmetic intensity—the number of floating-point operations performed per byte of data moved.

The core conclusion derived by the NVIDIA team is that the indexer's arithmetic intensity is proportional to the group size G (equal to the number of query heads divided by the number of key heads), and the decode intensity grows approximately linearly with the speculative decoding draft length D+1. Intuitively, G query heads share the same key head, and their scoring results are reduced to a single logit via weighted sum, meaning that for every logit written, G times the work of a matrix multiplication is done. Speculative decoding allows D+1 query tokens to be scored in a single pass over the key cache, further amortizing the bandwidth overhead.

However, this intensity curve has a hardware ceiling. A Tensor Core tile can hold at most 256 query rows, but under FP32 post-processing, register pressure limits practical usage to about 200 rows. G×(D+1) must fit within this tile; otherwise, the kernel launches more CTAs, causing runtime to jump stepwise. NVIDIA’s measurements show that at G=32, up to six draft tokens remain effectively free (runtime increases only 1.37x while workload grows 6x), and at G=8, up to 26 draft tokens can be accommodated. Beyond this boundary, benefits vanish immediately. This means group size and speculation depth compete for the same tile space—you cannot push both to their limits simultaneously.

Starting from this constraint, several design choices become clear. The number of key heads KH must be 1: adding another key head requires reading the entire key cache again during decode, causing linear runtime growth (runtime increases 7.6x when KH goes from 1 to 8), while FLOPs remain unchanged. The head dimension Hsz should be at least 128: below this value, Tensor Cores wait for CUDA cores to finish post-processing, resulting in insufficient hardware utilization; above 128, the bytes per key increase during decode, raising bandwidth pressure. FP16 post-processing expands the tile from 200 rows to 256 rows, providing a 1.31x speedup in a configuration of G=64 and D+1=4.

Sources :NVIDIA Technical Blog

Recherche
Suivant →
Connexion / Inscription
LectureRechercheRadarCadre d'investissement

It should be noted that this is an official blog post from Anyscale describing its product features. The article does not provide quantitative comparisons before and after using these metrics in real production environments (e.g., how much diagnostic time was shortened or how many percentage points utilization increased). Open-source Ray users need to set up Prometheus and Grafana themselves to view these metrics; the Anyscale platform includes built-in display. For senior teams already familiar with these engineering practices, the advice for each step of the diagnostic path itself is not novel. The new value lies in the stage-level timing metrics, which transform unstable GPU utilization into a problem that can be traced to a single bottleneck, rather than relying on guesswork.

Compression ratio CR does not change arithmetic intensity but linearly reduces total workload. CR=4 means four tokens correspond to one indexer key, reducing the workload of both the indexer and Top-k to one-quarter. The NVIDIA team emphasizes that compression, hierarchical indexing, and cross-layer index sharing are three methods to reduce the number of keys scanned by the indexer; these are orthogonal to the intensity optimizations mentioned earlier—they should be used together, not as substitutes for one another.

The cost of Top-k selection also follows the number of keys scanned, rather than the number of values retained k. NVIDIA’s measurements show that doubling the number of keys increases Top-k time by about 1.7x (prefill) and 1.5x (decode), while increasing k by 64x raises time by less than 2x. Two kernel techniques further compress this overhead: fusing Top-k into the indexer kernel eliminates the write-out and read-back of logits; in decode, using the selection result from the previous step as a guess allows verification in a single pass, running 2.1 to 2.4 times faster than standard radix select on DeepSeek V3.2 and V4.

Finally, selection granularity determines the efficiency of core attention. Token-level selection (such as DeepSeek DSA) scatters the k keys across various locations in the cache, where each query has a different access pattern, making it nearly impossible for two queries to share a fetch. Block-level selection (such as GLM 5.3 Flash, QSA) retrieves keys as contiguous blocks, serving an entire block with a single address service, allowing multiple queries to share the fetch of the same block. NVIDIA measurements indicate that with a budget of 2,048 selected keys, halving the block size from 128 to 64 almost doubles the core attention runtime—because each block introduces one query load, one partial result, and one merge step.

It is important to note that all the above measurements and conclusions come from NVIDIA’s own production-grade FP8 kernels running on NVIDIA GPUs. Tensor Core tile sizes, the scheduling relationship between CUDA cores and Tensor Cores, and HBM bandwidth characteristics are specific to NVIDIA architecture. The applicability of these guidelines to AMD MI series, Google TPUs, or other accelerators is unknown. Additionally, the acceleration ratios cited are vendor self-tests, with no independent third-party reproduction available yet.