ReadingResearchRadarInvestment framework
Sign in / Sign up
AllTechnologyProductsInvestingPublic marketsOther

Reading

2026-11-17100 posts

Refresh

NVIDIA Vera Rubin Platform Achieves Up to 2x Blackwell Ultra Performance in MLPerf Training v6.1

Material
2026-11-17 00:00 GMT+8

NVIDIA announced that its next-generation AI computing platform, Vera Rubin NVL72, debuted in the latest MLPerf Training v6.1 benchmarks, delivering the fastest time-to-train on six of seven tests. Compared to the highly optimized previous-generation GB300 NVL72 baseline, Vera Rubin NVL72 achieved up to 2x more performance per GPU. This result marks a significant step forward in large-scale AI training efficiency for the new architecture.

In the most demanding DeepSeek-V3 671B Mixture-of-Experts (MoE) model test, Vera Rubin NVL72 completed training in 4.14 minutes using 1,024 GPUs, while GB300 NVL72 required 8.03 minutes at the same scale. This halves the time-to-train, effectively doubling the number of experiments an AI research team can run in a quarter. Additionally, the platform maintained above 97% scaling efficiency when expanding from 256 to 1,024 GPUs, highlighting its networking architecture advantages for communication-heavy MoE workloads.

MLPerf Training v6.1 also introduced its first agentic LLM reinforcement learning (RL) post-training benchmark, Qwen 3.5 397B GRPO. Vera Rubin NVL72 led this new test with a time of 10.44 minutes, compared to 13.78 minutes for GB300 NVL72. This reflects a fundamental shift in AI development towards agent capabilities and the new system demands of post-training. NVIDIA stated that the Vera CPU played a key role, sustaining more than 22,500 concurrent RL environments to ensure GPUs are continuously fed with feedback data.

It is important to note that these results are official benchmark submissions by NVIDIA within the MLCommons framework, representing vendor self-tested results. While they follow standardized procedures, independent third-party reproduction has not yet been publicly reported. NVIDIA also noted that continuous software stack optimizations have led to faster training times in subsequent unverified re-runs on the same hardware, indicating that infrastructure retains performance improvement potential after delivery.

Sources:NVIDIA Technical Blog

Research

LanceDB Engineer Weston Pace: Multi-Base Paths Allow Migration of Tables with 10 Million Files Across Buckets by Changing Only 5 Strings

Material
2026-11-17 01:19 GMT+8

Uber’s AI infrastructure team presented LanceDB with a production requirement: distribute a single Lance dataset across multiple S3 buckets to increase throughput in scenarios involving petabytes of multimodal data and thousands of concurrent readers. This need exposed a fundamental tension in table format design—absolute paths sacrifice portability, while purely relative paths cannot reference files outside the dataset root. Uber needed both: multi-location storage and maximum portability.

Weston Pace, who has contributed to the development of Apache Iceberg and Delta Lake, observed that this debate over path design took a detour in two mainstream formats. Iceberg initially adopted absolute paths, arguing that they are unambiguous, allow files to be placed anywhere, and require no context for resolution. However, in multi-cloud environments, migrating a table means rewriting every metadata file containing millions of absolute paths. Starting in 2021, the community began discussing relative path support, and by late 2025, alignment on the v4 relative path design was nearing consensus, though not yet finalized. Delta took the opposite approach: starting with relative paths, it added absolute paths in 2021, laying the groundwork for shallow cloning introduced in late 2022. This evolved into a hybrid model, gaining flexibility but losing its original portability guarantees.

Pace argues that the common flaw in these designs is that the flexibility of absolute paths is rarely used in practice—files almost always reside under the table root. Moreover, once files span multiple locations, garbage collection, credential distribution, and migration reasoning become difficult. Crucially, the zero-rewrite promise begins to crack when operations requiring absolute references are introduced, such as shallow cloning, file imports, tiered storage, or multi-bucket distribution. What is truly needed is not flexibility, but maximum portability: migration should be simple, fast, and predictable regardless of the table's current distribution topology.

Lance’s starting point differs from other formats. It rests on two design premises: predictability over flexibility—all files reside within a known directory structure (data/, _versions/, _transactions/, _deletions/, _indices/), making path resolution deterministic; and strict portability—every path is relative, so copying a dataset makes it work immediately without updating metadata. With these premises established, the problem was reframed: how can we keep paths relative while allowing a single table to span multiple storage locations?

The answer is an explicit concept called a Location Base. A BasePath array is added to the manifest, where each entry contains an id, name, is_dataset_root flag, and path string. Files reference their base via a base_id (a protobuf varint encoding, occupying only 1 byte for up to 128 bases) plus a relative path within that base. When is_dataset_root is true, the base is treated as a standard Lance dataset root, requiring the addition of a data/ subdirectory to resolve data files; when false, the base points directly to a file directory, suitable for multi-bucket distribution scenarios.

Sources:lancedb.com

Research

FBI warns: AI hacking tool ARTEX breaches networks in under three hours

Material
Verified 2026-11-17 01:47 GMT+8

The Federal Bureau of Investigation (FBI) released Cybersecurity Advisory FLASH-20261113-001 on November 13, 2026, stating that the open-source, large language model-driven autonomous penetration testing framework ARTEX has been weaponized by threat actors to automate multiple phases of cyber intrusions. The advisory highlights that ARTEX significantly reduces the time required to chain attack techniques, with some observed intrusions progressing from initial targeting to obtaining privileged access in under three hours.

Since at least August 2026, malicious cyber actors have used ARTEX to target organizations globally across sectors including financial services, government, technology, defense, and healthcare. The framework lowers the technical barrier to entry, enabling threat actors with minimal expertise to conduct complex, multi-target operations simultaneously. Notable campaigns include attacks against South Korean financial institutions resulting in the exfiltration of personal information from approximately 67,000 individuals, and a supply chain compromise affecting ConnectWise Cloud Backup customers that enabled access to more than 2,250 downstream tenants.

Although the original developer removed the GitHub repository in October 2026 following reports of abuse, ARTEX continues to proliferate through community forks, localized variations, and commercial versions such as ARTEX PRO. As of early November 2026, researchers identified over 1,600 publicly exposed ARTEX console instances. The FBI encourages organizations operating internet-exposed applications, self-service portals, cloud infrastructure, and partner systems to implement recommended mitigations to reduce the likelihood and impact of ARTEX-enabled intrusions.

Sources:ic3.gov

Research

ByteDance Volcano Engine: Three Mechanisms of Lance Format Solve Autonomous Driving Data Training Bottlenecks

Material
2026-11-17 01:19 GMT+8

Each autonomous driving test vehicle generates terabytes (TB) of multimodal data daily (camera images, LiDAR point clouds, audio), which rapidly expands to exabyte (EB) scale at mass production levels. In a case study published on the LanceDB blog, ByteDance Volcano Engine's LAS team described three specific bottlenecks encountered by their client (referred to as Client A) when managing such data: the need to rewrite the entire dataset when adding new columns for annotation iterations, high bandwidth pressure from point cloud data, and only 60% GPU utilization during training.

For the first type of problem, traditional solutions like LMDB or Pickle require rewriting the full dataset when adding new label columns, consuming significant storage and computing resources. The Lance format supports incremental Schema evolution through Manifest metadata—new columns are written only as metadata pointers, while historical data files remain unchanged. Volcano Engine claims this mechanism reduces storage costs by 30% and shortens the processing time for 10 PB of annotation data from four days to one day.

The second bottleneck is the transmission and storage bandwidth for point cloud data. Lance defines ZSTD encoding at the Schema level; compression is transparent to read/write operations, requiring no additional processing steps. Volcano Engine reports a LiDAR point cloud compression rate of approximately 70% (compressing 1 PB of raw data to 300 TB), an image data compression rate of about 5%, and an annotation metadata compression rate of about 40%.

The third bottleneck is training efficiency. Traditional I/O paths cause amplification effects due to full reads, leaving GPUs waiting for data. Lance’s point query support enables column projection (reading only fields required for training) and lightweight Shuffle based on row indices, avoiding manual index construction. Volcano Engine states that column projection and lightweight Shuffle increased training GPU utilization to 96%; on the inference side, the team reported utilization rising from 60% to 90% on 8×A100 GPUs, with corresponding inference throughput increasing by 50%.

It should be noted that all the above metrics are self-reported data from the ByteDance Volcano Engine team, and the case was published on the official LanceDB blog with the client identity kept anonymous. There is no independent third-party reproduction, public test scripts, or verifiable hardware and dataset configuration information. While the logic of the mechanism design (incremental metadata, built-in Schema compression, column projection) can be independently verified from Lance open-source documentation, the specific performance numbers should be regarded as vendor claims.

Sources:lancedb.com

Research

NVIDIA Publishes Provisional MLPerf Results: Vera Rubin Platform Shows >10x Energy Efficiency Gain Over GB300 in Agentic Inference

Material
2026-11-17 00:00 GMT+8

NVIDIA published preliminary results for its latest AI superchip platform, the Vera Rubin NVL72, in the MLPerf Endpoints v1.0 benchmark suite on November 16, 2026. The data indicates that at a shared minimum end-to-end interactivity of 50 tokens per second per user, the Vera Rubin NVL72 delivers over 10 times the throughput per kilowatt of the previous generation GB300 NVL72. This result highlights the new platform's energy efficiency advantages when handling agentic workloads involving multi-turn conversations and tool calls.

The tests covered three open-source models: Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.6-35B-A3B. At matched query concurrency, the Vera Rubin NVL72 achieved 3x to 5x higher end-to-end interactivity. Conversely, at fixed interactivity thresholds, it demonstrated 5x to 10x higher throughput per kilowatt. For instance, running Kimi K3 at 1,024 concurrent queries, the Vera Rubin system achieved 41,414 tokens/sec with 100.89 kW power consumption, whereas the GB300 system reached only 13,905.8 tokens/sec at 60 kW under the same concurrency.

NVIDIA attributes these gains to the Rubin GPU architecture's HBM4 memory, Tensor Cores, and sixth-generation NVLink, combined with context-aware serving techniques like prefix caching, state reuse, and Dynamo routing. These optimizations reduce redundant computation and balance load. Additionally, model-specific execution optimizations, such as DeepSeek's MegaMoE kernel and Qwen's multi-token prediction with Tensor Core verification, help minimize data movement and sequential decoding steps.

It is crucial to note that these results are marked as "provisional" and have not yet undergone final peer review and verification by the MLCommons Association, meaning they are subject to change. The GB300 results used for comparison have been verified by MLCommons. For readers tracking AI infrastructure efficiency, this data offers an early reference for new hardware performance under standardized agentic loads, though real-world production metrics await independent validation.

Sources:NVIDIA Technical Blog

Research

ProfitScore Market Extremes Index Reads 98/100: SSRN Paper Author John McClure Advocates Stress Testing Over Predicting the Top

Material
2026-11-16 21:31 GMT+8

In his SSRN-published paper "Priced to Perfection," John McClure constructs a six-component composite index that ranks U.S. stock market valuation, profitability, and leverage levels individually before averaging them, yielding the ProfitScore Market Extremes Index. As of August 2026 data, the index reads 98/100, placing it at the 99th percentile of its own historical record. In the abstract on Quantpedia, McClure explicitly states: The related inputs and limited history of extreme events for this composite index constrain its effectiveness as a forecasting tool—but portfolio decisions need not rely on predicting the market top.

The index's core mechanism hinges on profit margin assumptions. The S&P 500's price-to-sales ratio currently stands at 3.78x, approximately 77% higher than the dot-com bubble peak, while the P/E ratio of 27x is actually lower than the 29x seen then. McClure argues that this gap is almost entirely supported by today's abnormally high profit margins—margins that are 89% higher than during the internet era. This implies the investment thesis relies heavily on a single assumption: that current margin levels will persist. Structural improvements might support higher profitability, but even partial normalization would weaken earnings per share unless revenue growth is sufficient to offset it. If multiples contract simultaneously, these two effects will compound.

Relative valuations are equally unsettling. The CAPE yield exceeds the 10-year TIPS yield by only 0.03 percentage points, the lowest equity risk premium recorded in the TIPS era. McClure acknowledges this spread is an imperfect measure of expected equity compensation, but it does raise the threshold for accepting equity risk: investors require a convincing earnings-growth outlook to justify bearing additional uncertainty. Meanwhile, margin debt growth has outpaced the market itself by 17 percentage points, constituting a potential source of forced selling.

McClure likens this leverage dynamic to the load line once required by the UK Parliament to be painted on every ship's hull: exceeding this line carries the risk of sinking. His ProfitScore Margin Extremes Index and its marked line previously signaled two months before the market tops in 2000, 2007, and 2022. The current market has been operating above this line for twelve months. His actual advice is not to sell or time the market, but to conduct stress tests: assess the consequences of weakening margins, contracting multiples, and simultaneous stock and bond declines, then judge whether expected returns are sufficient to bear these scenarios. For portfolio managers, this is more feasible than predicting the top—it does not require knowing when the turn will occur, only whether one's portfolio can withstand it if it does.

Boundaries to note: The composite index has only reached above 90 in two periods, resulting in a very small sample size; four of its six components have been ranked since January 2000, one joined in 2002, and one in 2005, meaning the full six-component history itself is quite short. McClure himself acknowledges these limitations. Additionally, the paper utilized Anthropic's Claude for assisted editing and data entry, with all analysis and conclusions remaining the author's responsibility. We rely on Quantpedia's summary and citations of the SSRN paper; the original paper could not be directly accessed due to access restrictions.

Sources:quantpedia.com

Research

NVIDIA, IBM and Fermilab Announce Quantum-Classical Codesign Platform Collaboration; Release TCBP-JOBS Decoding Algorithm

Material
2026-11-17 00:00 GMT+8

NVIDIA, IBM and the U.S. Department of Energy’s Fermi National Accelerator Laboratory announced on November 16, 2026, a new collaboration aimed at developing next-generation modular architectures for quantum computers, scientific applications and supercomputing integration. The core of this effort is a jointly developed “quantum-classical codesign platform” that combines NVIDIA’s accelerated computing and software with IBM’s quantum computers to help researchers scale quantum systems and explore their scientific capabilities.

As part of the collaboration, IBM and NVIDIA are colocating NVIDIA DGX B200 systems with state-of-the-art IBM quantum computers, including those powered by the IBM Quantum Nighthawk processor. This platform allows researchers to develop and evaluate error-correction codes, decoding algorithms and quantum-classical applications driving scientific discoveries in high-energy and nuclear physics (HENP). Fermilab’s expertise will focus the platform’s work on quantum error correction and HENP applications, helping define requirements for future fault-tolerant quantum computers.

Separately, NVIDIA released a new decoding algorithm called TCBP-JOBS, now available through the CUDA-Q QEC 0.9.0 library. According to NVIDIA, TCBP-JOBS accelerates the computationally intensive parts of quantum error-correction research by up to 13,000x, cutting the computation time for researching new quantum computing designs from 12 weeks to just 45 minutes. This improvement is intended to help scientists more efficiently evaluate candidate error-correction approaches and identify designs that use physical qubits more effectively, shortening the path to practical fault-tolerant quantum computing. Note that the speedup figure is a vendor-reported result specific to decoding in research workloads.

NVIDIA is also expanding the open-source Slurm workload manager with a quantum interface to simplify integrating quantum processors into supercomputing environments. IBM is contributing to this integration through its Quantum Resource Management Interface. These moves indicate that major technology companies and national laboratories are attempting to address engineering and maintenance bottlenecks in merging quantum computing with traditional high-performance computing by standardizing software and hardware interfaces.

Sources:NVIDIA Newsroom

Research

TokenEconomizer: How Repository Experience Reduces Token Costs for AI Coding Agents by 20% to 30%

Material
2026-11-17 01:01 GMT+8

Developers use AI coding agents in the same codebases day after day, yet each session starts from scratch: rediscovering how to run tests, loading tools that are never called, and spinning in circles on the same error. Microsoft Research, UC Berkeley, and the Microsoft Security team profiled Claude Code across 60 real bug-fixing tasks on SWE-Bench to quantify the scale of this waste. They identified three recurring patterns: once a repository had accumulated at least five completed tasks, 30% of tool calls repeated actions from earlier tasks in that repository; in every turn, the agent received 23 tool descriptions, 16 of which were never invoked; and 17% of failed tasks consumed 32% of the total budget.

The core idea behind TokenEconomizer is to turn completed sessions into guidance for subsequent ones, without modifying the agent or the underlying model. It operates through three mechanisms. Micro-skills identify recurring patterns across multiple sessions and generate one-line notes added to the agent’s starting context; these are only added after several sessions confirm the pattern and are retired if later evidence contradicts them. Tool pruning removes tools that the repository history suggests the agent is unlikely to use, while always retaining core tools for running commands and editing files; removed tools can be restored if needed. Failure-loop intervention detects whether an agent is repeatedly hitting the same error, using a judge model to determine if it is truly stuck; if so, the agent receives a short note suggesting a change of direction.

In tests across 277 SWE-Bench tasks, six repositories, and five agent-and-model combinations, TokenEconomizer reduced costs by an average of 18.4%–30.9%, saving resources in all 30 repository-combination pairs with no performance loss. Savings grew as experience accumulated: early tasks might be more expensive, but by the sixth task, cumulative costs per repository fell below the default baseline. Costs dropped by 16.0%–21.2% on continuous web development tasks in Web-Bench, and by 6.4%–19.6% in security auditing (where each task involves a new repository).

Important boundaries to note: these figures come from vendor-reported benchmarks and lack independent replication; savings depend on the accumulation of repository history, offering no benefits during cold-start phases; the 18.4%–30.9% figure is a cross-task average, meaning individual task savings may be higher or lower. Additionally, the reported costs already account for the overhead of TokenEconomizer’s own model calls. The plugin is open-source and available on GitHub, requiring no additional API keys, with learned content retained locally on the developer’s machine.

Sources:Microsoft Research

Research

OpenAI Expands Academic Researchers Program to Provide Free Frontier AI Access to 200,000 Scientists

Material
2026-11-17 01:04 GMT+8

U.S.-based artificial intelligence company OpenAI announced on Monday that it is expanding its Academic Researchers program to provide free access to its frontier models for up to 200,000 researchers. This move doubles the number of participants originally offered when the pilot program launched in July.

The company will award 50,000 new grants on a first-come, first-served basis. Each grant allows the recipient to extend access to up to two collaborators. OpenAI stated that it will not collect data from users enrolled in this specific academic program.

This expansion complements other initiatives like ChatGPT Edu, which provides campus-wide access as universities increasingly integrate AI tools into their curricula and research infrastructure.

Sources:semafor.com

Research

Box Launches Context Retrieval and Architect, Leveraging Nvidia Nemotron to Optimize Enterprise AI Workflows

Material
2026-11-17 00:47 GMT+8

At its annual BoxWorks conference earlier this month, enterprise content management vendor Box introduced several upgrades to its agentic AI offerings. Two core new features stood out: Context Retrieval, a tool designed to help AI agents dig out relevant information from enterprise content stores, and Box Architect, an agent orchestration builder. Ben Kus, CTO at Box, confirmed in an interview that these functions aim to solve two major obstacles enterprises face when deploying AI: getting data ready for AI use and setting up and optimizing workflows.

The Context Retrieval tool is designed to make it easier for agents to find relevant information without users first having to curate source documents into a virtual content store or build a catalog of the entire repository. Kus noted that while general-purpose Large Language Models (LLMs) excel with public knowledge, they often struggle with an organization's specific terminology and proprietary data. The tool uses hybrid retrieval techniques to find context. According to Kus, the retrieval engine leverages Nvidia Nemotron technology, an open-weight model that allows enterprises to run it in their own private setups for enhanced security. By combining this with a vector database for semantic matching, the system significantly reduces token consumption and improves retrieval speed and accuracy compared to traditional keyword matching.

Box Architect, soon available in beta to Enterprise Advanced customers, is an agent orchestration builder created to apply the power of the latest generative AI models to building more effective enterprise workflows. Kus explained that many companies find it difficult to evaluate how various components and capabilities work best together to achieve business outcomes, which hinders AI adoption. Box Architect not only helps companies utilize Box capabilities to optimize existing processes but also automatically finds the best solution by iterating through different approaches. While the initial optimization consumes a lot of tokens, Kus argued that this upfront investment is worthwhile for automated processes that need to run thousands or millions of times, as it leads to long-term efficiency and accuracy gains.

In addition to these specific tools, Box also released industry-specific agent solutions for Banking, Public Sector, and Media & Entertainment, along with new threat protection and agent security controls. These updates reflect how enterprise software vendors are trying to bridge the gap between general AI models and actual operational needs by providing more refined context management and automation tools.

Sources:diginomica

Research

Together AI's Agent Evaluation Practice: Using Weaker Models to Find Gaps, Two-Sentence Document Fixes Raise GPU Cluster Task Pass Rate from 27% to 100%

Material
2026-11-16

When testing whether coding agents could correctly query GPU cluster pricing, Hassan El Mghari and others at Together AI observed a surprising phenomenon: even with skill packages enabled, Claude Sonnet 4.6 passed only 4 out of 15 attempts. The failure mode was highly consistent—Sonnet treated unmarked sample output from the CLI reference documentation as live inventory, skipped the region lookup, and guessed the cheapest option. The problem lay in Together's own documentation and links: the skill file pointed to the marketing page for GPU clusters rather than the pricing table. After opening the marketing page, the agent saw no actual prices and had to guess.

The fix was unexpectedly simple: adding two sentences to the skill file instructed the agent to check /v1/compute/regions for real-time regions and instance types, and to open the pricing table for current rates. After this change, the same task achieved a 100% pass rate across all tested models. However, this discovery came to light thanks to an easily overlooked evaluation strategy: using both strong and weak models simultaneously. During the same period, Opus 4.8 passed 13 out of 14 attempts with the exact same configuration—it noticed that the region response lacked prices and continued searching until it found them in the documentation. If Together had tested only with Opus, this gap might have remained hidden.

To address this, Together AI built an internal agent evaluation system. They first compiled a list of common tasks developers perform on the platform (such as batch inference, building voice agents, and managing auto-scaling), each essentially serving as a prompt for the agent. Then, for each task, they ran three configurations: Baseline (the agent has only default tools, simulating a standard developer workflow), Documentation MCP (direct connection to documentation via an MCP server), and Skill Packages (loading instructions and examples specifically written for the agent). Each task was run twice with three different models, generating hundreds of runs for a complete evaluation matrix. Scoring was handled by an LLM judge, which assigned one of three verdicts: Pass, Incomplete, or Fail.

The Together team emphasizes that fail and incomplete verdicts do not necessarily tell you where the problem lies—whether it is the agent, the API, or missing evidence; they simply indicate where to start investigating. They also examine passing runs because agents often manage to bypass confusing responses to complete tasks, and those detours themselves expose areas needing repair. The GPU cluster task also exposed an API-level issue: during a baseline run, Opus 5.5 received a 400 error when calling /v1/compute/instance-types. The error message suggested checking /v1/clusters/regions for a list of regions, but that route did not exist and returned a 404. Together's own error message sent the agent down a dead end, and human developers following the same prompt would have been stuck as well.

The second case involved restructuring the skill package. Together previously maintained 14 separate skills, divided by product surface, which overlapped with each other and could bloat context if multiple were loaded. After merging them into a single unified version, the initial evaluation passed only 54 out of 72 runs (75%), with costs doubling. Transcript analysis identified three problems: the router was too long, causing agents to read 57% more skill content; code examples referenced models that were no longer online; and example scripts contained minor bugs. After cutting the router to under 6,000 characters, replacing all examples with confirmed online models, and adding automated periodic live-testing, the unified skill package passed 70 out of 72 runs (97.2%), which is 7 percentage points higher than the original 14 skills.

Sources:together.ai

Research

Claire Vo, Founder of ChatPRD: AI Products Sell Compute Power; Free Trials and Promo Codes Have Become Attack Surfaces

Material
2026-11-17 00:02 GMT+8

Claire Vo, founder of ChatPRD, discovered that a fraudster had turned her custom Model Context Protocol (MCP) interface into a round-the-clock compute worker. This case reveals a new category of attack surface facing AI products: every fraudulent model call generates real cash costs.

Claire Vo recounted this experience on the How I AI podcast. Token fraud caused ChatPRD to lose tens of thousands of dollars and occupied much of her engineering time—even though the company had passed SOC 2 compliance audits, underwent regular penetration testing, and was operated by an experienced technical founder. She pointed out that this differs fundamentally from traditional SaaS abuse: every fraudulent model call incurs real cash costs.

She categorized six specific forms of attack: bulk registration of free trial accounts, resale of promo codes, credit card testing, product relaying, and the aforementioned MCP abuse. The promo code issue was particularly thorny—codes intended for partners were stolen and resold on secondary markets to provide compute power for other AI businesses. Upon discovering that roughly half of her promo codes had flowed to resellers, she completely stopped issuing them.

On the technical defense side, Claire Vo established a real-time classification mechanism: when a session exceeds ChatPRD’s set abuse threshold, a zero-data-retention Jev loop classifies the activity. Confirmed fraud results in either a ban or a silent downgrade to a cheaper model—the latter does not notify the user but rapidly reduces the attacker's economic returns to zero. She also used Vercel Firewall to block traffic from specific countries, which significantly reduced abuse but forced her to rethink ChatPRD’s internationalization strategy.

The cost of these defenses was concrete. Claire Vo ultimately canceled free trials, abandoning a key channel for product-led growth and acquiring enterprise leads. Credit card verification gates and paid trials failed to stop fraudsters because automated tools could bypass them in bulk. Her conclusion is that for AI products, trust must be earned through verification rather than granted by default.

The core principle she summarized is: if a company sells AI, it is essentially selling compute power. Founders should track dollars rather than request counts, continuously monitor model spending, close every free entry point, and verify the client rather than just the account. She also observed an interesting pattern: when expensive models are no longer available to be stolen, fraud quickly disappears.

Claire Vo found that the same group of fraudsters attacked multiple mainstream AI products simultaneously by rotating proxies. She has begun discussing the problem publicly and filed cybercrime reports with the United States Federal Bureau of Investigation (FBI) with the assistance of Codex. This indicates that token fraud is not merely an operational incident for individual companies but a structural risk facing the entire AI industry.

The boundaries of applicability for this experience are equally important. ChatPRD uses high-end models billed per token and offers free trials and promo codes as growth levers; these two factors together amplified the economic impact of fraud. For AI products using cheap models, pure pay-as-you-go pricing, or those targeting enterprise customers, the severity of the attack surface may differ significantly. However, the underlying logic is universal: as long as each call carries a real marginal cost, free or low-cost entry points constitute an attack surface.

Sources:lennysnewsletter.com

Research
Next →
Sign in / Sign up
ReadingResearchRadarInvestment framework

Using Uber’s setup as an example: bucket-1 is the primary dataset root (is_dataset_root: true), retaining the standard directory structure and versioned manifests; bucket-2 and bucket-3 are pure data buckets (is_dataset_root: false), storing only .lance data files. All locations are registered in the manifest’s base_paths, and each fragment points to its storage location via a base_id. During writes, data is distributed round-robin across data buckets; during reads, the system automatically expands all registered bases in parallel.

The core benefit of this design is that migrating a table with 10 million files distributed across 5 buckets requires changing only 5 base path strings, rather than rewriting 10 million file references. Pace summarizes this into four features—controlled flexibility (multiple locations with explicit structure), maximum portability (change bases, not files), composable portability (choose which bases to migrate together), and operational clarity (garbage collection and credential distribution only need to iterate base paths, not file lists).

Multi-base paths also unlock other scenarios: staging hot data onto high-performance parallel file systems like Lustre; distributing data across regions so training and inference read from local storage independently; switching only the affected base path rather than the entire dataset in case of a single-base failure; and shallow cloning and branching experiments—clones reference the source dataset via base paths, storing only the manifest and new files.

It is important to note that this is a design document for the LanceDB proprietary format, published by the vendor. The article describes Uber’s requirements and design principles but does not disclose specific throughput improvement figures or independent benchmark test results. The design assumes the number of bases remains small—Pace argues that whether through file imports, shallow clones, or multi-bucket distribution, the number of distinct bases stays low even if the table references millions of files. Whether this assumption holds in extreme scenarios, and how multi-base paths perform in actual production, still requires independent verification.

From June to October, the pass rate for Documentation MCP rose from 67% to 96%, and for Skill Packages from 75% to 97%, while the Baseline only increased from 75% to 82%. The Together team notes this is not a controlled comparison—different snapshots used different models and tasks, and some improvements stem from underlying model advancements. However, they attribute the extra 20+ percentage points gained by guided configurations over the baseline to document and skill fixes revealed by the evaluations. Their core conclusion is: anything developers can do manually should be programmatically achievable via CLI or SDK; documentation, CLI help text, and API error messages are the first places agents look when stuck; and improving agent experience often requires only small changes, but without running evaluations, you may never find the right modification point.