LesenForschungRadarAnlageframework
Anmelden / Registrieren
AlleTechnikProdukteInvestierenBörseSonstiges

Lesen

2026-11-0159 Beiträge

Aktualisieren

NVIDIA Cuts Rubin GPU Memory and NVLink Bandwidth Specs as First Production Hardware Ships

Thema · 英伟达Rubin规格下调Wesentlich
2026-11-01 11:00 GMT+8

As NVIDIA's next-generation AI computing platform, Vera Rubin, enters mass production, the company has quietly revised the final shipping specifications for its Rubin GPUs and associated systems. According to reports by technology outlet ServeTheHome, the new specs show reductions in memory and interconnect bandwidth compared to preliminary figures announced last year, though scale-out networking bandwidth has seen a slight increase. This shift occurs as hyperscalers begin deploying the first NVL72 systems, indicating that delivered hardware will perform differently from early marketing claims.

Specifically, the HBM4 memory bandwidth per Rubin GPU has dropped from an initial target of 22TB/s to 19.2TB/s, a reduction of approximately 11%. This stems from adjusting the memory stack operating frequency from an expected 10.75Gbps/pin to 9.375Gbps/pin. While this remains higher than competitor AMD's MI455X at 7.6Gbps/pin, the cut will directly impact training and inference workloads reliant on high memory bandwidth. Additionally, NVLink (Scale-Up) bandwidth for multi-GPU interconnects has fallen from 3.6TB/s to 3.0TB/s, a 17% decrease.

To compensate, the Rubin GPU's Scale-Out networking bandwidth has increased from 0.4TB/s to 0.45TB/s. For the rack-scale system Vera Rubin NVL72, total memory bandwidth drops from 1580TB/s to 1400TB/s, and total NVLink bandwidth falls from 260TB/s to 216TB/s, while scale-out networking rises from 28.8TB/s to 32.4TB/s. Notably, the HGX NVL8 system based on Rubin faces a memory capacity cut from 2.3TB to 2.0TB, forcing each GPU to use only 7 HBM4 stacks and resulting in a 26% loss in memory bandwidth.

NVIDIA had previously stated all values were "preliminary and subject to change," making such adjustments common before server hardware launches. However, significantly lowering key bandwidth metrics just before mass production suggests supply chain or technical implementation challenges, possibly related to memory controller speeds, stack yields, or a combination thereof. For customers planning to procure Rubin platforms, this necessitates a re-evaluation of performance expectations for memory-intensive AI models.

Quellen:ServeTheHome

Forschung

Ben Carlson: Market Structure Has Changed, Historical Valuations Are Not Directly Comparable

Wesentlich
2026-10-24 04:30 GMT+8

Ben Carlson is a portfolio manager at Ritholtz Wealth Management. His article begins with a small anecdote: a friend cleaning out a rental property found a business section from a 1960 newspaper showing the S&P 500's sector composition at that time—425 industrial companies, 25 railroad companies, and 50 utilities. Carlson then lists today's S&P 500 breakdown: 83 industrials, 76 financials, 75 information technology, and 60 health care, with other sectors also represented. He acknowledges that while the modern market is indeed more concentrated in top names, the types of companies represented by the index are far more diverse than in 1960. His conclusion is that historical valuations are difficult to compare directly to today's markets precisely because the objects being compared are not the same.

Carlson cites this week's column by Wall Street Journal writer Jason Zweig, which details specific structural changes in the market over recent decades. When individual investors bought or sold small lots of stock, commissions often reached 2% or higher, and institutional costs were no less than 1%. Mutual fund purchase loads typically ranged from 5% to 8%, plus annual management fees of 0.5% to 2%; Vanguard's first index fund carried a purchase load as high as 8.5%. Regarding minimum price units, stocks traded in increments of one-eighth of a dollar (12.5 cents) until 1997. Zweig gives an example: if a small-cap stock closed at a bid price of $1, and someone bought it the next day at an ask price of $1.125, a 12.5% return was generated out of thin air. Today, stocks trade in increments of one cent or less, rendering such bid-ask bounces immaterial. Carlson points out that while gross returns from the stock market may be lower in the future than in the past, investors have never been in a better position regarding net returns after costs.

Carlson argues that Regulation FD (Fair Disclosure), implemented by the U.S. Securities and Exchange Commission in 2000, may be the most important yet least discussed change to market structure this century. Before this regulation, companies could disclose material information to select individuals without revealing it to everyone simultaneously. Once regulators required that any significant corporate information be made available to all investors at once, it became much harder for hedge fund managers and other large active investors to win using proprietary information. He writes that this is why there haven't been nearly as many legendary investors beating the markets recently—the playing field has been leveled. In his new book, A Fabulous Debt, Robin Wigglesworth dedicates a chapter to explaining how Michael Milken essentially created the high-yield bond market in the 1970s. In college, Milken came across an old book by Walter Braddock Hickman containing an exhaustive study on corporate bond performance from 1900 to 1943. The study concluded that investors paid lower prices and demanded higher yields for lower-quality bonds; default rates on these bonds were higher; and higher yields more than compensated for the higher default rates. This seems obvious today but wasn't at the time. Wigglesworth quotes a future colleague of Milken's at Drexel: "Maybe 2,000 people read Hickman's book, and 1,999 ignored it." Milken was the exception; he devoured the jargon-filled text, dense footnotes, and bond yield charts. Milken went on to largely control the junk bond market in the 1980s, though his story did not end well. It is incredible to consider how little people knew about fixed-income markets back then.

Quellen:A Wealth of Common Sense

Forschung

SemiAnalysis: Trainium's Shift to Native PyTorch Training Is the Right Move; Compilation Time Is the Real Bottleneck for Researchers

Thema · TrainiumWesentlich
2026-10-24 03:06 GMT+8

For AI chips, inference and training are two entirely different engineering problems. Inference deals with forward propagation of a few fixed architectures and has a limited number of hot-path kernels; training requires a software stack that covers backward propagation for thousands of operators, multiple precision modes, and model structures that researchers may modify at any time. In a paid article published on October 23, 2026, SemiAnalysis explicitly stated that while Trainium has already proven itself in the inference field—Anthropic is running production inference and reinforcement learning on Trainium2 and Trainium3—it has historically struggled in research training because the requirements for API coverage and flexibility in training are far higher than in inference.

Amazon's response strategy involves a major software route correction. Previously, Trainium accessed the PyTorch ecosystem via PyTorch/XLA, relying on lazy tensor graph capture without a first-class eager mode, and did not support native PyTorch distributed APIs such as torch.distributed, DTensor, or SimpleFSDP. For researchers switching to Trainium, this meant relearning how to write training loops. The new TorchNeuron backend abandons this path and instead provides a native PyTorch experience: users simply replace .to("cuda") with .to("neuron"), theoretically allowing most ATen operators to run out of the box, while supporting both eager mode and torch.compile. SemiAnalysis believes this shift will significantly broaden the adoption range of Trainium3.

The SemiAnalysis engineering team conducted several weeks of practical testing on Trainium3 racks and Trainium2 hardware. Training the Qwen3-32B model under the TorchTitan framework yielded an average throughput of 11,099 tokens/second with 23.89% MFU in eager mode, and 14,156 tokens/second with 30.47% MFU in compiled mode, representing a 1.28x improvement in throughput. Training continued to function normally after they modified the model architecture. In the FlexAttention benchmark test, Trainium3 passed 89 out of 96 test cases, achieving a pass rate of 93%; all seven failed cases were compilation timeouts under large batch sizes and long sequences, rather than numerical errors. Regarding matrix multiplication performance, Trainium3's MFU was superior to Nvidia GB300 at medium scales (above 1 GFLOP), lagging only above 100 GFLOP, and regaining the lead above 1 TFLOP.

However, what truly hinders researcher adoption is not peak performance, but compilation time. SemiAnalysis's practical tests revealed that for the Qwen3-32B training run, the first step took 184.6 seconds in eager mode and 255.7 seconds in compiled mode, which is 30 times and 55 times the subsequent average step time, respectively. The compilation time for gpt-oss 20B was even higher, reaching 27 minutes and 17 seconds. The article traces the root cause of this issue to the historical design of the Trainium compiler stack: in inference scenarios, compilation is completed offline and cached once per image, so it does not affect production inference latency; but in training scenarios, researchers frequently modify model architectures, triggering recompilation with every change, which directly interrupts the iteration rhythm.

Quellen:newsletter.semianalysis.com

Forschung

Startup Memory Race Condition in vLLM Multi-Instance GPU Sharing: Cisco DevNet Engineers Explain Why Two Flags Are Indispensable

Thema · vLLM多实例显存竞态Wesentlich
2026-10-24 00:10 GMT+8

Time-slicing divides the GPU's compute time among multiple instances but does not partition video memory (VRAM). The Cisco DevNet platform engineering team configured AWS g5.2xlarge instances to run three time-sliced replicas per NVIDIA A10G (approximately 23 GiB VRAM) for their GPU learning labs. This means three learners share the same VRAM pool with no hardware isolation boundary between them. Since the A10G does not support MIG (Multi-Instance GPU), hardware-level partitioning is not an option.

The issue lies in vLLM's startup logic. Each vLLM instance determines its own KV cache size based on the free VRAM it sees at startup. When three instances start almost simultaneously, each believes it has exclusive access to most of the VRAM and requests a large chunk of cache independently—resulting in overselling. Typically, only the first instance starts successfully, while the other two report errors: "No available memory for the cache blocks." This phenomenon is documented in vLLM community issue #10643, but no complete solution for multi-instance coexistence was provided there.

The Cisco team's fix requires two flags working together because vLLM makes two separate memory decisions at startup. The first is --kv-cache-memory 3120851354, which fixes each instance's KV cache at 2.91 GiB, stopping dynamic measurement from free memory. Without measurement, there is no race—three instances no longer compete for the same unoccupied space.

However, fixing the KV cache alone is not enough. vLLM performs a second check at startup: it verifies whether current free VRAM covers its utilization target (default 90%). Even with a fixed KV cache, this check remains. For example, with a 92% target, once the first instance has occupied VRAM, the second instance fails upon startup because the free VRAM (17.96/22.06 GiB) is less than required (20.29 GiB).

The second flag, --gpu-memory-utilization 0.30, resolves this specific check failure. Lowering the utilization target to 30% means each instance only needs to confirm that free VRAM covers approximately 6.6 GiB to start. After the three instances' KV caches total about 8.7 GiB plus model weights, the remaining VRAM still satisfies the 30% target, allowing later-starting instances to pass the check and coexist with already resident peers.

Each flag eliminates an independent failure path: --kv-cache-memory eliminates allocation race conditions, and --gpu-memory-utilization 0.30 eliminates startup check failures. Without either one, the three-instance solution cannot run stably. After configuration, all three vLLM engines start simultaneously and successfully complete the full experiment workflow.

This mechanism's applicability extends beyond Cisco's learning labs. Any scenario involving concurrent startup of multiple vLLM instances on the same GPU—such as multi-tenant inference services, parallel evaluation in CI pipelines, or shared development environments—will encounter the same race condition. Specific values need to be recalculated based on total GPU VRAM, number of instances, and model size, but the logic of "fixing cache to eliminate race conditions and lowering utilization targets to pass startup checks" is universal.

Quellen:Cisco Blogs

Forschung

Bayesian Reasoning in AI Safety Debates: Eneasz Brodski Rebutts Vaden Masrani's Misreading of Toby Ord

Wesentlich
2026-11-01 04:54 GMT+8

The Askwho Casts AI podcast released an audio reading of Eneasz Brodski's article "I Have Lost My Ability to be Charitable to Vaden" on October 31, 2026. Brodski, author of the "Death Is Bad" blog, wrote this response after interviewing Vaden Masrani for The Bayesian Conspiracy podcast, concluding that his opponent argued in bad faith.

The article specifically rebuts Masrani's characterization of Toby Ord's work in "The Precipice." Brodski points out that Masrani claimed Ord arbitrarily started with a 50/50 probability and adjusted it based on "vibes," whereas Ord actually cited a 2016 survey of over 300 machine-learning researchers. This survey found that researchers assigned an average 50% chance by 2061 of AI accomplishing every task better than humans. Brodski argues that dismissing this as "stupid bullshit surveys" misrepresents Bayesian updating, a formal method for adjusting beliefs in response to evidence.

Brodski also criticizes Masrani for portraying Bayesian Rationalism as detached from reality. He emphasizes that rationality involves entangling beliefs with reality through experiments, tests, and arguments, not making up numbers. The essay further addresses Masrani's dismissal of Will MacAskill and the broader AI Safety field, which Brodski describes as thousands of professionals working on theoretical and technical problems, contrary to Masrani's depiction of them as students writing scary blog posts.

For readers interested in the methodology of AI safety debates, this resource provides a detailed, attributed counter-argument to specific claims about risk estimation and epistemic norms.

Quellen:askwhocastsai.substack.com

Forschung

Goldman Sachs CEO David Solomon Launches Small Business AI Institute in Livestream Townhall

Thema · 高盛小企业AI研究所Kurzfassung
2026-10-23 23:16 GMT+8

Goldman Sachs released a livestream resource on October 23, 2026, celebrating the launch of its new 'Small Business AI Institute'. The event featured a townhall discussion between Chairman and CEO David Solomon and small business owners, moderated by NBC News journalist Christine Romans.

According to the publisher's description, the Institute aims to partner with leading AI companies—specifically Anthropic, Google, Meta, and OpenAI—to provide small business owners across America with AI training, tools, and support. Additionally, Goldman Sachs plans to extend the reach of its existing '10,000 Small Businesses' program to another 10,000 businesses nationwide.

This resource is suitable for professional readers interested in AI industry implementation and digital transformation support for SMEs. While a full transcript is not currently available, this official launch event clarifies Goldman Sachs' strategic direction and key partnerships in empowering small businesses with AI.

Quellen:youtube.com

Forschung

Vector Database Company Qdrant: Asymmetric Quantization Reduces Single-Vector PDF Retrieval Memory to 64 Bytes per Page, Retaining 99.2% Quality

Thema · QdrantWesentlich
2026-10-23 08:00 GMT+8

When searching PDF page images, the Qdrant team previously typically recommended using late-interaction visual models (such as ColPali or ColQwen), retaining one vector for each image patch to avoid losing relationships between words in the layout during OCR extraction. However, late-interaction methods generate approximately one thousand vectors per page, and their MaxSim similarity metric cannot directly utilize HNSW indexing, making scaling difficult on datasets with billions of points. Out of curiosity, the team tested Cohere Embed 5 Pro—a model that represents an entire PDF page image with a single dense vector—on five public English corpora from ViDoRe V3 (computer science textbooks, financial reports, EU labor reports, US Air Force technical manuals, and US FDA pharmaceutical documents, comprising 12,969 page images and 1,489 English questions). The results showed that its nDCG@10 score surpassed that of several late-interaction models.

The team then explored how to turn this finding into a scalable solution for production environments. Embed 5 outputs default 2048-dimensional vectors, which consume 8 KB per page when stored as float32. Qdrant’s approach involves stacking three compression techniques: first, leveraging Matryoshka Representation Learning (MRL) supported by Embed 5 to retain only the first 512 dimensions of the vector rather than all 2048; second, applying 1-bit binary quantization to document vectors, keeping only the sign (whether each dimension is greater than zero); and crucially, asymmetric quantization—where query vectors maintain 8-bit precision while document vectors use only 1 bit. The intuition is that when dimensions are truncated to 512, the equal-weight sign count of 1-bit quantization becomes coarse, giving the same weight to a dimension just barely above zero and one with a large positive value; retaining magnitude information on the query side allows important dimensions to carry more weight during scoring. During retrieval, compressed vectors are used to find the top 20 candidates, which are then re-ranked using the complete float32 vectors stored on disk.

The final recipe consists of 512 dimensions, 1-bit documents, 8-bit queries, plus top-20 re-ranking, occupying only 64 bytes in RAM per page—1/128th of the 8 KB required for a complete float32 vector, while retaining 99.2% of the ranking quality. The team performed some napkin math: for 10 million PDF pages, ColPali’s float32 vectors would require approximately 4.8 TB, complete Embed 5 vectors would need 76 GB, and this recipe requires only 610 MB of RAM. However, the team also noted boundaries: Embed 5’s training data is not public, and they wrote,

"We might have accidentally tested on the training set, but if not, wow!"

. Additionally, 610 MB is the starting RAM usage; the complete vectors read from disk during re-ranking will gradually fill the OS page cache. The recipe still retains approximately 95 GB of complete and truncated float vectors on disk.

The team also tested the gain provided by Cohere Rerank 5 for OCR text retrieval pipelines: after embedding ViDoRe V3’s OCR text search using Embed 5 Pro, they re-ranked the top 20 pages using Rerank 5 Pro, raising the average nDCG@10 across the five corpora from 66.8 to 73.2; when the text search used the 64-byte recipe, it similarly rose from 66.2 to 71.7.

Quellen:qdrant.tech

Forschung

DeepSeek Open-Sources Super-agent Architecture for Autonomous Cross-Session Tasks

Wesentlich
2026-10-31 21:49 GMT+8

Chinese AI company DeepSeek has open-sourced the underlying architecture prototype for its 'Super-agent' personal agent system on GitHub as of October 31, 2026. The solution aims to address common pain points in traditional Coding Agents, such as fragmented context during multi-task collaboration, the need for users to manually switch sessions, and the inability to run continuously over long periods.

The Super-agent employs a dual-layer architecture: an upper-layer Router handles message reception, task allocation, and global context maintenance, while lower-layer Workers are specific Agent sessions executing tasks. This design allows a single development task to seamlessly transition from IM group discussions to GitHub PR reviews without requiring the user to re-explain context. Additionally, the system introduces 'Speculative Execution,' where agents proactively attempt background processing (such as reproducing bugs or organizing meeting materials) upon detecting potential needs, presenting results only if viable solutions are found.

To achieve 24-hour online availability and self-recovery, DeepSeek separated computation, state, and execution environments, introducing an independent Watchdog module to monitor other agents. In a simulated collaborative environment containing approximately 300 messages and 20 work requests, DeepSeek's internal tests showed that Super-agent had a median task completion time of 12.4 minutes and a 95th percentile of 15.7 minutes, outperforming the traditional single-session approach's 29.5-minute 95th percentile.

It is important to note that these performance metrics are based on DeepSeek's internal simulation experiments and have not yet been independently verified. Furthermore, efficiency gains come with increased token costs; disabling the event direct-pass mechanism resulted in a roughly 30% increase in Router decision counts and a 37% increase in read-cache tokens.

Quellen:量子位

Forschung

Joachim Klement: A Professional Civil Service Can Delay the Economic Damage of Populism by Approximately Four Years

Thema · 民粹主义与经济衰退缓冲Wesentlich
2026-10-23 14:00 GMT+8

In his investment newsletter, Joachim Klement offers a practical judgment for political risk assessment: following the election of a populist leader, an independent and professional civil service can delay economic downturns by approximately four years. This conclusion stems from a 2026 cross-national study by Massimo Morelli and colleagues, which examined the differential impact of populist leaders on economies in countries with varying bureaucratic quality. Klement also references a 2023 study indicating that populist leaders typically cost the economy about 10% of GDP over 15 years—a heavy burden particularly for Western European nations already struggling with sluggish growth due to demographic pressures.

Morelli et al. divided the sample countries into high and low groups based on their bureaucracy scores. This score does not measure the size of the civil service but is derived from three indicators in the V-Dem Democracy Database: rigorous and impartial administration, merit-based recruitment systems, and remuneration protections for civil servants. In other words, it measures the independence and professionalism of the civil service rather than its sheer scale. This distinction is crucial for investors, as a bloated but non-independent bureaucracy does not offer equivalent protection.

The results show that countries with a professional and independent civil service can delay economic decline by approximately four years after the election of a populist leader. Klement stresses that this protection is a delay, not a prevention—economic downturns eventually set in even in countries with the highest bureaucratic quality. However, the key lies in the time window: if the populist leader steps down after a single term, the four-year buffer may be sufficient to avoid the worst economic consequences. Yet, once a populist wins a second term, the situation becomes dangerous.

For investors, this mechanism provides a specific dimension for political risk assessment: when a populist leader takes office, the quality of the country's civil service determines whether economic damage manifests immediately or if there is an observation window of approximately four years. Klement’s own stance is noteworthy—he admits he is not a fan of civil servants but believes they may serve as the last line of defense for the economy in the current political climate. The boundaries of this judgment are as follows: the study is based on historical cross-national data, the four-year figure is an estimate from the international sample, and actual timelines will vary depending on a specific country’s institutional resilience, the radicalism of the populist leader’s policies, and external shocks.

Quellen:klementoninvesting.substack.com

Forschung

Goldman Sachs Podcast: Daan Struyven Explains Why Oil Prices Remain High Despite Supply Recovery

Wesentlich
2026-10-23 12:00 GMT+8

The official Goldman Sachs podcast 'Goldman Sachs Exchanges' published an episode titled 'Why Oil Prices Are Surging Despite Recovering Supply' on October 23, 2026. The episode features Daan Struyven, co-head of Global Commodities Research and head of Oil Research at Goldman Sachs, recorded on October 22, 2026.

According to the episode description, Struyven notes that although Middle East crude flows have rebounded, two main factors keep prices elevated: physical buyers bidding up available crude amid growing downside risks to supply (precautionary physical buying), and bond and equity prices being unusually sensitive to oil prices, prompting institutional investors to buy crude to hedge their portfolios (portfolio hedging).

The program also discusses challenges in the refined products market. Persian Gulf exports of refined products like diesel and gasoline are still only at around 40% of their 2025 average levels due to damage to refineries and underinvestment in manufacturing. Struyven also offers views on metal markets, expecting copper prices to increase another 10% by the end of 2027, and suggesting gold may climb amid central bank demand.

This episode is suitable for investment readers interested in macro commodity dynamics, particularly those seeking to understand the mechanisms behind the current divergence between oil prices and supply recovery.

Quellen:youtube.com

Forschung

PC Watch Reporter Norihiko Wakasugi Discovers AI Agents Act as Dispatchers, Not Executors, in Automated Editing Workflows

Thema · AI智能体调度员角色Wesentlich
2026-10-23 05:18 GMT+8

In October 2026, PC Watch reporter Norihiko Wakasugi published a detailed account of his practical experience: he borrowed an ASUS ProArt P16 laptop equipped with NVIDIA's RTX Spark and spent three days building a system capable of automatically generating YouTube Short clips after live streams ended. He did not write a single line of code; all development was completed conversationally using Claude Code. The runtime environment utilized the Hermes agent paired with the local large language model Qwen3.8-27B.

The system's workflow operates as follows: it retrieves live stream recordings from the company NAS, extracts audio using ffmpeg, performs full-text transcription via whisper.cpp, and then segments the content by topic. Each segment of 45 to 70 seconds is scored based on criteria including the growth rate of viewer comments, laughter detection, and facial smile recognition (implemented using sherpa-onnx and MediaPipe). Non-overlapping high-scoring segments are selected as candidates. Next, the local LLM reads the text surrounding these candidates to determine if they are independently understandable and contain punchlines, adjusting start/end points and titles accordingly. Finally, three to five clips scoring above seven points are selected to generate vertical short videos.

What is truly noteworthy is not the system itself, but the author's assessment of the agent's role during its construction. He found that executing repetitive tasks in standardized workflows—such as transcription, segmentation, scoring, and clipping—via Python scripts written by Claude is far more efficient than having Hermes invoke large model inference each time. Hermes effectively serves as a dispatcher passing instructions between the user and these scripts, summarizing results. The author explicitly stated that if the sole goal were functional implementation, this system could operate entirely without Hermes; the agent was integrated into the workflow specifically to test its capability boundaries.

This finding creates tension with the prevailing narrative around agents currently popular in the industry. While the sector widely promotes allowing AI to autonomously complete complex, multi-step tasks, Wakasugi’s actual experience suggests that when a workflow can be broken down into deterministic steps, the optimal use of large model agents is to step back: acting not as executors, but solely as dispatchers. He candidly admitted in his article that the combination of Hermes and the local Qwen3.8-27B often felt sluggish during conversational interactions, which was the direct reason he chose to use cloud-based Claude Code for development while retaining only the local agent at the runtime end.

Of course, this architectural judgment rests on clear prerequisites. The author’s workflow is highly standardized—live stream transcription, comment alignment, clip scoring, and vertical cropping—where input and output formats are fixed, making them suitable for encapsulation in code. If a task requires significant open-ended judgment or if the process changes frequently requiring immediate adjustment, the dispatcher mode may prove insufficient. The author also noted that approximately 70% of the AI-generated clips could be used directly, while the remaining 30% required manual correction. These corrections were executed by sending instructions via Telegram, specifying crop ranges, identifying speakers, adjusting subtitle timing, adding missing subtitles, and designating camera zooms.

Quellen:PC Watch

Forschung

Bardia Shahali, VP of Sales at Granola: Why AI Infrastructure and Application Layers Require Different Sales Organizations

Thema · Granola销售策略Wesentlich
2026-10-22 08:00 GMT+8

The AI industry comprises two markets: infrastructure and applications. Bardia Shahali, Vice President of Sales at Granola, discussed the differences in building sales organizations for these two markets on Tomasz Tunguz's Office Hours program. He has built teams at both levels—first constructing the sales team at Fireworks AI, where he grew revenue from roughly $3 million to $400 million, and now leading application-layer sales at Granola.

At the infrastructure layer, deals can be very large and close very quickly. Shahali noted that at Fireworks, customers are AI-native companies growing at extraordinary rates, with individual deals reaching tens of millions of dollars; an eight-figure contract can be closed within two weeks. However, the cost is fierce competition—every inference service provider chases the same clients, with higher stakes and denser competitors. In this environment, the infrastructure layer requires a different team configuration: FDEs (forward-deployed engineers) and SEs (sales engineers) must work alongside AEs (account executives) because they are delivering a highly complex product.

The logic differs at the application layer. Granola builds software products for end users, and its sales structure resembles previous-generation SaaS: AEs collaborate with CSMs (customer success managers) to deliver solutions, but productivity, adoption, and expansion speeds are typically higher and faster.

This dynamic environment imposes new requirements on organizational management. Shahali mentioned that Fireworks previously set sales incentive plans annually but has now switched to quarterly adjustments because product iteration is too rapid—model capabilities reach a new peak approximately every 45 days, causing the competitive landscape to shift quickly. Annual planning cannot keep pace with this rhythm. He also pointed out a common risk: if a sales team is too small, it lacks the capacity for outbound outreach and market expansion.

It should be noted that these observations stem from Shahali’s firsthand experience at two companies and are limited in sample size. The revenue figures for Fireworks AI are self-reported and have not been independently audited; whether Granola’s application-layer sales model represents the broader AI application sector remains to be verified by more practice. Nevertheless, the differences between the infrastructure and application layers in terms of deal size, cycle length, competitive density, and team structure provide a useful analytical framework for understanding how AI companies operate commercially: the same sales methodology cannot be applied to both layers.

Quellen:Tomasz Tunguz

Forschung
Weiter →
Anmelden / Registrieren
LesenForschungRadarAnlageframework

Carlson concludes that the market has indeed changed significantly—there are more products and services than ever before, costs are lower, and automated investing makes it easier to maintain discipline. However, human nature remains the only constant across all market cycles. In a world with unprecedented amounts of information, market knowledge, data, and analysis, one must fight against overconfidence about the future. Knowing more about what happened in the past and having more data about the present does not make predicting the future any easier.

SemiAnalysis proposes an important judgment framework regarding this issue: for ML research ablation experiments, the most critical factor is not the fastest stack, but decent performance combined with broad support for model architecture modifications, enabling researchers to iterate without fighting the stack itself. This means that MFU percentage is not the sole indicator; the breadth of API coverage, numerical correctness, and iteration efficiency jointly determine the practicality of a training stack. Trainium performs well in API coverage and numerical accuracy, but compilation time—a seemingly minor engineering issue—actually constitutes the core friction preventing researchers from migrating from CUDA.

AWS has already marked solutions for this problem in its roadmap. The upcoming version of TorchNeuron introduces a new compiler backend featuring extensive precompiled NKI kernels and a re-architected fast-compilation stack, aiming to reduce the time-to-first-step by up to two orders of magnitude while maintaining minimal or zero performance regression. SemiAnalysis believes that if this promise is fulfilled, compilation time will cease to be a topic of discussion. Additionally, Trainium is rapidly investing in the vLLM/SGLang ecosystem, providing hardware access and early access to SGLang based on native PyTorch, a strategy consistent with Google's approach of integrating TPUs into the vLLM/SGLang ecosystem via TorchTPU.

It should be noted that SemiAnalysis's tests are first-party results and have not been independently reproduced; the article also explicitly acknowledges that Trainium is far from matching the maturity of the CUDA ecosystem. TokenSwitch, a first-class API in PyTorch for MoE dispatch/combine, has not yet been implemented on Trainium and currently relies on a custom NeuronEPTokenDispatcher class. The promised two-order-of-magnitude improvement in compilation time is an AWS commitment rather than a delivered result. Nevertheless, SemiAnalysis's core argument—that native PyTorch interfaces solve the API coverage problem, while compilation efficiency is the next key variable determining adoption speed in the training market—provides a more operational dimension for evaluating AI accelerator training readiness than peak FLOPS alone.

From a performance perspective, generating four preview videos took approximately 30 minutes in total. This broke down to about 6 minutes for transcription and topic segmentation, 6 to 7 minutes for clip selection, and roughly 3 minutes per preview video. After corrections, final export for each clip took about 5 minutes. The author did not compare this against previous outsourced editing times, as this was his first time using a PC for video editing, but he felt that completing previews in under 10 minutes per clip was quite fast.

At the end of his article, Wakasugi clearly defined the system's positioning: it is not a demonstration of autonomous agent work, but rather a hybrid architecture featuring human-in-the-loop oversight, agents acting as dispatchers, code handling execution, and AI performing judgment. This conclusion stems from the friction encountered during actual construction and operation—for instance, bloated chat logs forced the system to re-read approximately 80,000 tokens for each reply, taking about 3 minutes, prompting him to have Claude write a plugin to reset sessions. These details indicate that the bottleneck in current agent workflows often lies not in model capabilities themselves, but in engineering-level context management and execution efficiency.