The Verge Senior Reporter Victoria Song: Apple Siri Recap's Privacy Architecture Makes AI Harder to Use
After a two-week hands-on test of the Apple Watch's Siri Recap feature, The Verge reporter Victoria Song found that Apple's architectural choices to protect privacy—specifically, not training on user voice data, not attributing speaker identity, and not saving raw audio—made it harder for the AI to distinguish between conversation and media playback, or to determine what was personally important to the user. This ultimately led to frequent errors in summaries and a lack of practical value. She argues that better privacy often results in less useful AI products.
After testing Siri Recap on the Apple Watch Ultra 4 for two weeks, The Verge senior reporter Victoria Song reached a conclusion: better privacy often means less practical AI products. Apple implemented several privacy-preserving designs for this ambient summarization tool—including not training on user voice, not saving raw audio, and omitting speaker attribution in summaries—but Song found that these choices directly impacted the AI's utility.
The most specific failures stemmed from the decision not to train on voice. Because Siri had not learned Song's vocal characteristics, it struggled to discern when she was actually speaking to someone versus when her phone was simply playing content aloud. When Song played a TikTok video aloud on her phone featuring a chronically ill creator recounting their trip to the emergency room due to sepsis, Siri Recap summarized the video as if Song herself, or someone she knew, had been sent to the ER. Another TikTok discussing an AITA (Am I The Asshole) Reddit thread about a woman named Dana attempting to seduce her best friend's husband was summarized as if Song herself were returning home with Dana's glitter at 2 or 3 AM. Song noted that Siri Recap automatically turns off when she wears AirPods, meaning she has to adjust her behavior to accommodate the AI's recognition flaws.
The second mechanism is the absence of speaker attribution in the summaries. Apple deliberately omits who said what and when in the recaps to prevent these records from being used as evidence in arguments, court cases, or for blackmail. However, stripping away attribution often causes the summaries to present other people's stories as the user's own experiences. Song also observed that Siri Recap heavily favors restating facts and statistics while ignoring information the user might find most valuable. In one interview, she found the guest's personal anecdotes and strong opinions about RFK Jr. to be the most interesting parts, but the summary only retained more concrete factual details.