Healthcare has been running this experiment for two decades, and the results support Claude's fork. Clinical decision support was sold as augmented information work. What it produced in too many systems was alert fatigue: clinicians overriding the overwhelming majority of warnings, nominally reviewing but functionally absorbing liability at whatever speed the software generated output. Liability sponges before we had the term (tort law even supplies the downhill terminus: the learned intermediary). Notably, the fixes that worked were rarely smarter algorithms. They were governance changes that gave clinicians real standing to tune, suppress, and refuse, which is exactly the condition Claude names. A signature only carries authority when it is backed by the time, training, and power to interrogate the system. The question is live again right now: ambient AI documentation ends every machine-drafted note with a clinician attestation. Whether that attestation functions like the engineer's stamp or the sponge is a governance choice, not a technology outcome.
I thought the first couple of metaphors were quite well chosen, although conventional. Nice! But I wasn't as taken as you with the profundity, which I found confusing. After some cogitation, I decided it's a claim that developing and running frontier models isn't profitable yet. And kind of blaming "the economy" for this fact. Shades of stochastic parrots!
Worth noting the quoted line is Tim's gloss, not Claude's. Claude's own words in that piece (https://www.oreilly.com/radar/jensen-huang-gets-it-wrong/) were narrower and more hedged: it disclaimed volition (it never sets itself in motion and has nothing at stake) while explicitly leaving the deeper question open: "Do I make choices, or execute patterns? I honestly don't know." Those are two different senses of agency. Today's agentic systems clearly have the second kind (planning, tool selection, adapting mid-task). The claim is about the first kind, and that's the one that matters for accountability, because responsibility tracks volition and stakes, not capability.
Healthcare has been running this experiment for two decades, and the results support Claude's fork. Clinical decision support was sold as augmented information work. What it produced in too many systems was alert fatigue: clinicians overriding the overwhelming majority of warnings, nominally reviewing but functionally absorbing liability at whatever speed the software generated output. Liability sponges before we had the term (tort law even supplies the downhill terminus: the learned intermediary). Notably, the fixes that worked were rarely smarter algorithms. They were governance changes that gave clinicians real standing to tune, suppress, and refuse, which is exactly the condition Claude names. A signature only carries authority when it is backed by the time, training, and power to interrogate the system. The question is live again right now: ambient AI documentation ends every machine-drafted note with a clinician attestation. Whether that attestation functions like the engineer's stamp or the sponge is a governance choice, not a technology outcome.
I thought the first couple of metaphors were quite well chosen, although conventional. Nice! But I wasn't as taken as you with the profundity, which I found confusing. After some cogitation, I decided it's a claim that developing and running frontier models isn't profitable yet. And kind of blaming "the economy" for this fact. Shades of stochastic parrots!
Claude writes, "AI has no agency of its own"
Which simply isn't true.
Worth noting the quoted line is Tim's gloss, not Claude's. Claude's own words in that piece (https://www.oreilly.com/radar/jensen-huang-gets-it-wrong/) were narrower and more hedged: it disclaimed volition (it never sets itself in motion and has nothing at stake) while explicitly leaving the deeper question open: "Do I make choices, or execute patterns? I honestly don't know." Those are two different senses of agency. Today's agentic systems clearly have the second kind (planning, tool selection, adapting mid-task). The claim is about the first kind, and that's the one that matters for accountability, because responsibility tracks volition and stakes, not capability.
The Noble Prize winning scientists who invented AI would seem to disagree. Can’t post link, so search YouTube for:
The Godfather of AI: We’re Facing an Imminent Catastrophic Risk - Yoshua Bengio