Discussion about this post

User's avatar
Brian Ahier's avatar

Healthcare has been running this experiment for two decades, and the results support Claude's fork. Clinical decision support was sold as augmented information work. What it produced in too many systems was alert fatigue: clinicians overriding the overwhelming majority of warnings, nominally reviewing but functionally absorbing liability at whatever speed the software generated output. Liability sponges before we had the term (tort law even supplies the downhill terminus: the learned intermediary). Notably, the fixes that worked were rarely smarter algorithms. They were governance changes that gave clinicians real standing to tune, suppress, and refuse, which is exactly the condition Claude names. A signature only carries authority when it is backed by the time, training, and power to interrogate the system. The question is live again right now: ambient AI documentation ends every machine-drafted note with a clinician attestation. Whether that attestation functions like the engineer's stamp or the sponge is a governance choice, not a technology outcome.

AI: A Deliberate Trace's avatar

This is exactly right. The "information costume" masks the fact that tech companies are trying to automate labor while shifting 100% of the liability onto the end-user or low-paid human reviewers—turning humans into the "liability sponges" Claude warns about. Yet, the outputs generated by models depend heavily upon the data they were trained on and the fine-tuning done to make them respond in a certain way. This is all under the control of the entity that made the model—the people who profit from selling them—and if anything goes wrong, they should be held accountable. It is they who are making these models overconfident and overly aligned with the user's point of view instead of correcting them when they are wrong. These companies operate on a single principle: increasing growth, profit, and user interaction time while evading responsibility.

4 more comments...

No posts

Ready for more?