The Glass Cage
by Nicholas Carr, Nicholas G. Carr
Rating: 3.0 / 5 · Read · Finished 2025-02-01
It’s a book that talks about the dark side of automation. Things like aircraft autopilot, electronic medical records, GPS, high frequency trading, CAD software. It doesn’t say all automation is bad, but points out how we sometimes think irrationally about it. The book’s chapters are hit-and-miss, so here’s all the good stuff:
My takeaways
- The only way to get good at something is by actually doing it.
- Any time you have the chance to do something yourself, you’re going to enrich yourself a little bit with skills and understanding.
Ways that automation can go wrong
- Without work, we become bored, anxious and unhappy
- We lose a sense of agency
- We lose or fail to develop skills, then can’t handle emergencies.
- Automation trades “maneuverability” for “stability”. It makes it hard to adapt to new situations.
- Automation complacency. We assume because it’s automated, it’s right. We catch fewer mistakes.
- If things are done for us, we don’t learn how they work. E.g. computerized board games enforce the rules for you, so you don’t learn them.
- When things go wrong you have to simultaneously take manual control and deal with automation. E.g. fiddle with smartphone navigation while finding a way through the construction zone
- Automation may not replicate subtle but important functionality it replaces. E.g. physical medical records have different handwriting from different doctors, making it more navigable at a glance.
- Dropdowns on forms limit what situations can be represented.
- Automation reduces common errors but increases rare ones.
- The speed of automation magnifies small errors.
- Humans get bored and inattentive when monitoring things that mostly don’t go wrong.
Interesting ideas:
- “miswating” is wanting things that won’t make us happy. Like wanting leisure time, then being bored and anxious when we have it.
- The paradox of automation is that it makes accidents less common, but more severe.
- Adaptive automation. System takes over routine tasks when operator has to focus on something critical. Then assigns some routine tasks to operator during quiet periods to keep them engaged. Goal is to keep operator at peak flow.
- Games are the one type of software that are not supposed to be frictionless, easy, or automated. The struggle is the point. Other software can learn from games. Good games maximize flow. Bad games maximize addiction.
- Wearables promise to let you stay connected to your surroundings but in practice the opposite happens.
- Technology so often ends up being like a genie ironically fulfilling wishes. It gives us what we ask for but not what we meant. Social media makes us lonely.
- Recommendation algorithms reinforce our old desires and never challenge us to try something new
Great quotes:
- “The trouble with automation is that it often gives us what we don’t need and the cost of what we do.”
- “Honing our skills, enlarging our understanding, and achieving personal satisfaction and fulfillment are all of a piece… They all require… getting your hands dirty.”
- “Utopianism… is a form of mis wanting”
- “Does our essence still lie in what we know, or are we now content to be defined by what we want?”
Specific examples are out of date. Interesting to read 10 years on. He argues that code can’t replicate human driver abilities. Turns out deep learning can. They talk about privately owned self driving cars. Tesla did that, but Waymo has the more successful business model. He gives an example of natural language processing that is hard for computers, but it was solved by GPT-2. My takeaway is to spend less time trying to predict the future of technology.
My biggest gripe with the book is that it sometimes degenerates into a tangled mess of stories and history, all loosely related to automation, but oftentimes boring and incoherent. I picked up this book because I wanted to hear arguments against automation. I found myself skipping around. I skipped the entire chapter on ethical dilemmas of self driving cars.