Has technology become an elaborate band-aid for institutional and social failures? Are we surrendering our agency to tech?
From AI-generated podcasts replacing basic conversations to chatbots deployed to fix complex government processes, we are increasingly treating software as a substitute for problem solving. As we rely on algorithms to navigate basic daily tasks, we risk dulling our critical thinking, excusing systemic inefficiency, and yielding our agency to tech.
Let's dive in.
Jared Giles
Founder, SquareOne
[email protected]
The Tech-First Trap
I recently attended a panel on how AI will impact education. The talk spanned topics including how to combat bias, how AI is changing the classroom, and the merits of a moratorium in schools. It was a fruitful conversation, but one comment stood out to me. When discussing how technology can help, a panelist remarked that people can use AI to “make government forms easier to understand.”
I immediately thought: Or we could make the form less complicated.
Yes, artificial intelligence can help translate forms, but if someone’s experience with a government form or process is only improved because a chatbot walked them through it, shouldn’t we question why it was so complicated to begin with? Why is a form’s design so broken that we need cutting-edge software just to parse it?
In a similar story, OpenAI CEO Sam Altman tweeted that parents can use ChatGPT to make a podcast about their kids' upcoming games and birthdays to create a bonding moment between parent and child. But as animator Alex Hirsch succinctly replied: "What if you just talked to your children?"
We are using artificial intelligence to build elaborate workarounds for human and institutional failures, rather than fixing the failures themselves.
My concern is that our reliance on AI is encroaching upon our ability to think critically and manage every day challenges, and that we will eventually become dependent, with AI replacing any agency we have as humans.
We already see this anxiety mounting in our education system. If tools like ChatGPT and Claude remove the friction from learning to read, write, comprehend, and problem solve, what happens to the foundational cognitive stamina of the next generation? As a study conducted at Wharton and Penn found, when young people bypass the struggle of wrestling with complex ideas, they may lose the basic muscle memory required to interrogate flawed systems, troubleshoot hardship, and think independently.
Played out over the next decade, this shifts to a structural problem. If we fail to improve cognition in our talent pipeline today, we risk building a future where artificial intelligence, rather than human ingenuity, determines which policies are drafted, which inventions go to market, and what programs are funded.
Artificial intelligence is an extraordinary multiplier for human capacity. But when treated as a substitute for effort, it can perpetuate poor systems rather than cure them. If we aren't intentional about how we deploy these tools, we risk falling into a trap of our own making: becoming too reliant, losing our capacity to analyze, and ultimately surrendering our agency to technology.
National Spotlight: Critical AI Literacy & Statewide Frameworks
How do we build education systems that prioritize agency over shortcuts? State education departments are beginning to pioneer frameworks that center human reasoning over a reliance on software.
The Problem: Rushing generative AI into K–12 classrooms without guardrails risks eroding students’s problem-solving ability, writing discipline, and cognitive independence.
The Intervention: State education agencies (including recent statewide frameworks in South Carolina and Alabama) are enacting AI literacy mandates centered on agency. Rather than using AI to replace student thinking, the curriculum requires students to audit AI logic, verify outputs, debate algorithmic bias and AI ethics, and explicitly justify when not to use automated tools.
The Takeaway: True digital readiness sharpens a young person’s capacity to question, reason, and decide. Preparing the next-generation requires protecting critical thinking and judgment as non-negotiable core competencies.
ICYMI
[LISTEN] The Ezra Klein Show: The A.I.s Are Already Out of Control with former OpenAI Board member Helen Toner.
[READ] MIT Technology Review: A recent article by Michelle Kim outlining how AI generates its own bias when used in hiring.
[READ] Pew Research Center: Study highlighting “How Teens Use and View AI” and the impact they think it will have on society.