The promise of artificial intelligence often comes down to how much it can do for us: answer questions, solve problems, make decisions, and complete work faster.
But as those systems become increasingly capable, Foster School of Business Assistant Professor Max Kleiman-Weiner is focused on what happens to our own capabilities when AI does more of the work.
It’s a question he approaches through the cognitive science of caregiving.
“In the typical pro-social mode of interaction, we think, ‘If I’m being nice to you, I’ll help you achieve your goal,’” Kleiman-Weiner says. “But what’s distinct about caregiving is that sometimes the best way to help is actually to hold back.”
A good parent, mentor, or manager doesn’t solve every problem. If someone has the capacity to grow and learn, helping may mean challenging them or letting them struggle, Kleiman-Weiner says.
“That’s how we learn,” he says. “We learn from trying and oftentimes failing.”
Kleiman-Weiner is bringing that insight to a new research collaboration with Natasha Jaques, assistant professor at the UW Paul G. Allen School of Computer Science & Engineering, through the Toyota Research Institute’s Human-AI Kaizen Initiative. Their project, Kaizen Thought Partners: Pluralistic AI that Empowers and Adapts to People, was one of four selected from 194 proposals submitted by researchers at more than 70 universities.
The three-year project will explore what it would take for AI to support human growth and autonomy, rather than simply becoming ever better at doing things for us.
As AI gets better at doing things for us, Max Kleiman-Weiner is asking what it will take to keep humans capable, curious and in control.
Beyond ‘human in the loop’
The Toyota Research Institute (TRI) call offered Kleiman-Weiner a chance to apply his work on caregiving and human cooperation to increasingly powerful AI systems — and to question the familiar prescription to “keep humans in the loop.”
That idea, he says, can become too simplistic.
AI capabilities already form what researchers sometimes call a jagged frontier: Systems can be remarkably capable at some tasks and much less so at others. Today, that makes it natural to assign humans the parts machines do poorly.
But those boundaries keep moving.
“If it just comes down to, ‘Well, today they’re not as good in this dimension, so humans should be in the loop there,’” Kleiman-Weiner says, “we can all, I think, foresee a time where that might not be the case.”
So he is interested in a harder question: If AI becomes better than us at more of the work, are there still reasons we should continue doing some of it ourselves?
The risk he’s focused on is disempowerment.
Kleiman-Weiner uses overparenting as an example. A parent who solves every challenge for a child may make life easier in the moment, but the child loses opportunities to learn how to handle those challenges independently.
He sees a similar risk with AI. The concern is not only that people may relinquish control to increasingly capable systems, but that over time they may become less capable of doing those things themselves.
His earlier work had examined caregiving across relationships such as parents and children, mentors and mentees, and managers and employees. Much of it centered on the same underlying question: When does helping someone support their growth, and when does it begin to undermine their autonomy?
That made the TRI project a natural extension of work he was already pursuing.
“How might we avoid disempowerment?” he says. “How might we avoid situations where humans not only don’t have control, but don’t have the capacity for control?”
Preparing people for independence
One aim of the project is to make the distinction between assistance and empowerment something researchers can actually measure.
For this project, Kleiman-Weiner models caregiving as a two-stage process.
In the first stage, a caregiver and learner work together. The caregiver — in this case an AI system —can teach, demonstrate, change the environment, or provide direct help.
In the second stage, the learner operates without the caregiver’s assistance.
“The success of their behavior will be scored by how well that learner now does in the second stage where the caregiver is removed,” Kleiman-Weiner says.
In other words, an AI system that makes someone highly productive while it is present may not necessarily be empowering them. The more revealing test could be what that person can do afterward.
That raises another challenge for the model: What future is the caregiver preparing the learner for?
Kleiman-Weiner says the framework has to account not only for what that future might look like, but also for how uncertain it is. A century ago, he notes, one generation could have had a reasonably good sense of the world the next would inhabit. Today, “it feels like every six months we’re living in a new world.”
“When you ask, ‘Well, what’s the right preparation?’ It’s going to look completely different because we have so much uncertainty and because the pace of change is so fast,” he says.
Who is adapting to whom?
Even when AI leaves people in control, it can still influence how they think and work. That raises another question for the project: Who is adapting to whom?
For Max Kleiman-Weiner, the future of AI isn’t only about what technology can do for us. It’s also about ensuring people retain the skills and autonomy to act for themselves.
“We’re all using the same AI systems,” Kleiman-Weiner says. “They all write in a really similar way. They all make recommendations in a really similar way.”
As people absorb those outputs, reproduce them, and put AI-influenced content back into the world, the same patterns can become reinforced. Kleiman-Weiner describes one possible result as “epistemic lock-in”: Instead of continually generating and encountering new ideas, people may increasingly reinforce the same ones.
The project examines that influence at the level of individual users, too. How much does a person have to change the way they would naturally think or work simply to collaborate with an AI system?
One way Kleiman-Weiner and Jaques hope to counter those effects is through better personalization. They are developing ways to measure a human user’s “adaptation cost,” or how much the person has to alter what they otherwise would have done in order to work with the AI. The goal is to design systems that adapt more readily to people’s preferences and ways of working, rather than requiring people to conform to the AI.
Good human collaboration offers a useful model.
“When it’s working well, humans are really thinking carefully about, you know, what’s the best way for me to present this information for you to understand,” he says.
The same kind of responsiveness matters when working with AI.
“The idea that I just have to ask the right question reduces the interaction to a single moment,” he says.
His best experiences have been iterative: asking follow-up questions, drawing out what the system knows, and revising his own understanding along the way.
“We’re kind of working together on a shared mental model,” he says. “That’s when I’ve had the most productive interactions.”
What humans bring to AI
That intersection between human behavior and technological change is one reason Foster is an exciting place for his work, Kleiman-Weiner says. The business school allows him to operate “at the interface,” studying basic questions about how people think while also considering how organizations should prepare for rapid changes in technology and work.
Those questions increasingly shape his teaching.
Kleiman-Weiner teaches Consumer Insights, a market research course, and has also taught executive education on AI and digital transformation. When he first came to Foster, relatively few students were using AI, and much of the classroom conversation focused on understanding the technology itself.
Now, he says, the more interesting possibilities emerge when students combine increasingly capable AI with knowledge, experience, and perspective of their own.
“The power in these systems is when you bring your own set of expertise and story and background,” he says. “That’s when really interesting things can happen.”
One story Kleiman-Weiner shares with students comes from a “freestyle chess” tournament, where competitors could use any combination of human players and computers. Three of the final four teams included chess grandmasters. The fourth was a pair of amateur players using home computers.
The amateurs won.
Kleiman-Weiner’s takeaway is that the winners weren’t necessarily the strongest chess players or algorithm developers. They were especially good at combining what they knew with what the computers could do.
He sees a similar pattern among his students. The ones thriving aren’t necessarily those with the deepest technical background or the most experience, but those curious about how to combine what they know with what AI can do.
“Maybe that is the adaptive story for today’s environment,” he says. “How do you bring the knowledge that you have?”
Max Kleiman-Weiner is an Assistant Professor of Marketing and International Business at the University of Washington Foster School of Business. Natasha Jaques is an Assistant Professor at the University of Washington Paul G. Allen School of Computer Science & Engineering.