We Taught the AI. Now It Teaches Us.

AI is the new guy on the team. He's smart, talented and learning fast. The boss wants us to show him the ropes. He picks things up quickly and some things he's already doing better than anybody else. Soon, he'll know all our tricks. Will they still need me?

Artificial Intelligence is based on machine learning, mostly "supervised" machine learning. We, the supervisors, show the machine learning model 10,000 cats, and 10,000 non-cats, and it slowly assembles in its mind a high-dimensional "map" (I use a lot of quotes) to help it learn what's a cat and what's not. We are the ones that teach the machine learning model, the "AI".

And now the roles are about to reverse. The AI will be the one teaching us.

Will it take our jobs?

First thing to ask: is this going to take human teachers' jobs? And an immediate follow-up question: teach what and to what end? When the pocket calculator became cheap and popular, we started questioning whether or not we should still be learning the multiplication table by heart. The answer still isn't clear. But now, if AI can do everything (or will be able to within 2 to 10 years), what's left? Do we need to learn how to program? Learn how to 3D model? Do we need to learn how to play musical instruments or do music production?

So the open questions are: Who will do the teaching? What will they teach and to what end?

What chess can teach us

We can learn a lot from how chess has evolved in recent years. Computers now easily dominate even the best player in the world, Magnus Carlsen. It was reasonable to think that once chess was "solved" it would cease to be interesting. Much like tic-tac-toe isn't very interesting. Or alternatively, that we will watch different software programs battle each other, cheering on to our favorite version. That has not happened. People are as excited about chess as they ever were, excited to learn chess, compete and watch others play.

Not only that, having chess programs readily available uncovered new techniques in chess, new variations of openings were shown as ok, preparation for a competition was transformed thanks to chess programs, real-time analysis of games became possible and more.

Furthermore, and relevant to our teaching topic here, the ability to learn and practice chess in a more effective way by younger players results in stars like Gukesh and Faustino Oro solidifying a career in chess at such a young age.

So the takeaways from chess to me seem like:

1. Humans will enjoy learning and performing even if computers are much better.
2. Humans will enjoy watching other humans perform and compete.
3. Computers can help us learn things more effectively and at a younger age.

That last point is important and further supported by research.

What the research says

Chess is one anecdotal case study, but the research points the same way. The classic paper by Bloom from 1984 showed that the average student working with a personal tutor outscored 98% of students in a conventional classroom. Two standard deviations (two sigmas) better. The problem, which he called the "2 sigma problem", was that it was too expensive to provide every student with a personal tutor. One-on-one tutoring doesn't scale.

Guess what, Bloom. It's possible now using AI!

The biggest opportunity: homework

I think the most obvious opportunity to change the way we teach is in the homework workflow. Homework sucks. Not because of the reasons the kids will tell you, and not just for the kids, but it sucks at the university level too. Homework assignments suck because the feedback (the actual learning) comes with a big latency, happens at most once, and is usually limited.

The reason for all of this is that teachers don't scale. I'm sure all the teachers would tell you themselves. If you have 18 students in your class, and you need to read a five-page report and give detailed feedback to each, this would take a long long time. As a result, the feedback will likely be less extensive, the timeliness of it will suffer as well, and by the time the student receives the feedback, they already forgot what they thought while making it, they may or may not read the feedback and would likely not follow up with follow-up questions or try to fix their mistakes. One time, limited feedback given a long time after the assignment was submitted. And for that reason, the current workflow for homework sucks. I rest my case.

For the first time in history, we don't have to do it this way. LLMs can give the student feedback in real time, they can correct any misunderstandings they have, they can teach the student things they may have misunderstood for months or years, solidifying the foundation, they can even jump in to offer feedback while the student is working if they are headed in a wrong direction. That is Bloom's loop! The tutor gives a test, corrects the mistakes, gives feedback, and circles back with more tests until it sticks. Bloom called that engine mastery learning; run for a whole class it reached the 84th percentile, and paired with one-on-one tutoring it hit 98%, the full two sigma (later work has trimmed that number, but the direction holds). An AI tutor can finally run both halves, one-on-one mastery learning, for every student at once.

I teach in Berkeley's MIDS program. All the students are motivated. All the students already have experience in the workspace. They want to be there and want to learn. They do however come from very different backgrounds. Some likely know more than me on certain subjects. I'm not supposed to admit it. But some do. Others describe themselves as non-technical since maybe they programmed their first Python program a semester or two ago. Each student came to learn, each student has a lot to learn. But in the classroom we focus on one lesson plan and a set of learning objectives. If I explained programming fundamentals, many would be bored. If I talked about advanced neural network architectures too early, I'd lose many other students. Classrooms, even online ones, are typically limited to one lesson plan.

But an LLM at home, an AI tutor, can "run" with the student. It backtracks to fundamentals when needed, comes up with ad-hoc quizzes or assignments to verify understanding, and answers the student's curiosity when they ask a good question, even one beyond the planned scope, because that helps motivate or put things in context.

The student gets all the feedback they could possibly want, they are steered back on track if they are wasting time working in the wrong direction, the assignment isn't done until the student actually achieved the learning objectives, their curiosity is encouraged and supported, the foundation is repaired if necessary, learning tools are spun up on the fly as necessary, according to the needs, the context, and the teaching approach that fits the student. All that is possible now.

So... will AI take our jobs?

I am not sure how exactly AI will change the work we do and in what timescale. I do know it is empowering me to do much more and to do it better. I see it when I develop sites, when I write blog posts :), and of course when I teach. I can develop stronger and more comprehensive lessons, answer students' questions faster and with greater consideration, and that's just the start. For me as a teacher, I want to maximize students' mastery of the subject and I know AI can help me excel at it.

But this goes beyond my work and experience. At the beginning of this blog post we asked a few questions: Will AI take our jobs? Who will do the teaching? What will they teach and to what end?

I will not pretend to know which jobs survive, or on what timescale. What I do believe is that people will continue being curious, wanting to learn and pursue mastery, just like they kept on learning chess when the engines got stronger. They actually ended up learning more, starting younger, all because mastery itself was the point.

And here is the part that matters even for the broader jobs question:

You cannot supervise what you do not understand.

If nobody learns the fundamentals because the AI handles them for us, we stop producing the people who can direct the AI well. Understanding does not get less valuable in an AI world. It gets more.

So who does the teaching? For now, teachers and AI together, with the teacher's role elevated rather than erased: the AI runs the loop, and the human sets direction, judges, mentors, and verifies. For the foreseeable future, understanding is what keeps a human in the loop. That is worth working toward, and it is exactly what I am trying to build.

The Uriversity prototype

The software I am developing to leverage AI toward education is called Uriversity. I like names and this one amused me. It will do for now.

The first prototype is built on a Jupyter notebook viewer, it cuts the notebook into segments so as not to overwhelm the student. It has open ended embedded quizzes that you can answer and get feedback in the form of a chat. The AI chat, while explaining, can spin up a handful of tools to help make sure the student understood, including charts, quizzes and code segments. You can view anonymized cleaned up versions of questions by other students to recreate the awesome classroom experience of learning from other students' questions. I tried to embed what I've learned from seven years teaching at Berkeley into the system.

It's a prototype for now.

I would love to hear what you think of the ideas here, and sign up to join the waiting list to give Uriversity a try.