Rethinking Computer Science in the Age of AI

Transcript

Opening

0:00 So basically the model for a house 0:30 Report on AI application. Unclear in recording So 1:00 I think we last talked in two thousand and five when I was interviewing at MIT, so it's good to see you again. Likewise. Yeah. Both gotten a little older since then. Yeah. A little wiser, maybe. Up here in the tall chairs. Like, Mike wasn't tall enough, so we had to- We had to put the extra tall chairs up 1:30 here. I wanted to make sure that you could see us.

What changed his mind

Um, so I guess my first question to you is really, what was sort of your personal epiphany when you realized that AI would have such a big impact on education? Um, you know, let alone everything else.

Yeah, I mean, I guess so I got asked to chair this-- co-chair of this committee on AI and education. Um, I think I went into this committee thinking that what we would do is make some recommendations for how professors needed to, you know, have some sort of AI 2:00 policy in their class and, um, maybe think a little bit about how they evaluate, um, students in an era of AI. And we started, we started talking, um, about AI with just a lot of people on campus, like interviewing students and interviewing professors. And when we started to hear what they think about, well, like, "Well, my students don't really come to office hours anymore," and, um, you know, "I noticed that in, like, the living group that I'm in, students aren't spending as much time in study groups." And these kinds of effects 2:30 that kind of made me recognize that as a, like, campus, that AI actually has kind of surprising im-implications for what it means to, like, bring together people on campus and how, um, you know, the, the kinds of things that we sort of take for granted as being, like, the social aspects of learning are threatened by AI.

So I think that was it then, Tiffany. Unclear in recording Um, you know, I guess, of course, in computer science, like the fact that this thing would just like do any assignment that you might ask it to do, right? You know, 3:00 when the first time you- Tianyu and I, we had these labs that we had developed. With Tianyu a year ago in my, in my class was like, there's no way AI can answer these, like, GoDB concurrency labs or something. It's like, at some point, it just was like, you know, pretty much any assignment that you could give to this thing, you realized that, like, it could do it competently. And I think that changes the way that we think about how we teach it

Pretty much any assignment that you could give to this thing, you realized that, like, it could do it competently.

The coolest thing he has built with it

Yeah, 3:30 no, it's very similar, uh, thought. Uh, so another question just about you. What is sort of the coolest thing you have personally done with AI? You know, coolest for you. Like, the thing that, you know, sort of, um, you know, uh, made you feel delight to some extent.

Um, well, I mean, it just makes you feel like this-- I mean, the coding agent thing just makes you feel like a super- like a superhero. You know, these things that would have taken, would have taken, you know, weeks to do on your own, you can 4:00 just do. And you can, like-- What I love about it is that you can, like, build software that solves people's problems, you know? And it just, it's, like, so good. Like, we had this-- I-- This isn't, I don't know if this is the best thing I've done, but we had this, uh, you know, like, thing that one of the things you have to do is figure out how to nominate for awards, right?

And this is the kind of thing which is like, it's just annoying to keep track of, and maybe you have some 4:30 You know, like it's like super-- a-an AI-based tool to keep track of what are all the-- when's the ACM dissertation award nom? Because we always in the department at crayon, Unclear in recording like, we have to figure out who we're gonna nominate for the ACM dissertation award, and we have two weeks to figure it out 'cause we forgot, you know. So you have to run a process to find the best dissertations that, you know. And, like, this is a perfect example of a tool that in literally a day you can have a tool that is, like, great at notifying and keeping track of these things for faculty to go. They can upgra-- upload their 5:00 nominations for their students and their colleagues. You know, it's like a, a super simple thing that just, uh, solves a problem that the department had that, like, has been long-standing

A simulation of the Battle of Trafalgar

That's an ex- Here's another thing. This is, like, kind of a funny, goofy example, but one that I think is cool, which is one of the things that I spend a lot of time doing is talking to people about, um, how AI might change the way that we teach. And so one thing that we've been thinking about are some examples of how-- 5:30 not just how it changes. I think we all can imagine how we might use it in computer science, but how does it affect some other discipline? Like, how would it affect, uh, humanities?

So my co-chair of this AI committee and I, his name is Eric, and we were talking about something, and he suggested, "Well, like, what if you could, in the humanities, you could have it, like, I don't know, like reenact stuff, like make simulations of battles and stuff." So we were like, "Oh, what if we did a simulation of the Battle of Trafalgar?"

Okay. So this is a naval battle. I knew nothing about the 6:00 Battle of Trafalgar. Um, it was a naval battle, uh, between the, the British and the S-- the, the Spanish, and the British, like, sunk a bunch of Spanish ships. And, um, it was a, a turning point Like a, a big important naval victory for the British. So the, um, anyway, the-- it was a really interesting experience to build this simulation. So I built this simulation. You get... It's like you, you can imagine AI is very good at this. It's got the ships, and they're in 3D, 6:30 and they drive at each other. They, they, they bo-- they kinda go into each other, and they start shooting at each other, and some of them sink. But it's like the, the, the AI goes out and figures out the names of all the ships. Who are the commanders of all the ships? Uh, how many ships were there? What were their approximate positions? How did they approach each other? Which ships sank which other ships? Who died? How many ships left, you know? And so you get this whole reenactment of this thing in the end.

But you also-- Like, the other thing that is I, I really like about this example is that as the 7:00 person who's building it, I actually, um, found myself having to- There you go. Uh, I found myself having to... Uh-oh. Somebody's, like, leaning against it back to back, I think. Uh, I found myself having to think about whether what it was producing was correct, you know. And so I had to, like, not only did I learn a lot about the battle, but then I had to, um, like, as somebody who represents themselves, as somebody who thinks about responsible use of AI, I was like: 7:30 "Shoot, I better make sure this thing isn't making a mistake," you know? Like, it's not hallucinating something. And then I went off and I, like, fact-checked everything that it was producing. It was just like a really, actually a really interesting, uh, kind of a novel way to think about learning something that's like-

Advice for students in an uncertain field

Uh, on a slightly more serious note, you know, I think one of the things many CS students are feeling is that like five years ago, they started a major with kind of a predictable future. You get a degree, you get a high-paying job with a tech company, maybe 8:00 you go to graduate school, become an assistant professor, get grant money out there, and things, you know, are a little bit more uncertain now. Um, you know, things are just changing rapidly in industry. Things are changing in the country. Do you have sort of thoughts or advice to students on how to navigate that uncertainty?

Uh, I mean, it's an excellent question. I think, uh, you know, I, I think any faculty member who tells you that they have an answer is... I mean, uh, it would be-- I, I think, I think nobody knows with perfect clarity what's gonna 8:30 happen in the world, so I can appreciate that it's- I continue to believe that the field of computer science The old part Super-- Unclear in recording first of all, super interesting. Obviously, they're at the forefront of, like, what is happening in the world. Um, we have to train engineers who know how to build and 9:00 maintain and make these things better

Personally, I just don't see, like, the need for people who are experts in computation to go away. I think the nature of the jobs will change somewhat. I mean, it's clear to me that we're not gonna be writing-- individually writing code in the way that, you know, the, the vast majority of lines of code will be written by AI in the future. And that's gonna change the nature of the job and the nature of the skills that we need. I think that, um, uh, the people who are good at, 9:30 uh... But, but it's also clear that with these AI tools, the people who are good at using them, for whatever the definition of good at using them, can, can do, you know, more remarkable things than the people who aren't. So I think that there are-- there clearly are, you know, uh, uh, um...

And, and that, um, you know, when you talk to people in the industry, one of the things that they reflect about the, the state of AI tools right now, and of course, it might change in a few years, but is that, um, especially in very large software engineering projects, they find that the 10:00 tools, um, struggle to, uh, they struggle to get the tools to, um, work in the way that maybe when you do a small project like building the Battle of Boulder Bay do Unclear in recording because, um, the agents struggle to sort of reason about these very, very large code bases. And, and so you end up with situations where, for example, you'll have duplicate functionality that's implemented in slightly different ways that's hard to reason about. And so I think for the time being, being somebody who can understand what it means to 10:30 build a correct piece of software that is robust, that meets certain kinds of performance criteria, that meets the requirements of a, um, you know, some high-level specification is pretty important.

And I think, like, if you think about what software does, like, software is built to meet some kind of human need. And so we need engineers who can think about how we design software that, you know, meets those needs. Like, if I think about my awards website as an example, right? There's a set of specifications, 11:00 like requirements that that website has, and it's quite complicated, and it's not easily expressed as like a few lines of English. And so thinking about how we... Like, we need to train engineers who think about how to specify software, verify that software meets the requirements we want, um, satisfies a set of engineering goals. Um, I just-- I mean, to me, it feels like there's a lot of, like, interesting things to still learn in this discipline, even if, um, maybe the, the 11:30 thing that, like, most comes to mind when people think about computer science, which is programming, is, is gonna

Assignments, specification and verification

Does that, um, sort of fundamentally change the kind of assignments we wanna have? I mean, if I think about the assignments I have in my class, we sort of give people a very detailed specification exactly what an agent might want, and we have very detailed tests, and the students just do the coding piece in the middle. And so to some extent, what you said is the thing that I've asked my students to do is the thing they will never have to do again. And the things that I've done for them are 12:00 exactly what they should be doing for themselves.

The thing that I've asked my students to do is the thing they will never have to do again. And the things that I've done for them are exactly what they should be doing for themselves.

Right. Right. Right. Um, I mean, I do think, and I, and I'm not the first person to say this, I do think thinking about how we... You know, we, we, we have traditionally viewed some of those higher level sort of design tasks or specification tasks or thinking about the sort of those aspects of like software engineering that are around 12:30 Things that are sort of relegated to the most, uh, final like sort of capstone level classes that maybe many of our students don't even experience, um, because we're so focused on teaching people the nuts and bolts of coding, right? Or the nuts and bolts of how to build apps So, uh, I, I think that's the-- to, to me, that's the, the-- Yes, we do. I think we need to be thinking more about, you know, teaching students how to correctly specify and how to correct reason about the 13:00 correctness of software, maybe how to use formal verification tools as a part of the software engineering process, how to think about testing principled way. I mean, there are, like we know Software engineering has not been the core of our how to, how to engineer good and correct software 13:30

Universities move slowly. AI does not.

Another question that, that certainly comes up here, and I imagine most universities, is that, you know, the AI and the educational environment, uh, what they come to, what they come prepared with is changing really rapidly. But universities historically have moved very slowly. That there's, you know, layers of governance, there's processes. You have to, you know, plan new courses a year in advance. Um, do you have a thought on how universities, uh, can sort of adequately 14:00 prepare students and sort of mandate this environment when, you know, things are moving so fast, uh, in the outside world?

Yeah. I mean, it's a, it's a good point. I always... I say We were talking about this earlier, like, you know, to make a change, like you wanna add a class, a required class to your curriculum, to your degree, like to get a change like that put through, you know, it's like maybe if you started on September first, by September first of the next year, you could get that change made. But you know, if you start in 14:30 January, it's gonna be September, you know, two Septembers from then before like that class is that-- you can enroll students in that new curriculum with that new requirement, right? So you-- academia is just not set up to operate on a timescale of needing to, um, you know, if you-- if every little change to your cour-- the, the, the sort of the timescale to AI of changing is not the, not the timescale

And that makes sense. I mean, a, a lot 15:00 of academic disciplines are-- things don't change fast the way they're changing in AI and machine learning, right? Like, math hasn't changed in, in, you know, until the last six months. The way we practice math has not changed in a very long time, right? Uh, so, uh, yeah, I, I think, I think we, we have to find a way to be kind of nimble.

An AI variant of the systems class

Um, one of the things we're trying to do at MIT to be more nimble, at least in computer science, though I'm the head of computer science, so I get to kind of 15:30 nominally the... Our, our curriculum has an EE depart-- like, an, an EE division, an AI division, and a, a CS division that students can major in any of those three, like, submajors. So within the CS major, I'm in charge of that curriculum. And of course, we're thinking a lot about w- how we should change how we teach people how to program and other things like that. But one of the things we're, uh, have been thinking about is creating, um, within the existing classes, creating variations of those existing classes 16:00 that use AI in, uh, a variety of different ways.

So, um, as an example, uh, we have a computer systems class, and that computer systems class is a weird class at MIT. We've taught it without... It doesn't involve programming. So students read papers, and they do on-paper design of these systems. Why did they do it this way? Because we felt that, um, having people actually implement some of the systems that they could design wasn't really practical. It would be too much to do in one 16:30 semester.

So, um, we're actually creating a new variant of this class, which, um, we're coupling with our, um, upper-level software engineering class, where they apply software engineering principles and principles that they learn from the software systems class. They can get credit for both of these classes, and they take the AI variant, which means that all of the project, all the assignments for the class are using coding agents to build things, some artifacts. So imagine you could build a complete... You know, you learn about how TCP/IP works in 17:00 the networking component of this systems class. You can build a complete TCP/IP stack, right? From all the way from the raw hardware up and all the layers. You build each of the layers up, you test it, you specify it. And, um, because you're also learning this in conjunction with software engineering, you're thinking about things like specification, interface design, some of the aspects that you might learn in a software engineering class as well.

So I, you know, we-- Is this gonna work? I don't know. We're creating an experiment to try it. We'll probably 17:30 have, you know, fifty of the two hundred and fifty or three hundred odd students who will, will take the normal variation of this class, take the AI variation of this class next semester, and we'll see. Maybe, maybe it'll fail, maybe it won't. But, um, you know, I, I kind of expect that this is the way we're gonna start

And I could see an interesting variation also where you, uh-- One of my favorite exam questions is to give a system and then ask, like, what if you remove this feature? How well it would work? And say, like, what if you have two CPI things Unclear in recording 18:00 and you remove, you know, selected X, what happens? And then students can implement and play with it and- Yeah. Actually experiment with it. You know, what if it's additive increase, additive decrease instead of multiplicative decrease? Like, what does it actually look like in a way that, you know, in the past would have been much too hard to experiment with? Right. You can see, like, okay, or, or, you know, hear about things like congestion collapse or whatever, you can see them actually happening in practice- Yeah ...under real settings or understanding the... Yeah. Yeah. It's quite powerful.

Where should universities invest?

So on sort of a related note, I mean, I think, uh, 18:30 this is-- AI is, is definitely causing kind of a transformative change at universities. Have you thought about where and how universities should really invest in updating their curriculum?

Um, I mean, I talked to a professor at one university who said at his university, the provost provided ten thousand dollars across the whole university to update curriculum, you know, across five hundred professors, um, which, you know, probably is not enough and not a great place.

Freshman writing, and the subjects under most pressure

Yeah. Yeah. Um, well, I think one of the things that we 19:00 noted in this report that we wrote was that- Garbled in transcript Fields that are most obviously affected. But if you think about somebody, for example, who teaches like freshman writing, 19:30 I mean, these people at MIT are just, ah, man, they are in-- the, the, the people who teach freshman writing feel incredibly stressed out by AI because the way that they used to teach writing was by giving these writing assignments, these take-home assignments that of course are now the most vulnerable ones.

And you kind of imagine that a freshman who comes to MIT to learn some engineering discipline is not, like, super excited about being in a freshman writing seminar. And one of the things you see from students is that I think where they're most inclined to use 20:00 AI is maybe when they're being asked to do a thing that they feel like is, uh, not super interesting to them. Um, and so the freshman writing people I think really struggle with this. So we have to help them to think about how they adapt, and I think it is pressing.

Um, with respect to writing, I mean, I think we, we know some things we can do, having little frequent and They check in with the students to talk about what they're writing, how they, how it's going 20:30 All these things require work and, and time and adaptation.

The AI Education Exchange

So at MIT, um, we have, uh, created this thing, this structure that-- So in response to this report that we wrote, that there's a structure that's called the, uh, Education-- the AI Exchange. The AI Exchange has a education component, a research component. Along with the co-chair of this report that we wrote, we're, we're running this thing we call the AI Education Exchange.

So I think the-- I like the term 21:00 exchange because I think it's-- this isn't about telling departments what they should teach exactly, but more about recognizing that there are people across campus who are all struggling with similar issues, whether it's how to teach people how to write or how to teach people how to code or how to teach people how to prove things, and that we should bring together these different communities who have different Need to respond to these challenges in different ways. We should sort of create these, uh, these practice and have these people talk to each other and share knowledge. 21:30

Uh, as a part of this, MIT has allocated pretty significant resources, so we're aiming to allocate addition-- The way we're thinking of this is, uh, T- mostly TAs, but also some undergraduate researchers, post doctoral fellows, and others who work with instructors to do AI awareness adaptation. Our aim is to adapt, and then we have sta-- We're hiring some staff as well who will do help with development, like programming, and also help with sort of pedagogy, 22:00 ensuring the things that instructors are doing is rooted in educational best practices. Um, and we're aiming to do this for about twenty-

I grow and macro you go Like resources look like Unclear in recording

Well, I think like, I was thinking how many courses does MIT offer? And like, you know, when will this ever gonna-

Yeah, but so twenty-- if you think about twenty classes a semester. Right. Like, okay, 22:30 so y-you know, it's-- if, if you could do the, like, starting from the largest class to the smallest class, and you can do twenty classes a semester, in a year and a half, you'll get sixty classes. This is probably like, you know, it probably is a half or two-thirds You know, you do all the big intro writing classes, you do the big, you know, two or three classes, big service classes in every department. And those-- And then once you've done those, the nice thing is that so many faculty are involved in teaching those that they 23:00 maybe get some ideas about how they might do adaptation going forward.

So, um, this is a very significant... We're talking about, you know, four or five full-time staff, and we're hiring somewhere between fifty- It's a substantial allocation of resources to this. Um, and you know, I think I've been pleasantly surprised by MIT's

And, and I would say, like I think that 23:30 we talked to the provost and the president and the chancellor about this, and part of this was like Eric and I were the co-chairs who were partially running this AI education hub thing. We just felt like we couldn't do more. As the two people who were like doing this, we felt like we didn't personally have the bandwidth to do more

Practice, mastery and motivation

On sort of a related note, uh, one of the, one of the, the answers I've, I've heard or the analogies I've heard, um, to the problem of how do you get 24:00 students to learn basic skills is to think about it like, like exercising or working out. That, um, you know, we're used to sort of students doing things only in response to being evaluated on it. You know, there's things that student-- that we can't always evaluate, but students, we, we hope students will do because it will help them and they need to learn. Do you have thoughts on how to sort of motivate students in this sense?

You know, I had this similar analogy of like, is programming going to be like being a musician, right? Like you can, you can play music on a CD, and it's great, and if you wanna be a good 24:30 musician, you have to practice, you know, for a decade. And, you know, will programmers have to practice, uh, you know, and sort of work through that point where AI is better to get to mastery? Um, you know, how do we get students to have that mindset of this is a skill that I need my own motivation to work through. I can't rely on grades for everything.

Yeah. Um, I've talked to a lot of students about this. I, I, I alluded to this a little bit before, and one thing that you, you see with students is that I think when you ask students, like, "Why 25:00 do you..." They, they genuinely do feel like these are things they should learn, and they oftentimes will say, "I don't wanna use-" Recognize this But that doesn't, I think, give us a formula for how to actually teach people. Um, 25:30

For you, one of the things that higher education does, like we probably all have that experience of like, at least for me, I know there are a lot of things that I learned in university that I probably wouldn't have learned on my own because it's just hard to practice things on your own, right? Mm-hmm. So having a structured environment in which you practice things, um, is a very good way to learn, right? Mm-hmm.

And so I do think we-- there's a value to education, which is putting people in an environment where they're asked to do 26:00 a thing that they're not experts at, and where they have the space to practice that thing that they're not experts at. And I think if we look at education that way, that, you know, recognizing, A, students want to acquire these skills, B, recognizing that, you know, that's what education is about, right? It's like creating a scaffolding that allows students to learn something in an environment where they're not expected to be great at it. Mm-hmm.

It's like creating a scaffolding that allows students to learn something in an environment where they're not expected to be great at it.

Um, that's education, right? But ed-education might look a little bit different in this world. Like, I think there does probably need to be more of an 26:30 emphasis on experiences, like asking students to do things that are hands-on, group projects, coming into the labs and classrooms where they work on things together.

Theory lab: making the struggle social

Um, like I, I... One example I like, we have a, um, a, a professor who is a, a theory professor and teaches a randomized algorithms class. So this is traditionally like the-- one of the hardest classes at MIT. Um, and the class is his-- he views the value and the most important thing about the 27:00 class are the problem sets, the things students work on outside of class. And these things are-- I took this class as an undergrad. I was terrible at it. Um, it's like they're just super hard. They expect you-- expects you to be able to prove stuff at like a, you know, like re, re-- uh, applies yesterday's proof set. And it's the kind of thing that I'm sure AI could

So he's actually asking the students, I call it theory lab. He's like, makes the students come in like one hour a week and work with other 27:30 students Problem sets and their TA there who can answer questions you want, but I don't know that the TA And then the students report actually liking it. Now, these are students who are mostly taking this class electively. They're not, you know, encouraged to be there. But his-- anecdotally, he feels like the students actually really enjoy this.

And my theory about this is that, um, this is partly about, like, creating a space for students where they recognize that, like, other students actually are 28:00 struggling with this too, and it's not easy, and it's supposed to be hard, and I'm not just gonna get it. You know, I think there's this temptation, um, when you're in your dorm room and you're struggling and you can't figure something out, like, what am I gonna do except for turn to AI and ask it? And that's, like, exactly

The analogy to music is, is good, right? Music requires a huge amount of practice, and we teach students 28:30

Can AI help rebuild community?

One of the things I liked about what you said was sort of there's a social element to students practicing in a lab, and I think this kind of goes to, you know, one of the things we've seen and many others have seen are students, you know, coming to office hours less or interacting with each other less because they can get help from AI. Um, and I know a big part of the, uh, of your report at MIT was sort of, you know, a human-centered approach to teaching. And I was wondering if you have thoughts on how, how AI in some ways can actually sort 29:00 of help us build community among students and sort of encourage them to socialize more, um, and to sort of meet with each other more than perhaps they were a few years ago.

Yeah. I mean, I know there's lots of interesting experiments that people are doing where they, um, you know, use AI as facilitators for- Class, you know, might not be possible to have. If you've got a class with a hundred students, you might not be able to have a instructor who's guiding discussions 29:30 like I don't know whether that's pedagogically a great idea or not, but I, I think it's interesting to experiment with things like this.

Um, you know, I know people are also doing, uh, some very interesting experiments, like on, uh, at MIT, people are doing interesting experiments where they, um, task people to use AI to, like, um, in preparation for an in-person AI-free experience. You know, so, like, um, like for example, there's a, like a, 30:00 uh, a class, a humanities class on, I guess you'd call it like rhetoric or argumentation, where people are expected to, uh, come to class and have a discussion about some topic that they might not be experts on, but formulate an argument. Right? And so what they've done in this class is they've built an AI tool that's like a argument, like a... You, you prepare your argument with the AI beforehand so that when you show up to class, you've sort of rehearsed a little bit of what you want to say. So you can see why that, like, is a way that you're actually helping 30:30 students to develop the skill that they might not feel confident about, um, that they can practice a little and show up to class and be able to do.

Students who will not answer without asking AI first

So I, I think there are interesting things you can do. Um, I, I'm super worried about the, the things you mentioned, though. I just, I, I... It does, it does worry me that, um, you know, students aren't showing up in class and these kind of reports that you hear about, you know, students who don't-- I think increasingly students are not developing the confidence. Um, some 31:00 students are not developing the confidence in their own thinking. You know, like they, they feel like, "Oh, well, I, I don't-- I'm not comfortable making an answer or saying something without consulting AI first," and that kind of an attitude.

I think- We have to be really careful about that. And I think, again, a part of it is like, I don't know, maybe it's just about-- It's stupid, but maybe it's just about getting students to, um, like, be okay with making mistakes, you 31:30 know? Like, it's a kind of a small thing, but, like, I think, I think, like, our, our students have been brought up in an environment where they're, like, expected to be perfect.

Moving assessment back into the room

Uh, right. Well, I mean, certainly I think the high school application process, it might-- it certainly might be- Encourages students to be- Yeah. -perfect. And I think one of the things we've seen faculty try to do is to sort of think about, you know, as part of their, uh... A lot of people are moving assessments to be in-person assessments and sort of have, have things in class, you know, talk to students more. And I think perhaps there's a 32:00 chance that bringing back sort of face-to-face communication, you know, can rebuild some of the community there. That, you know, we used to have auto graders for things and online exams, and by moving away to that, we're sort of kind of giving them back those sorts of things.

Yeah. And you hear people, I mean, people who've been through... And, you know, it's obviously hard to make this scale at the scale of a university like Wisconsin. But, like, you hear people who've been through these, uh, like, programs where they had oral exams, for example. And, like, while that sounds 32:30 horrifically frightening, right? Both for the instructors and for the students. You know, I think when it's done right, people report that it's actually a really cool experience because you get to spend some time with, uh, somebody who's really smart, talking to them about your ideas. And, you know, it's-- it can be-- If it's done right, I think it can actually be a learning experience.

And there's, you know, there's some other stuff, right? Like, again, a stupid obvious thing we've said forever, but it's like we should tell 33:00 students that they shouldn't have their laptops out in class if they're not supposed to And like I think a lot of instructors have been hesitant to put policies like that into place, um, even though we recognize that students are super distracted in class and, you know. But like maybe AI is an excuse to say like, "You know what, guys? I don't want you to be using AI in class, so let's just put the devices away." And maybe everybody will just be a little bit more okay with that because

What should we still be teaching?

33:26 Department asking is sort of like, what will st-- what should we be teaching students in the future? What is the knowledge base? Um, and you alluded this to a little bit earlier about, you know, enduring ideas on, on software architecture and things like that. But in general, you know, what do you think some of the lasting knowledge base that CS department should be teaching is, um, going forward?

And how has that changed? Boy, um, it, it is a million-dollar question. I mean, and it's, uh, it's just one of these things where we-- you 33:56 know, I think because of the pace at which things have changed, it's a hard question to give a really definitive answer to because, you know, I guess when the AI thing happened, right, when ChatGPT first came out, it was like, "Oh, well, you know, yeah, it's pretty good at some things, but it'll never do X.

You know, it's never gonna be able to write a big piece of software. It's never gonna be able to, like, write an operating system." You know, and like these things that it can do now, it's never gonna be able to do a super hard proof. Like, these things that it can do now are things that I think nobody would have expected that it could-- 34:26 would be able to do in, in a very short period of time.

How far does the gradient go?

So I think we don't know what part of the gradient we're on and how long the gradient will continue. And it kind of is like if the gradient continues the way it has for enough years, then you see, you know, th-this sort of the extrapolation that you see the AI, uh, you know, AI companies saying of like, "Okay, you know, humans don't actually need to do any thinking anymore at all."

I mean, you know, that's-- I, I think personally, 34:56 we should never, we should never get there. I mean, even, even if it is able to do, um, all the things that humans can do competently, I mean, fundamentally it, it comes back to it's like music or something, right? I mean, fundamentally there still are things that-- or playing chess or whatever.

Like, humans still do these things even when computers are good at them, right? Because there's some inherent value in doing things with your mind. And I would-- you know, I think there's a role for universities in that world, but it, it is certainly a diminished role, and we don't know whether we're gonna get there or not.

Humans still do these things even when computers are good at them, right? Because there's some inherent value in doing things with your mind.

Read code, even if you do not write it

So okay, back to the question 35:26 of like what we should actually teach. I think for now my belief is that, um, at least within the world of software, like people still need to be able to understand what software is and how it works. Um, so that means they at very least need to be able to read code. I mean, maybe they don't need to be able to write code.

We probably don't need to train our students to like, write large volumes of syntactically correct code anymore. It's like people don't write assembly language anymore, right? I mean, we have 35:56 compilers who do this. And you know, I think there's a good analogy that an L1 is like a Unclear in recording

In some ways, right? Um, but I think there are all these under-- other underlying principles. You know, you talk to people who think about... You know, as you hear this word, you probably heard this phrase, people like throw around computational thinking. Like, what does computational thinking mean? I think one way is that clearly thinking about the world as, like, I-I-I-I think about how to break tasks down into these small 36:26 little units that I'm, you know, gonna...

And that's how I solve problems. Like, that's a useful way to think about solving problems in general, right? Um, so there clearly is something to be taught there. And then these things that I alluded to before around, um, how do you think about the composition of those little units into larger pieces of software convince yourself that those compositions have certain kinds of products or any of your properties that you would like 36:56

And then how do you 37:26

It's a lot to do and I think a part of it though is also, and I hope students appreciate this, that like I think it helps for faculty to just acknowledge that like, like you know, we don't, we don't know 37:56

It used to be

What industry actually hires for

I think the role of industry is in sort of helping shape the future skills development. So should we go to people at, you know, Google or Amazon and say, "Tell us, you know, what you wanna hire 38:26 people for?" I mean, I-- it's always useful to hear what people want. We haven't done this. We've never done this before, right?

I kind of think, I kind of think that, um, what you want You know, I kind of like if you ask why, why does industry hire undergraduates, right? It's-- they, they usually don't hire them because they have a very specific set of skills, right? It's not like I'm hiring you because 38:56 I think that's mostly why our students are getting hired, right?

Students are getting hired because they know that a student who can, you know, graduate from Wisconsin or MIT is able to solve hard problems, right? That they're able to-- They have this credential that, like, um, means that they're able to do the kinds of skills that they have seen over the years that Wisconsin or MIT grads can do, right?

Mm-hmm. And so that's what we have to continue to do, is to produce students who can 39:26 solve hard problems, and then I think industry will, you know, keep hiring. Insofar as industry, you know, I, I-- There's a world in which the software industry doesn't hire anybody anymore, but I, you know, I'm assuming that is not the world that we're in.

Um, you know. I guess the other thing about the software industry, though, of course, is that this is, uh, we don't exactly know where we're gonna land on this spectrum, right? But it is true, I think, that some of the classical pieces of software need fewer engineers to maintain them. But on the flip side, uh, it, it appears to me that we're 39:56 gonna write ten, if not a hundred times more, a thousand times more software than we wrote before because now suddenly, like my little example from the beginning of the, the tool for tracking awards, right, you can have a bespoke software system for almost any purpose you want, right?

And so there's a lot of demand for people know-- who know how to build and maintain and run and operate those systems. Uh, and even if, like, the first version of that award system was something that I could build quickly in a few days, like, being a person who 40:26 can maintain and, uh, run that thing, who can convince myself that it's, like, safe and secure for people to upload data to this thing, you know, that-- Like, it's not-- There are some skills that you need to be able to do and build and operate this sys-- these kinds of software, and I think there's a role for our students to play in that.

So it may be that, you know, instead of eighty percent of our students going to these, you know, the Microsofts and Amazons and Googles of the world, we'll see that eighty percent of our students are going to the, you know, 40:56 companies-

What should every CS student learn about AI?

On a related note, I think the other big question is, that we have is like, what, what should every CS student learn about AI? You know, currently AI is not a required course. You could graduate from our department never learning a single thing about AI. I think, uh, which would be a mistake, I think. So any student here, don't do that.

But, um, you know, thoughts on what is some of the main, you know, what, 41:26 what should students learn? Yeah. For what it's worth, so we have three degrees at MIT. Our computer science degree also, I think-

The, I mean, I, look, I think the obvious thing But it's like these things are so-- The, the sort of agentic programming tool set is amazing. I don't know that, like, you don't have to learn 41:56 a ton about AI. You don't actually need to know a ton. These tools have gotten good enough that you don't need to know a ton about AI

More like learning to develop a car. Yeah, exactly right. There-- So how much of under the covers do you really need to understand? I mean, um, you know, I don't know that you need... Like, a, a part of the, the victory of these LLMs or whatever has been that they have gotten it to the point that it's kind of like

But 42:26 so-- but of course, it doesn't mean that some students won't want to learn how it works under the hood. Um, you know, I think in a lot of disciplines there still is a lot of work to be done taking and translating those learnings from large language models and coding and pushing them out into other disciplines.

Um, and the, the AI frontier companies will do some amount of this, but, you know, they're not gonna cover all of the educations of doing this. So I think there's an exciting opportunity for students 42:56 to think about how these things work under the covers if they're interested in applying AI in-

I think, I think it's, like, probably, like, you could get through... I, I mean, I, I think it, it also seems to me that it behooves students to learn about how 43:26 these things work because, um, these are going to be the tools that are sort of the future. So, like, thinking about how does-- You know, and it doesn't necessarily mean you study super deeply how to train an LLM, but, like, you know, thinking about what goes into building an agentic harness or what goes into, you know, how did, how do these, uh, system-- how are these systems being, uh, trained to be able to follow instructions so precisely?

I mean, sort of 43:56 learning about the, the process that these companies are going through

Now because it's quite likely

What would you throw out of the lifeboat?

I mean, uh, another way of asking the question, is there anything that is required for CS majors that you think AI is more important to learn than whatever they have to learn now? Sure. What would you throw out of the lifeboat to make space? Um, and the answer might be 44:26 none because of the . No, I mean, I do think that, I mean, it's a little bit, you know, every, every topic is gonna have its defenders, right?

Mm-hmm. So it's a bit tricky to figure this out. But I do think that, as I said already, I think we need-- At MIT, we're already sort of thinking of maybe like the, the three-course intro sequence on programming probably could use, you know, at least the idea of a two-course intro sequence. And then we could introduce something on agentic AI, on agentic 44:56 systems as a, as part-- or part of their curriculum You know, I, I don't know.

I mean, it's, it's the, the-- there's the question about theory. I think we don't yet know what computer science theory really will look like. Um, you know, and, and being able to do algorithmic analysis of code is clearly important. Um, but you know, the, the degree to which we teach proof techniques, for example, I don't think that one-

It's not clear to me that 45:26 those techniques are as important maybe as we would have

Is that a question for Tianyi? Unclear in recording How-- Do you wanna take audience questions? We didn't sort of talk about a plan

Research, grad students, and what a professor is for

Okay. Um, I was gonna ask a, a couple of questions about research. Um- Sure. And I guess 45:56 one, one question I have is, I've heard a couple professors here say they can now do their research sort of with AI and agents faster than they can with grad students. Um, and as sort of question, um, you know, how-- what is the, uh, how do you think about the, the dual responsibilities of being a professor at MIT of both doing research and educating grad students?

You know, if you could do all the research with no grad students, uh, and maybe produce more, but not 46:26 produce grad students, would the eventual outcome allow that? For some professors. I don't, I don't personally think it's an eventual outcome. I guess part of it is that, like, I'm at this point in my career of where, you know, one more thing on paper or whatever, you're not, you know, it's not gonna move the needle one way or the other, right?

So, like, I, I view fundamentally my job as a professor, I think, as training young people, not so much as like... You know, I guess I have some research goals and things like that 46:56

You know, I think my responsibility is to train a generation of young people to do research. Mm-hmm. Um, and I often get asked this question, you know, there's the trickle-down version of this question, which is the grad students say, "Well, why should I bother?" 47:26

And it's like, well, you know, because somebody bothered to take the time to train you to be a student, you know? And so you have to-- like, we have to invest in training people. I mean, that's what we're in the business of doing, is training people in academia, right? It's, um, and yeah, satisfying the funders and making the research progress is important, but, uh

I think it's different, but I've had some senior faculty express to me the concern that, you know, 47:56 they-- for them

thing. Like I need to get tenure, so I need to have a certain number of grants and papers and a certain degree of impact, right? And if AI helps them to get there more quickly, then, you know, they have a sort of a rational choice. The, the rational thing to do would be to choose to use AI. So I mean, I'm a little bit sympathetic to that, but I do think, you know, I do think as a, as a, as universities like, um, we need to make sure that we're 48:26 evaluating junior faculty on their...

I think mentorship should be an important part of how we evaluate junior faculty. Um, you know, and I recognize that there are some-- I'm sure there are people

Questions from the room

Okay, why don't we go to some questions?

Question. I don't have to think about this Unclear in recording 48:56

Discussion, uh, among the PL family about what kinds of concepts are gonna survive in the near future, at least. And I was a little 49:26 surprised, but then immediately converted to the idea that, uh, a programming paradigm's

It was one of these because it gave you the tools to sort of argue with the la-- large language models about how you s-should structure the code and, and organize your thoughts about

Are there other things like that in other parts of 49:56 computer science that you've seen where-- I mean, this course was described as it's taught in every computer science department in the country except when you have PL faculty on faculty because we don't like to teach it and we don't think, we think it's below us or something.

But, um, despite

majority of comment. Unclear in recording Are there other classes where, where, where what, what is being taught is 50:26 sort of- Okay ... a conceptual framework to increase the AI tools effectively? Well, look, I mean, I think I, I, I sort of alluded to some of the things that I think might be available to computer science more generally is 50:56

The concepts that survive

Things that, like, we're gonna need to ensure that it-

Maybe AI will

Right? But I, I kind of think that those things actually require a little bit more deeper understanding to understand, okay, like, well, you know, public key cryptography is a good solution for this 51:26 particular problem because of X, Y, or Z, right? Like, that sounds like a thing... Or, or, you know, decomposing this piece of software into the following components will, uh, you know, I don't know, help me to reason about its, you know, help me think about, like, like how to scale it to make it more performant.

So by separating the, you know, in a database system, I separate the query processing front end from the storage back end so that I can independently scale storage and, uh, 51:56 query access. Like, maybe an LLM would just decide to do something like that, but my experience with it is that it's unlikely to come up with architectural decisions about how to build software in that way.

And I think-- So I think there is a lot of value to teaching some of those, you know, conceptual, higher-level understanding and thinking about the construction of software, um, that continues to be, you know, really important. You know, I think the same thing could be said about sort of thinking about efficiency, algorithmic efficiency, asymptotic behavior of 52:26 things, you know, whether a particular implementation is a good implementation of something.

Again, like, LLMs are pretty good at, like, if you said, "Okay, let's try to rephrase this thing this way, or let's optimize this piece of code." But I think that right now, anyway, for large code bases, humans do still need to kind of understand how those large code bases work. Any of you who's ever worked in industry, you know, you, you know that for any large piece of software, there's a small group of people who have this, like, very deep 52:56 People are incredibly critical to the organizations, and I think it's a lot of our graduates who go and become those people, and we need to train those people.

We need to continue to produce and train those people. Um, I, I think there's also kind of a reality in which maybe we don't, you know, we've been training a vast number of people, right? Maybe we just... At MIT, sixty, fifty percent of the undergrads have been in computer science for the last three years. Like, I don't know that we actually need that to be the case, right?

Like, um, like I, 53:26 I, I mean, much as it's good for our department, we get to hire a ton of faculty. We get a lot of resources. I don't, I don't know that it has to be like that, right?

A student who chose not to use AI

I'm curious from-- So from a student perspective... Can you guys hear me? Yeah. Yeah. Uh, from a student perspective, one experience that comes to mind is I took a class this summer on machine organization and programming, and there was, like, these Canvas quizzes in the 53:56 class where I actually intentionally decided not to use any AI tools to help me on them.

Um, and I was actually put at a disadvantage because of that. Uh, so I'm curious if there's, like, one or two changes that you would suggest faculty make to their coursework so that students who maybe wanna learn the material, but, uh, you know, it was difficult concepts, and it's something that's very human error-prone to where the LLM actually outperforms, you know, the average student who's trying to learn it.

Um, are there any changes that you 54:26 suggest that faculty make to the coursework to help, uh, encourage that active learning over just, like, relying on the, uh, AI tools? Can I ask a quick clarifying question here? Which is, I'm wondering why, why you feel you were put at a disadvantage by not using AI tools?

Uh, well, the tools were, like, a hundred percent accurate

I mean, is it-- Uh, but do, do you think that is it because you wouldn't do as well as other people? Or would you do-- Will you 54:56 do-- Or did you feel like you would get a worse grade if other people used AI and you didn't? Uh, it's more of a grading thing. Okay. So I think, I mean, so the, the, the, the-- There's a bunch of subtlety, actually, to your question.

It's a very good question. Um, so first of all, exactly your answer there points out one of the challenges with grading. Like, if we didn't-- If students weren't so worried about grades, right? Like, maybe this just the whole conversation wouldn't matter as much, right? Um, 55:26 but we've created this culture of grades really mattering.

Uh, one of the things that we talk about in the report is this sort of like, you know, Harvard came out with this thing last year where they said, "We're gonna cap the number of grade-- number of As we give because grade inflation is rampant," blah, blah, blah, right? But like, I, you know, maybe if you weren't worried about whether, you know, using AI was gonna make you get a, you know, a B in the class or something, this wouldn't then be an issue, right?

Or not using AI, excuse me. Um, 55:56 but I do think that professors also need to-- Like, this is a very good example of a case where the professor could have done something. Instead of doing an offline Canvas quiz, right? You could have had an in-class quiz or turned these quizzes into some smaller, lower stakes, more frequent in-class evaluation, for example.

And that takes some time out of your class. You have to redesign it. It's not a, like, you know, it's not trivial to do this. Um, or you can have proctored classes, 56:26 exams outside of class. I mean, you know, there are ways that we have to evaluate students that are, um, robust to AI in a way that, you know, what you just described was not robust to AI.

And I think, um, you know, professors need to, to think about this. Like, if it's important for students to demonstrate mastery of the material and exams are an important part of the class and the way that they are demonstrating mastery, then probably, like, having those exams be things where people can just use AI if they feel like it 56:56 is, and, and, you know, get a very high score on the exam is not a great outcome for the class.

And I think professors should think about what are the learning out-- Start-- You know, there's this kind of simple idea in education that is obvious in retrospect, but they call it backward design. Start from what you want the students to know and need to be able to do, and then figure out how you design, uh, like, co-- in-class exercises, exams, evaluations that, you know, actually show that students are achieving 57:26 those outcomes, you know?

There's this kind of simple idea in education that is obvious in retrospect, but they call it backward design.

The K–12 literacy crisis

A feedstock question. Unclear in recording So what, if anything, should our community do... Oh. What, if anything, should our community do for our K through 12 literacy crisis? It's hard to imagine millions of US kids using any of these tools effectively if they can't read properly. So what, you know, what should this community do?

Because that's an, a civics busting issue, right? Unclear in recording 57:56 Yeah. Boy, um, it's, you know, I, I, I'm not sure I have a super great recommendation, right? I mean, it's like teach your kids to read. I mean, I think, um, you know, one thing we as higher education people can do is to

You know, a thing we're trying hard to do at MIT is to try to make sure that a part of the, um, you know, that, that our... You know, we teach a very small and elite group of people. And so I think 58:26 some of the criticisms that you hear the rest of the country lever-- you know, putting towards the, the top tier, especially of higher education, it are, are valid in the sense that we are only reaching this very, very small group of people.

And so I think it's, it's a good idea for, uh, higher ed in general to think about how we reach out to communities and engage with communities more. Mm-hmm. That's not an answer to the literacy crisis. This question is so complicated and mixed up in technology and 58:56 device use and the sort of role models that, you know, we're, we're showing to students and a whole bunch of other things that I, I don't have a super great answer for.

But I do think that in higher education, we can be more engaged with the communities that we operate in, and that, um, you know, if... A-again, like if students-- If we can help students to see that learning to read and engage with complex texts is the way to get to the place they wanna be. "I wanna be a scientist.

59:26 I could be a scientist, but I have to be able to read these texts in order to be able to do that." That's, you know, may-maybe that may... You know, and if that's a realistic view, that's a realistic career pathway for somebody and not the unrealistic ivory tower, maybe that's a way that we can inspire some kids to want to do this.

Because I, I think that a part of the thing about reading, like the MIT kid taking a humanities class who just wants to get through it, like a part of it is that students don't necessarily connect that ability to engage with hard tasks with 59:56 some outcome that is like a, an achievable outcome for them, right?

So I, I, I mean, I-- It's really hard though. Yes.

Taste, and what the frontier labs are buying

Um, I'm doing something about aerial science Unclear in recording and, uh, o-one of my friends d-- is doing something about and he's, uh, he's shocked by the paper one point two which published this afternoon, 1:00:26 uh, performance in, uh, uh, uh, a robotic benchmark. And, uh, I am doing another area, another similar areas about aerial science.

And what are, what we were like, uh, is that more likely that, uh, the, the, the generative AI, uh, especially like the AI in large company, uh, might like, uh, switch out our experience and replace the, uh, uh, uh, uh, without such research field. Uh, should we-- or, 1:00:56 or like more likely that, uh, our experience and taste may, may, or may, may still work, uh, uh, after that?

Yeah, I mean, I think you used a good word there, like taste. I mean, I do think that there's a lot of value in, like, understanding, thinking about the important problems to work on and the problems we can work on. I mean, when you look at what the, the frontier AI companies, the way they're addressing this taste problem is to throw a lot of money at the people who have taste, right?

When you look at what the, the frontier AI companies, the way they're addressing this taste problem is to throw a lot of money at the people who have taste, right?

So how have they done so well in computer science theory? They've hired 1:01:26 world-class theorists to tell them what problems they think should work on. I, I think that, um, you know, it's likely that they are doing the same thing in some domains of science. And so we will, I'm sure, see some breakthroughs from these models in science.

So I, um, you know, it's clearly, uh, you know, one of the directions we're gonna head. I-- you know, I do think academia is a, is, is, it's a uniquely diverse... Big research universities have a uniquely 1:01:56 diverse collection of people across a wide variety of fields. I think that may be our, you know, our sort of superpower that, like, uh, every university has a concentration of diverse thinkers that is, y-you know, as broad as what any one of these frontier companies has.

And how, how do we think about leveraging it? I mean, there's an opportunity, um-

It, it will be very interesting 1:02:26

It will, it will be the case that I think some of these breakthroughs that we've seen in, in math and theoretical computer science will also happen in scientific disciplines, at least scientific disciplines where there's a verifiable outcome. You know, you have something that you can test that convinces you that something's right.

You know, a really good solid biological model of an organism

1:02:56 Quite likely

I think we're about out of time. Or do we have time for one more? Finish the final comment. Okay. Sure. Uh, let's say one more. Who has the mics right now? Yeah, there are three questions left. Okay. Okay. We'll stop after that

What is the department actually doing differently?

Not on the nano level? Nano, yeah. Unclear in recording Um, so of course there's been a lot of 1:03:26 conversations here, and obviously everyone's been having these conversations, uh, themselves about, you know, what should we be doing about AI? What should we be planning for the future? How should we be planning for the future? What should be changing?

And like you started with, I think the default answer to a lot of these questions is, uh, no one really knows, right? Um, and it's certainly the case that no one really knows what AI will be like in a few months and beyond that. Um, we do know what the, what AI is like now, and like you mentioned, what the gradient is, or we 1:03:56 can have maybe, I think we have an approximate of what the gradient is now.

So I guess my, my question is, um, with respect to the current abilities of AI and the current approximate of the gradient, um, what are you specifically, what is, what is your department doing specifically, uh, that's different from what it was before? For example, my workflow as a research grad student is changing.

Um, so what are you specifically changing with respect to AI's capabilities? Uh, I mean, so I think I alluded to some of the things we're doing in teaching the, in the 1:04:26 curriculum. So trying to think, um, pretty, pretty broadly about how, you know, we know what-- Like, I guess I would view it as like, let's assume that like we have AI at the capability as, you know, we know it's at least as good as it is today, right?

So like, how, how should we be teaching given that? You know, I think that's already we know software engineering is changing. Already we know that, um, we have to do much more ambitious things. Uh, do we owe it to our students to create more, 1:04:56

you know, sort of, um, big... Allow them to do more big, uh, big kind of engaging, um, challenging projects given what AI can enable them to do? Sort of like probably what, how your, your research workflow has been enabled, right? We can enable undergrads and grad students in educational experiences to do much more, uh, you know, I think remarkable things that challenge us even more.

So, 1:05:26 um, I guess those are the-- You know, and then within research, I mean, in my research, much like my students are like, you know, they build software systems, so they're of course using AI tools to build everything, trying to make sure they have the resources to do it, access to the, the models that they need.

But, uh, you know, spending a lot of time talking about it because, you know, there is a lot of uncertainty. So I've been trying to reassure students and make sure they feel comfortable and understand what some of the challenges are. Um, talking to the things, you know, I think it's important to talk about-- have conversations 1:05:56 about what is appropriate and what is not appropriate use of AI.

So that's what I talk to my students a lot about. I love to show them examples of, like, terrible AI-written papers that are like, you know, this is written by an AI, and I cannot under-- I'm, like, an expert in this field in theory, and I cannot understand what these-- this thing is talking about because it makes up these, like, horrible multi-word noun strings that define concepts.

You know? Like, it thinks that it can just like... Anyway, uh, it's like, so talking to the students about, you know, what 1:06:26 is... And, and, like, within research groups, within communities, I think we need to have these conversations about, like, what is normal and what do we expect and what is okay. Like, is it okay to use this to write recommendation letters?

Is it-- Should you expect that your professor might write a recommendation letter? Is it okay to use this to write a research statement? Is it okay to use it in your field to write code or to write a proof or to write a paper or to write code or to write a proof or, you know, like, I, I just... So, so it's a combination of many 1:06:56 different things, but, you know, I, I think we're, we're doing a lot of stuff.

I mean, it may sometimes feel like we're spinning our wheels, but I think actually even all this talking is actually doing stuff. So thanks. I know. Who else had a microphone? Um, so Uh, can you turn the microphone on? 1:07:26

Will writing well matter more?

Why don't you go ahead? Okay, my question is that, like, uh, what do you think hu- like, humanities classes or English classes will play, like, a bigger role in, like, computer science? 'Cause if you don't know how to, like, write a prompt properly with- mistakes like you do that compared to someone that knows how to write 1:07:56

I was joking with somebody the other day that maybe, you know, at MIT, we don't have a computation requirement for the freshmen

get computation into the undergraduate curriculum, curriculum by, uh, claiming

I guess you're kinda right. I mean, I don't, I don't actually know. These things don't seem to care very much if you write, like, really half-formed, badly written 1:08:26 English. They somehow still magically figure out what you meant. But, um, it, it would-- it's a very-- it's-- it'll be very, it-- I think a very interesting question of how sensitive, sense-- how sensitive the algorithm is.

I think there's some evidence that people have done some studies that show that, like, badly written, sloppy English does

Bugs in them or that have security flaw, like really kind of weird, surprising effects. So, uh, there 1:08:56 is an interesting, uh, sort of, you know, it's almost like sociology or biology or something, studying the behavior of LLMs under different operating conditions that I think is a field- I don't, I don't

Will the role of the university shrink?

You made, uh, two statements. So you discussed pedagogy a lot, and you discussed, um, the tools a 1:09:26 lot. So one thing that you seem to, you know, square on is that in pedagogy, students must be excited in this new era when it's very easy to cheat or get through all of their assignments. And then also that it's possible with the expanding domain of AI tools that intelligence could become a more humanistic endeavor because like you said, even in a world where AI can do all of these things, there's still a role of the research university, uh, similar to, you know, chess, right? 1:09:56

But I also find-- I'm a math and CS student. There's a large amount of apathy among a lot of especially undergrads. Like, a lot of people came into, you know, their universities looking for a job. Do you think that in the future, the role of the university might shrink? The role of research might shrink? Like, like broadly, do you think that in general it's, uh, because we have these tools, the amount of people enrolled in universities and the amount of people 1:10:26

Um, it's a very thoughtful question. Thank you. Uh, the-- Uh, so, you know, I think, I think in, in-- there is really is a kind of in, in math and theory, a thing that has been for a real like a decade, right? It's like, what is-- how do we-- what is the value system in these fields? What is the, like, point of doing these proofs when like-- Because, because I think in 1:10:56 especially the, the more theoretical the field, the more the, like, value was in the proof itself.

Like, you proved a hard thing, and that's how you demonstrated that you were good at it, right? And, uh, the, the, these, these-- So, so when AI can do that, then it's harder to, like, understand what is good work in that area, right? It's a little bit different in software because, you know, I think good software is not-- You know, yes, there is a little bit of like somebody could build an impressive piece of software.

Now AI can automate that. But a lot 1:11:26 of good software is more about, oh, you came up with something that-- with this idea, this really elegant idea or design or something like that. And so far, AI has not automated that as in nearly the same capacity. Um.

The impact on jobs is still somewhat uncertain, right? I mean, I think we're still... I, I, you know, I, I think we are seeing fewer people already enrolling in the EECS 1:11:56 majors. Um, I don't think that there's evidence yet that AI is dramatically impacting, um, jobs outside of computer science and math, right? Like other engineering disciplines seem to be robust and fine.

So whether that existential crisis spreads, it might. In computer science, I s-- you know, at least when I look at MIT job outcomes, I think the students are not getting the same degree, the same number of job offers that they used to get. 1:12:26 Um, but there's this complicated effect of like the post-COVID. You know, there's this huge boom of hiring and companies, you know, scaling back on the amount of hiring and some amount of the layoffs that we're seeing are really just companies laying off people who they-- when they overhired during the COVID boom.

Uh, and then this, whatever we call AI washing, it's-- people are calling it, and they claim that this is in the service of AI, but it's really just a way that they can reduce their workforces. So these effects are in, in 1:12:56 tension with each other. Although I do think there are probably fewer software engineers in the world than we did.

At least big companies may be able to do what they could do before with-

A diminished role, or a different one

So it's a really, a really complicated world. I, I do think that it is not impossible that the role of academic computer science, you know, sort of will be a little bit diminished if we're more like a humanities department where we're studying, uh, 1:13:26 computing because it's an important artifact as opposed to the generator of jobs.

That's a different role, you know? It's a-- it's maybe a bit more like even math these days. It's a more of a service. You know, okay, we need to train some mathematicians, but we don't need, you know, thousands and thousands and thousands of mathematicians. Maybe we don't need as many programmers as, as we have.

I mean, we're gonna, we're gonna have to wait and see, I think, how it shakes out. Um, the students will have a look at their feet and where the jobs are. Right now, my perception is 1:13:56 that our students in computer science, at least at MIT, I can't speak for other places, are still getting pretty good jobs with salaries

Thank you. Let's thank Sam again .