Education & MAMLMs: Josh Gans’s view is that our students already find it much easier to ask questions of ChatGPT than to go to office hours or email and get an answer back a day or two later, and so they will ask ChatGPT the questions. The result is that the average quality of the answers they get back will be low: ChatGPT has been designed and trained to exhibit mammoth amounts of verbal linguistic fluency—it can be quite persuasive—but its level of substantive knowledge and misinformation is that of your average internet s***poster.
Yes, it is possible, through "prompt engineering", to do something to direct ChatGPT's attention to that part of its lossy-compressed training data that contains reliable information. But our students do not know how to do that. And even those who claim that they do know admit that it is a black and unreliable art.
In these days of MAMLMs, we have already added an average internet s***poster to our course teaching staff. And that s***poster will have many more contact hours with our students then we professors and our TAs will.
Thus I believe that, given the evolving information ecology in which our undergraduates are already immersed, we have a very strong moral obligation to do what Josh Gans and Kevin Bryan and company at All Day TA are doing—to attempt to train the MAMLMs that are students are goig to consult over the next semester as the first-line answerers of their questions, and train them to be as high-quality as possible:
Josh Gans: The Value of AI for Uneven Work: ‘Our launch of All Day TA <alldayta.com… [because] we believed that the ability to serve students well… was fundamentally limited by the scarcity of teacher attention. The aim is to use AI to relax the teacher-attention constraint <alldayta.com>. But teachers (including professors and their teaching assistants) are winners when attention is scarce. So it is not surprising that a product like ours would raise concerns. However, we believe it is important not to exclude students from the equation. They are the ones who suffer when teachers have limited attention…. The first thing to note is the volume of questions… an average of 50 [per student] by the end…. 30 per cent of questions are… [asked] to understand concepts… 30 per cent… students… searching for definitions or places… where they can learn more… 30 per cent are questions that students are otherwise embarrassed to ask…. The final [10%] ones are mostly administrative questions. The point here is that you may have thought your teaching team was serving your students, but they likely have many unanswered questions…. Variability. You can see three “events.”… The course began… a mid-term and a final…. Imagine employing a person whose job it was to wait around all semester doing very little until a couple of days when they have to do more than would be humanly possible…. The data here show just how extreme the [question-answering] work requirements of a teaching assistant are…. Ajay Agrawal… pointed out to me that this is precisely why software is valuable. It is built for tasks that have precisely this sort of unevenness. AI means that software can now step in [and]… during these surge times… triage the low-hanging but important fruit of student queries…. Those who turn up at… assistant office hours or write emails… will be… for tail questions either from struggling students or very advanced ones. Thus, teacher attention will be better allocated to where it is of the most value and away from where it is trivial. This is the promise of the “software eating the world” trend coming into the teaching world… <joshuagans.substack.com…>
