A Humanist’s Adventures with AI
Like anyone who hasn’t been living under a rock for the past two years, I’ve been having an increasing number of conversations about generative AI. Because of the prominence of the topic in the world of teaching and learning, I’ve also had the chance to attend a range of talks, symposia, and other events about AI. Until very recently, however, I had mostly treated AI tools as sandboxes – I would try them out just enough so I could have a base-line familiarity with how to use them, but I didn’t actually, well, use them for anything meaningful.
Inspired by the Summer AI Institute organized by the Center for Teaching and Learning, I decided to give AI an actual whirl. I was particularly interested to see how AI would handle tasks related to the humanities, as those are the disciplines I am most intimately familiar with. A dozen app accounts and a few hours later, I can’t say I’m an expert but I do have some observations about the particularities of AI and the humanities.
Peculiar particulars: Explaining terms
I wanted to see how gen AI might do with explaining some Classics-related terms. I asked Bing Copilot – my favorite because it uses a higher-powered AI algorithm and provides links to its sources – the following:
In the field of Classics, people seem to use the term “reception” a lot. What do they mean by it?
The answer started out cogent, accurate, and accessible to the non-specialist. It moved from a simple definition to some core assumptions (“classics have been interpreted from different ideological and political stances”) and even the role of reception studies in the field (“This approach has invigorated classic studies”). Toward the very end, though, things took an odd turn as the response dove deep into theory land:
“It’s worth noting that reception theory originated in Hans Robert Jauss’ argument for a paradigmatic change in the study of literary history, his plea for paying critical attention to the historicity of interpretation or what he called Rezeptionsästhetik, which in turn owes a great deal to Hans-Georg Gadamer’s philosophical hermeneutics, particularly the concept of the ‘fusion of horizons.’”
A quick ctrl+F revealed that the final paragraph was lifted almost verbatim from an article by Zhang Longxi, published in the Journal of East-West Thought. Bing does add a footnote that leads to the article, but it doesn’t distinguish between direct quotes and material that paraphrases a source. (In some cases, the footnotes seem to have no clear connection to Bing’s output.)
Verdict: Helpful and informative, but a weird combination of vaguely sign-posted summary and borderline plagiarism.
Reading, fast and slow: Summarizing articles
Next, I wanted to see how Bing would do in summarizing a scholarly article and generating potential reading questions. I was especially curious to see how gen AI would fare in extracting main arguments from humanistic scholarship that often isn’t structured in as predictable a format as STEM articles tend to be. This lack of clear signposting, I find, often poses great challenges to undergraduate students. As such, I was wondering if gen AI could guide students in their reading.
I picked “Athenian Slave Names and Athenian Social History” by Kostas Vlassopoulos, partly because I wanted to read the article and partly because it laid out its main arguments more clearly and explicitly than your average Classics article. (To non-humanists: a lack of clarity is not an insult, as humanities thrives on nuance and complexities.)
I asked Bing to create questions students could consider before and while reading the article. Lesson one: Bing had no trouble accessing the text which is behind a paywall – one can hope Microsoft has a subscription! The “before” questions were adequate enough, as they aimed to invite student interest and hypothesizing. One example: “How do you think the study of slave names could contribute to our understanding of the role and function of slavery within Athenian society?” The reading questions were similarly useful – slightly vague, but they could walk students through identifying the main arguments, evidence used, limitations of the study, etc.
When I asked Bing to provide example answers, things got a bit weird. Like a student who’s done half of the reading, the responses seemed feasible on the surface but were a mixture of generalities and outright lies. That the names of enslaved people reflect “cultural and social dynamics of the time” is true to be sure, but it doesn’t capture Vlassopoulos’ observation about how the names of citizens and enslaved people overlapped considerably, with ramifications for, for example, how easy it would have been to distinguish people’s status. The answers mention pottery fragments as a source, but those aren’t mentioned in the article.
Guided by advice given in Teaching with AI, I decided to embrace roleplay and step into the role of a didact. I told Bing:
These sample answers do not correctly reflect the content of the article. Take a moment to rethink and try answering them again based on the article.
And lo and behold, the answers improved. The errors dropped off, and the responses were more specific. I still feel like I wandered into a disquieting uncanny valley when I told a machine to think more slowly and better, and it did, but there’s no denying the technique worked.
Verdict: Not very helpful, as it seems you need to check the answers for hallucinations.
Good thoughts or joyless answers? Specialized summary app
I also spent a bit of time dabbling in more specialized applications. Some seem more useful for STEM material – for example Explainpaper paraphrases any text you highlight, which I’d imagine is most helpful for text that is full of science vocabulary.
Humata, however, seemed potentially humanities-friendly – it even has a Zoroastrianism-inspired name (meaning “good thoughts,” apparently)! As ridiculous as it may be, however, the app filled me with bad thoughts instead of good ones.
The app is designed to quickly find answers from PDFs. It even highlights relevant parts of the text so you can (nominally) check where the app draws its information from. I uploaded the same article by Vlassopoulos. (I would apologize, but my Bing experiment showed AI sees all, so I can’t muster too much guilt over feeding Humata one PDF.) Within seconds, the app spat out a competent summary of the article along with suggested questions I could ask the app. And these weren’t vague “What do you think we can learn from slave names?” types of questions. Both the questions, and the answers I solicited from the app, were specific and detailed. The app is weirdly glitchy in that it highlights the wrong passages in the text, but the responses were all accurate and rooted in the actual article.
And here I was, looking at a machine that could mean I never have to annotate text again, painstakingly type notes into Zotero, nor skim through an article in search of the quote I swear I remember reading somewhere. Cue existential angst. Humata is efficient, adept, and completely removes the unique mixture of boredom, excitement, frustration, and joy that is seeing an argument unfold and take shape. It tells me the facts, but it removes the voice of the author and the shape of the argument.
Verdict: Extremely helpful, utterly terrifying.
Through outsourcing it, I’m beginning to understand how much joy there is in the work. I’ve seen some arguments about how letting AI design our course materials frees up more time for the “real work” of being in the classroom. But I enjoy designing courses: poking around online and flipping through books, collecting little tidbits that I might want to draw on during a lesson, envisioning what I’d like to see. With sophisticated enough prompts, AI can also skim research literature and identify the most relevant gems. But like many people in the humanities, I enjoy reading far and wide, whether each text directly addresses exactly what I need or not. Arguments about how AI work will become the new “C-level work” terrify me because from what I’ve seen, AI will soon provide arguments so eloquent and sophisticated that I won’t be able to improve on them – and it certainly already spits them out faster than I do. And yet, I like using my brain, flawed as it may be. Finally, I grieve the prospect of our students not having rich opportunities to do the work and find joy in it. In my conversations with students this coming year, I hope to move beyond concerns about academic integrity (which students seem quite concerned about) and instead talk with them about what is meaningful and, yes, joyful to them about learning.
Blog image generated using OpenAI’s DALL-E model after telling it what this blog post was about.
