This week we turn from the comparatively benign world of learning and training to enter that of AI in the wild – and the many problems that emerge with aligning or counter-aligning behavior.
The focus for the week is the now-classic “Stochastic Parrots” paper by Emily Bender, Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell. We’ll spend some time talking this paper through, as it has influenced what has widely come to be known as “Critical AI” (and launched a journal of the same name).
I have contributed to two of the papers this week. The first, on truth, is partly a response to the Stochastic Parrots paper, and other conversations in relation to AI and hallucination. Working in part from Foucault’s discussion of *parrhesia*, discussed in a previous week, in that paper we sought to think about the different ways truth itself is understood, and how this might reflect upon our judgments of AI. In particular I want to unpack the example of “two plus two equals”.
In the second paper, we look at the question of alignment from an historical standpoint. We argue there that alignment is closely connected to quite fundamental questions of how people use language normatively. Alignment in other words relates to processes of control via language: not only in the obvious cases of censorship, but also in the intricate ways in which language marks and reproduces social power. We are perhaps today on the verge of seeing AI that can employ subtle linguistic cues to align us - a kind of counter-alignment or manipulation, as interesting case of Claude (Denison et al. 2024) shows.
If you’re stretched for time, I’d read the articles in that order (Bender et al, Munn et al, Hristova et al, Denison et al). We’ll spend most of the time in class with the Bender et al reading.
For those looking for further reading in Critical AI, I’ve included several other key texts, including a reading list that will be especially useful for those of you also looking to teach Critical AI.
In addition – if you’re looking for a question to think about ahead of class – try to think of a single sentence that would articulate all you could ever want from your “dream” aligned AI.
Readings:
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021, March). On the Dangers of Stochastic Parrots: Can Language Models be Too Big?��. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610-623). https://dl.acm.org/doi/10.1145/3442188.3445922
Denison, C., MacDiarmid, M., Barez, F., Duvenaud, D., Kravec, S., Marks, S., Schiefer, N., Soklaski, R., Tamkin, A., Kaplan, J., Shlegeris, B., Bowman, S. R., Perez, E., & Hubinger, E. (2024). Sycophancy to Subterfuge: Investigating reward-tampering in large language models. https://arxiv.org/html/2406.10162v2
Hristova, T., Magee, L., & Soldatic, K. (2024). The Problem of Alignment. AI & SOCIETY, 1-15. https://doi.org/10.1007/s00146-024-02039-2
Munn, L., Magee, L., & Arora, V. (2023). Truth Machines: Synthesizing Veracity in AI Language Models. AI and Society. https://doi.org/10.1007/s00146-023-01756-4
Wider Readings on Critical AI:
Noble, S. U. (2018). Algorithms of Oppression. New York University Press.
Ruha, B. (2019). Race After Technology: Abolitionist Tools for the New Jim Code. Polity Press.
Stiegler, B. (2013). Die Aufklärung in the age of philosophical engineering. In Digital Enlightenment Yearbook 2013 (pp. 29-39). IOS Press. https://www.iri.centrepompidou.fr/wp-content/uploads/2011/02/Stiegler-The-Aufklarung-x.pdf
“Teaching Critical AI Literacies” reading list: https://docs.google.com/document/d/1TAXqYGid8sQz8v1ngTLD1qZBx2rNKHeKn9mcfWbFzRQ/edit?tab=t.0#heading=h.kgds7i8l6uca
Assignment(s):