All of you should have received a work request on Monday which is essentially the main project, you’ll be working on throughout this term. As you work through your blogposts and refine your main project, allow your topic to evolve - it could become more specific, broader, or even change completely. Remember, the goal is to gain new insights and see your topic in a different light by the end of the course. Here are the key timelines to keep in mind regarding your main project:
More detailed instructions regarding these timelines and the process involved will be shared either through the newsletter or admin updates in the coming weeks. Keep an eye out for those!
This week we will look at getting some general intuition into the complex topic of machine learning. A number of readings are assigned to this week, all quite technical (and many pivotal in the development of AI). My strong recommendation will be to watch the Grant Sanderson video – a master class in exposition from an expert in conveying visual intuitions around the mathematics of AI – and then download and scan or skim-read as many of these papers as possible. I would spend a little time on (a) Markov, (b) Mikolov et al., and (c) Vaswani et al, as each of these papers marks a critical conceptual as well as technical transition in the history of generative AI.
For people without technical backgrounds, do not worry if none of this makes any sense! While a proper understanding of deep learning would require many courses, we will be breaking things down in a way that hopefully provides some intuition in class.
Viewings / Readings:
*Highly recommended* Sanderson, G. (2024). Visualizing transformers and attention | Talk for TNG Big Tech Day ’24, https://www.youtube.com/watch?v=KJtZARuO3JY
Church, K. W., & Mercer, R. L. (1993). Introduction to the Special Issue on Computational Linguistics Using Large Corpora. Computational Linguistics, 19(1), 1-24. https://aclanthology.org/J93-1001/
Cope, B., & Kalantzis, M. (2023). Generative AI Comes to School (GPT and All That Fuss): What Now? Educational Philosophy and Theory, 13-17. https://doi.org/10.1080/00131857.2023.2213437
Markov, A. A. (2006). An example of statistical investigation of the text Eugene Onegin concerning the connection of samples in chains. Science in Context, 19(4), 591-600. https://alpha60.de/research/markov/DavidLink_AnExampleOfStatistical_MarkovTrans_2007.pdf
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26. https://arxiv.org/abs/1310.4546
Miller, G. A. (1995). WordNet: a lexical database for English. Communications of the ACM, 38(11), 39-41. https://dl.acm.org/doi/10.1145/219717.219748
Radford, A. (2018). Improving language understanding by generative pre-training. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379-423. https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017 [2023]). Attention Is All You Need. arXiv, 1706.03762. https://doi.org/10.48550/arXiv.1706.03762
Assignments:
- Contribute to the asynchronous conversation on course
materials.
- Blogpost 2: Updates on the main project (250 words - due Feb 10th)
Blogpost 2: As you research your topic, consider the connections between its underlying education theories and the broader landscape of AI in education. Share at least three scholarly sources, and reflect on how your ideas are evolving. Additionally, engage with at least two peers whose topics might intersect with yours.
For the asynchronous conversation here: After exploring the materials for this week, how do you think the development of attention mechanisms like those in Attention Is All You Need, has influenced current trends in AI? Reflect on how these concepts could shape the future of AI in education or other fields you’re interested in.