Tai-Danae Bradley | Category Theory and Language Models

Tai-Danae Bradley is a mathematician who received her Ph.D. in mathematics from the CUNY Graduate Center. She was formerly at Alphabet and is now at Sandbox AQ, a startup focused on combining machine learning and quantum physics. Tai-Danae is a visiting research professor of mathematics at The Master’s University and the executive director of the Math3ma Institute, where she hosts her popular blog on category theory. She is also a co-author of the textbook Topology: A Categorical Approach that presents basic topology from the modern perspective of category theory.
In this episode, we provide a compressed crash course in category theory. We provide definitions and plenty of basic examples for all the basic notions: objects, morphisms, categories, functors, natural transformations. We also discuss the first basic result in category theory which is the Yoneda Lemma. We conclude with a discussion of how Tai-Danae has used category-theoretic methods in her work on language modeling, in particular, in how the passing from syntax to semantics can be realized through category-theoretic notions.
Patreon: https://www.patreon.com/timothynguyen
Originally published on July 20, 2022 on YouTube: https://youtu.be/Gz8W1r90olc
Timestamps:
00:00:00 : Introduction
00:03:07 : How did you get into category theory?
00:06:29 : Outline of podcast
00:09:21 : Motivating category theory
00:11:35 : Analogy: Object Oriented Programming
00:12:32 : Definition of category
00:18:50 : Example: Category of sets
00:20:17 : Example: Matrix category
00:25:45 : Example: Preordered set (poset) is a category
00:33:43 : Example: Category of finite-dimensional vector spaces
00:37:46 : Forgetful functor
00:39:15 : Fruity example of forgetful functor: Forget race, gender, we're all part of humanity!
00:40:06 : Definition of functor
00:42:01 : Example: API change between programming languages is a functor
00:44:23 : Example: Groups, group homomorphisms are categories and functors
00:47:33 : Resume definition of functor
00:49:14 : Example: Functor between poset categories = order-preserving function
00:52:28 : Hom Functors. Things are getting meta (no not the tech company)
00:57:27 : Category theory is beautiful because of its rigidity
01:00:54 : Contravariant functor
01:03:23 : Definition: Presheaf
01:04:04 : Why are things meta? Arrows, arrows between arrows, categories of categories, ad infinitum.
01:07:38 : Probing a space with maps (prelude to Yoneda Lemma)
01:12:10 : Algebraic topology motivated category theory
01:15:44 : Definition: Natural transformation
01:19:21 : Example: Indexing category
01:21:54 : Example: Change of currency as natural transformation
01:25:35 : Isomorphism and natural isomorphism
01:27:34 : Notion of isomorphism in different categories
01:30:00 : Yoneda Lemma
01:33:46 : Example for Yoneda Lemma: Identity functor and evaluation natural transformation
01:42:33 : Analogy between Yoneda Lemma and linear algebra
01:46:06 : Corollary of Yoneda Lemma: Isomorphism of objects = Isomorphism of hom functors
01:50:40 : Yoneda embedding is fully faithful. Reasoning about this.
01:55:15 : Language Category
02:03:10 : Tai-Danae's paper: "An enriched category theory of language: from syntax to semantics"


Further Reading:
Tai-Danae's Blog: https://www.math3ma.com/categories
Tai-Danae Bradley. "What is applied category theory?" https://arxiv.org/pdf/1809.05923.pdf
Tai-Danae Bradley, John Terilla, Yiannis Vlassopoulos. "An enriched category theory of language: from syntax to semantics." https://arxiv.org/pdf/2106.07890.pdf