I have recently written a paper on understanding machine learning via the lens of Hopf algebra https://arxiv.org/abs/2302.01834.
Hopf algebras (which are really just tensors with recurrence relations built in) subsume convnets, transformers and diffusion model and also provide a theoretically better autodiff that operates within single layers as opposed to across entire graphs.
Furthermore, there is a correspondence between Hopf algebra and cyclical linear logic and Hopf algebras are related to zonotopes, which are polyhedra that have been used in verified numerical computation. I'm strongly convinced the LL connection can provide proofs over zonotopes which paves the way towards interpretable AI and will be central for XAI.
I know this sounds too good to be true but Persi Diaconis has also written a paper that shows how useful Hopf algebras are in the context of Markov chains https://arxiv.org/abs/1206.3620
I'm working on a next gen Hopf algebra based machine learning framework.
Your paper didn't pass my smell test at all, tbh. For example the formula you write about "product" and "coproduct" in section 3 is literally identical (as "=" is symmetric). In section 4.2 you write "the product is the standard tensor product" with a formula that doesn't at all involve the map m: A \otimes A \to A. The formula you write is the induced product on A \otimes A, assuming that you already have a product on A. The formula for "coproduct" is just an example[1] of a coproduct, not every coproduct has to look that way.
I just read your Coinductive guide to inductive transformer heads paper.
My mind is blown.
Is the Hopf Algebra based ML framework you are working on on your github? I took a glance, but you have 1500 repositories and it wasn't on the first few of them.
It is tempting to believe that category theory will shed new light on and simplify machine learning, just like it did in algebraic geometry, algebraic topology and other mathematical things. This is wishful thinking. Folks who care about doing something useful should stay away from this content.
Application in the sense they are using it is probably different than the sense you are using it. Although its still probably a fair question regardless.
The exponential growth in CS publication is much faster.
This repository is simply a testament that CT is slowly ramping up.
It's meant to show what kind of expressive power and breadth current CT models have, which to my knowledge isn't something that's well-known outside of our niche community.
I'm not experienced/well-read in either ML or CT, but awhile ago I remember hearing Tai-Danae Bradley equate "knowing a word by the company it keeps" to the Yoneda lemma, and I always thought that was kind of interesting (although I guess I'm not qualified enough to know whether that statement is useful or vacuous)
I'm still shocked no one has developed language learning software along these lines. I had a prototype in the works for thai years ago but never got time to get it off the ground. using statistical models trained on web corpus for a language learning app seems like a no brainer.
think of it like navigating a word as a point in a graph connected to every example context it is in, with associated words being clickable into similar context bundles. then make it differential between host and target language given a translation so you can see which contexts the translation fails and succeeds in.
category theory is 'native 2-dimensional' math. i.e. category theory explains everything in terms of graphs, where a graph is made from two different sorts of 'entities', nodes and vertices i.e. categories and morphisms
this being math, I wonder to which extent can category theory be re-expressed in terms of sets.
perhaps a better question is if category theory can be re-expressed (or founded on) functions?
lastly, I wonder if category theory can be expressed in terms of functions (i think maybe it can, without sets?) why shouldn't it be expressible in terms of sets (for some reason I don't think just sets are sufficient, may have to define functions (which possible in terms of sets) before 'expressing' categories starting with set theory)?
Set theory is fine, see the (Stack Project)[https://stacks.math.columbia.edu/browse] which develops a ton of modern Category Theory on ZFC (Zermelo-Fraenkel Set Theory with the Axiom of Choice) alone.
Alternative foundations of mathematics (Set Theory, Category Theory, Type Theory, and all their variations) can all mutually interpret the other by just postulating sufficiently large universes. You don't pick or advocate one based off its ability to encode mathematics, but instead based on its ability to express your intention and ideas.
Really its no different from programming language preference in my book.
In computer science we typically count from zero, not from one. So even though a category has two different sorts of entities: objects and morphisms, since we count from zero, ordinary categories are one-dimensional. Objects are zero dimensional and morphisms are one dimensional.
Since you are concerned with sets and functions, the following analogy is helpful to build your intuition for the subject.
Dimension 0: sets
Dimension 1: functions
Dimension 2: commutative squares
Dimension 3: commutative cubes
The majority of categories, expressed in terms of structured sets and functions, never touch on the second dimension. That is the domain of 2-categories, which although they have three types of elements are nonetheless considered to be two dimensional because we count from zero. That is also where commutative squares come in to play, because as you can imagine squares are quite obviously two dimensional.
Functions are 1-dimensional lines or arrows from one place to another. Sets are more analogous to points then anything else, and so naive set theory is zero dimensional. But I think you have the wrong question. You should ask the opposite question: what if functions can be re-expressed or founded on higher dimensional category theory?
Set theory is going the way of the dodo. There are modified versions of set theory, but afaik a lot of the more exciting work is happening around type theory and proof assistants these days.
adamnemecek|3 years ago
Hopf algebras (which are really just tensors with recurrence relations built in) subsume convnets, transformers and diffusion model and also provide a theoretically better autodiff that operates within single layers as opposed to across entire graphs.
Furthermore, there is a correspondence between Hopf algebra and cyclical linear logic and Hopf algebras are related to zonotopes, which are polyhedra that have been used in verified numerical computation. I'm strongly convinced the LL connection can provide proofs over zonotopes which paves the way towards interpretable AI and will be central for XAI.
I know this sounds too good to be true but Persi Diaconis has also written a paper that shows how useful Hopf algebras are in the context of Markov chains https://arxiv.org/abs/1206.3620
I'm working on a next gen Hopf algebra based machine learning framework.
Join my discord if you want to discuss this further https://discord.cofunctional.ai.
====
My account is currently rate limited so I will use this comment to respond to comments below.
red_trumped: What about Hopf algebras do I not understand?
gaze: Haha, it's been a while since I have commented about QC. What do I not understand about it? And what comment are you referring to?
red_trumpet|3 years ago
[1] https://en.wikipedia.org/wiki/Coalgebra#Examples
gexaha|3 years ago
nicwilson|3 years ago
https://arxiv.org/abs/2302.01834 appears to have a typo in section 4.5
S(hg) = S(g)S(g)
looks like it should be S(hg) = S(h)S(g) or S(hg) = S(g)S(h)
ianandrich|3 years ago
My mind is blown.
Is the Hopf Algebra based ML framework you are working on on your github? I took a glance, but you have 1500 repositories and it wasn't on the first few of them.
unknown|3 years ago
[deleted]
umutisik|3 years ago
lgas|3 years ago
epgui|3 years ago
AlexCoventry|3 years ago
What's the most compelling application so far?
bawolff|3 years ago
rmdamiao|3 years ago
bgavran|3 years ago
The exponential growth in CS publication is much faster. This repository is simply a testament that CT is slowly ramping up.
It's meant to show what kind of expressive power and breadth current CT models have, which to my knowledge isn't something that's well-known outside of our niche community.
adamnemecek|3 years ago
eigenform|3 years ago
haskellandchill|3 years ago
I'm still shocked no one has developed language learning software along these lines. I had a prototype in the works for thai years ago but never got time to get it off the ground. using statistical models trained on web corpus for a language learning app seems like a no brainer.
think of it like navigating a word as a point in a graph connected to every example context it is in, with associated words being clickable into similar context bundles. then make it differential between host and target language given a translation so you can see which contexts the translation fails and succeeds in.
donnowhy|3 years ago
this being math, I wonder to which extent can category theory be re-expressed in terms of sets.
perhaps a better question is if category theory can be re-expressed (or founded on) functions?
lastly, I wonder if category theory can be expressed in terms of functions (i think maybe it can, without sets?) why shouldn't it be expressible in terms of sets (for some reason I don't think just sets are sufficient, may have to define functions (which possible in terms of sets) before 'expressing' categories starting with set theory)?
voxl|3 years ago
Alternative foundations of mathematics (Set Theory, Category Theory, Type Theory, and all their variations) can all mutually interpret the other by just postulating sufficiently large universes. You don't pick or advocate one based off its ability to encode mathematics, but instead based on its ability to express your intention and ideas.
Really its no different from programming language preference in my book.
johnbernier|3 years ago
Since you are concerned with sets and functions, the following analogy is helpful to build your intuition for the subject.
Dimension 0: sets
Dimension 1: functions
Dimension 2: commutative squares
Dimension 3: commutative cubes
The majority of categories, expressed in terms of structured sets and functions, never touch on the second dimension. That is the domain of 2-categories, which although they have three types of elements are nonetheless considered to be two dimensional because we count from zero. That is also where commutative squares come in to play, because as you can imagine squares are quite obviously two dimensional.
Functions are 1-dimensional lines or arrows from one place to another. Sets are more analogous to points then anything else, and so naive set theory is zero dimensional. But I think you have the wrong question. You should ask the opposite question: what if functions can be re-expressed or founded on higher dimensional category theory?
mydogcanpurr|3 years ago
The reason you're looking for is that the category of sets is not a set.
Koshkin|3 years ago
Objects, not categories.
epgui|3 years ago
justatdotin|3 years ago
yoneda