Quaternion Knowledge Graph Embeddings (2019)

[vslira]

I'm not nearly smart enough to state this confidently, but doesn't [1] imply that exotic embeddings can always be replaced by larger Euclidean embeddings?

[1] https://en.wikipedia.org/wiki/Nash_embedding_theorems

[hansvm]

No.

Consider a square (with graph shortest path distances) for example. All euclidean embeddings have a minimum error of 20-40% or so. If you try to embed any graph containing that subgraph (embedding friends, advertisers, words, ...), you'll similarly have guaranteed error. Relaxing the square to a squircle, you'll see that error even for a really simple manifold.

Riemannian manifolds are a bit more special.

[palad1n]

Is anyone using this? It's from 2019.

[VHRanger]

These exotic (non linear algebra based) embedding representations are often slow to take off unless they have an obvious use case.

The other one that I've always been curious of is Poincarré Embeddings [1] - where the embedding also has a hierarchical representation of the space.

There's issues with these becoming popular:

1. Querying the embeddings requires more math knowledge than just "lol cosine similarity". This also requires to write code for the query

2. You can often easily match the performance with regular embeddings by just adding dimensions and training more. So the advantage of exotic embeddings has to be with the information in the more complex mathematical abstraction.

So they need a killer usecase to become popular, it's hard to move the needle.

[1]: https://arxiv.org/abs/1705.08039

[adultSwim]

QuatE is implemented in PyKEEN, a library for KnowEdge graph EmbeddiNgs, https://github.com/pykeen/pykeen

[adultSwim]

The paper was cited over 500 times, https://scholar.google.com/scholar?cites=1505183897130217150...

[cess11]

If someone is it's probably pretty niche. RDF is a bit simpler and still not commonly used due to its perceived complexity compared to RDBMS tables or JSON.

[ziofill]

I always wondered why bother with quaternions and abstruse math when at the end of the day they behave like some matrices of floats.

[barfbagginus]

I recently took the plunge and learned unit quaternions and dual quaternions for representing rotations, compound rotation-translations, and kinematic chains and geometric interpolation (serial and parallel robot arms, character skeletons, skinning).

The advantages I found are:

Unit quaternions represent rotations with less redundancy than matrices

It's easier and more intuitive to derive, manipulate, simplify, interpolate, and solve quaternions than matrices.

They're less abstract/more concrete than matrices.

Rotating with quaternions takes fewer multiplications than matrices.

Dual quaternion inverse kinematics are easier to derive and faster than matrices

Unit and Dual quaternions have an efficient implicit form, which further speeds up IK. See: https://www.researchgate.net/profile/Neil-Dantam/publication...

[godelski]

The problem is most people were taught math wrong and this typically only gets corrected when you get to high level mathematics like abstract algebra.

Imaginary numbers aren't "imaginary" they are how to do consistent math in a 2D framework. Quaternions? Well that's about 4D. Poincaré once said that math isn't about numbers, but relationships between numbers. For some reason we don't talk about mathematical structure (explicitly) until the late stages. I wouldn't necessarily call these concepts "abstruse" but they are a bit more abstract. A big part of the problem though, we often ignore the ground we are building upon and so when you finally look at it, it is new and confusing. But then again, the success of Bourbaki's New Math is arguable[0]

[0] https://en.wikipedia.org/wiki/New_Math#In_other_countries

[hi-v-rocknroll]

For 3D graphics, it makes rotation and translation simpler by adding another dimension. Maybe you had a less-than-stellar teacher for this topic. If so, here's a master lecturer elucidating it:

https://youtu.be/mHVwd8gYLnI

https://en.wikipedia.org/wiki/Transformation_matrix#Examples...

[somenameforme]

One of the big motivations for quaternions is performance. Not only are they much more compact than a rotation matrix (4 float vs 9), but many important operations, such as composing rotations, are also much quicker. The only real downside of them is that they seem to drive everybody to this obsession to try to truly understand them. After years of using them, I've learned to embrace my ignorance. The underlying math is not terribly complex (hur hur), but relationships between interdependent periodic rotations all packed in a tidy imaginary number package, is simply not something you're going to really get a good intuitive feel for - numerically speaking.

[authorfly]

Because they enable you to involve things typically unrelated (i.e. two types of embeddings), think and understand and manipulate them in a simpler mathematical model (a single vector) and see how you can then apply the most fundamental mathematical operations (e.g. cosine similarity) in terms of simpler outputs/metrics. It enables a form of creativity in other words; it makes you see things like the math in Transformers in a less anxious light as you understand how these things can overlay into a simpler intuitive understanding.

[pinkmuffinere]

As a concrete example, feedback controllers for attitude control (ie, pointing something) are imo easier to develop in quarternions than with 3x3 matrices. For one, the quaternion formulation isn’t over-constrained by the extra parameters.

This is essentially because quaternions are a remarkably good representation of rotations.

[meindnoch]

Rotation matrices (i.e. orthonormal matrices) are a complicated manifold in 9 dimensional space.

Rotation quaternions (i.e. unit quaternions) are simply the unit sphere in 4 dimensional space.

[truckerbill]

'Why bother with programming languages when at the end of the day they are just pages of text'

[pizza]

Simplicity depends on the observer