FluxEM: Algebraic Embeddings for Deterministic Neural Computation
Abstract
Neural networks often struggle with arithmetic because they treat numbers as arbitrary tokens. FluxEM maps arithmetic operations to geometry: addition becomes vector addition, and multiplication becomes addition in log-space. The result is systematic generalization from algebraic structure without learned parameters, with accuracy bounded by floating-point precision.
1. Problem
Large language models can describe calculus but still fail on arithmetic like
1847 × 392. Prior work (NALU, xVal, Abacus) focuses on training
models to learn arithmetic more reliably.
The issue is representation: embed numbers as arbitrary token vectors and arithmetic relationships must be inferred from data, which often fails to generalize beyond the training distribution.
2. Approach
Encode numbers so arithmetic operations are geometric operations in embedding space.
For addition, embed numbers along a fixed direction. Vector addition in embedding space equals arithmetic addition:
For multiplication, embed in log space. Addition in log space corresponds to multiplication in linear space:
These are algebraic identities over the reals for the magnitude component; under IEEE-754, they hold within floating-point precision bounds.
3. Formal Definition
3.1 Linear Embedding
For addition and subtraction:
where v ∈ ℝd is a fixed unit vector. The homomorphism property e_lin(a) + e_lin(b) = e_lin(a + b) follows directly from linearity.
3.2 Logarithmic Embedding
For multiplication and division:
where v_mag and v_sign are orthogonal unit vectors. Magnitude and sign are tracked separately, and the sign component is recombined in the operator definition.
Zero is handled explicitly outside the log embedding via a masking branch.
3.3 Embedding Layout
FluxEM uses a unified 128-dimensional embedding format:
| Indices | Purpose |
|---|---|
| [0:8] | Domain tag (identifies encoder) |
| [8:72] | Domain-specific content (64 dims) |
| [72:96] | Shared semantic features (24 dims) |
| [96:128] | Cross-domain composition (32 dims) |
4. Domains
FluxEM is not a single numeric embedding, but a family of deterministic encoders that share a common 128-dimensional layout. The first 8 dimensions identify the domain; the remaining regions store domain-specific content and optional cross-domain features.
The current implementation includes eleven domains. Each domain is designed around an algebra (or a small set of algebraic invariants) so that basic operations correspond to predictable geometric operations in embedding space.
| Domain | Examples | Encodes / supports |
|---|---|---|
| Physics | 9.8 m/s², [M L T^-2] | Units, dimensions, quantities, constants; conversion and dimensional analysis |
| Chemistry | H2O, 2H2 + O2 → 2H2O | Elements, formulas, bonds, reactions; stoichiometry and mass-balance structure |
| Biology | ATGCCGTAG, proteins | DNA/RNA/protein sequences, pathways, taxonomy; simple sequence statistics and structure |
| Mathematics | 3 + 4i, matrices | Reals, complex numbers, rationals, vectors, matrices, polynomials; arithmetic closed under float precision |
| Logic | p ∧ q → r | Propositional and predicate logic, type structure; satisfiability-oriented features |
| Music | {0,4,7} | Pitch, chords, scales, rhythm, atonal set theory; transposition/inversion (TnI) structure |
| Geometry | points, triangles | Points/vectors, transforms, shapes; distances and geometric invariants (e.g. area, centroid) |
| Graphs | G=(V,E) | Directed/undirected graphs; adjacency structure, connectivity, cycles, shortest paths |
| Sets | A ∪ B | Finite sets, relations, functions; union/intersection/composition-style operations |
| Number Theory | 360 = 2³·3²·5 | Integers, primes, modular arithmetic; factorization and arithmetic invariants |
| Data | arrays, tables | Arrays, records, tables; typed structure and summary statistics |
5. Precision
The magnitude homomorphism is defined over the reals. Under IEEE-754 float32/float64, precision is bounded by log()/exp() function rounding.
| Operation | Relative Error (float32) |
|---|---|
| Addition / Subtraction | < 1e-7 |
| Multiplication / Division | < 1e-6 |
"Closed under operations" means the result of any supported operation on valid inputs is a valid output, not that results are mathematically exact.
6. Limitations
| Case | Behavior |
|---|---|
| Zero | Explicit flag; log(0) masked to zero vector |
| Sign | Tracked separately from magnitude |
| Negative base + fractional exponent | Unsupported; returns real-valued surrogate |
| Division by zero | Returns signed infinity |
7. Usage
pip install fluxem
from fluxem import create_unified_model
model = create_unified_model()
model.compute("1847 * 392") # → 724024.0
model.compute("123456 + 789") # → 124245.0
model.compute("2 ** 16") # → 65536.0
8. Related Work
| Approach | Method | Difference |
|---|---|---|
| NALU (Trask 2018) | Learned log/exp gates | FluxEM: no learned parameters |
| xVal (Golkar 2023) | Learned scaling direction | FluxEM: deterministic, multi-domain |
| Abacus (McLeish 2024) | Positional digit encoding | FluxEM: algebraic structure, not positional |
References
- Euler, L. (1739). Tentamen novae theoriae musicae.
- Lewin, D. (1987). Generalized Musical Intervals and Transformations. Yale University Press.
- Trask, A. et al. (2018). Neural Arithmetic Logic Units. NeurIPS.
- Golkar, S. et al. (2023). xVal: A Continuous Number Encoding. arXiv.
- McLeish, S. et al. (2024). Transformers Can Do Arithmetic with the Right Embeddings. arXiv.