# Future directions from the GEO-AI critique
This note distills ideas from Jose Crespo's GEO-AI articles that are directly
relevant to the GeoGraphDB experiment stack, and proposes concrete experiments
to test them. The goal is not to build a general theory of cognition, but to
keep the project aligned with the geometric hypothesis: *structure should do
part of the work before the model starts burning data.*
---
## 1. The minimum transport stack: metric, connection, holonomy
Crespo's minimum GEO-AI stack is:
1. **Metric** — a local ruler that says what is near or far in context.
2. **Connection** — a rule for moving meaning through curved terrain without
turning it into coordinate soup.
3. **Holonomy** — a memory of the path, so the same word can return with a
different orientation.
What we already have:
| Metric | `spatial::octree`, `ricci::ollivier_ricci` | L2 proximity + curvature as a local distortion measure. |
| Connection | `parallel_transport::rodrigues_rotation`, `transport_score` | Carries a tangent vector across an edge via a Rodrigues rotation. |
| Holonomy | `parallel_transport::transport_angle`, closed-loop detection | Angle between incoming/outgoing tangents records path-dependent rotation. |
### Proposed experiments
- **Holonomy regression**: measure the transport angle accumulated around
closed loops in real corpus graphs. Does the magnitude correlate with
semantic drift (e.g., the same token used in two different domains)?
- **Loop-aware walker**: penalize or reward trajectories based on cumulative
holonomy. The hypothesis is that paths with large holonomy have crossed a
semantic boundary and should be rejected or flagged.
- **Contextual metric field**: instead of a single distance threshold, learn a
per-node or per-region scaling factor for the metric from training data.
---
## 2. Tropical algebra and the inability to undo
The critique is that flat AI uses selection operators (`max`, `argmax`, `top-k`,
`top-p`, routing masks) that behave like tropical addition: the winner is kept,
the losers are erased, and there is no algebraic inverse to recover the
rejected branches. Even `log-sum-exp` only keeps recoverability in the
two-branch idealization; in a real branching storm the inverse becomes
intractable.
What we already have:
- The walker uses `softmax` sampling and `top-k` candidate selection, so it is
not immune to the tropical problem.
- Beam search is planned in `docs/structure-decoder.md`.
### Proposed experiments
- **Branch-preserving decoder**: keep a tree of candidate trajectories rather
than a single winner. Each leaf carries its full path, score, and holonomy.
Compare final output quality against greedy decoding.
- **Tropical baseline**: implement a deliberately flat baseline that uses only
`argmax` at every step. Quantify how often it gets stuck in loops or dead
ends compared to the geometric walker.
- **Recoverable routing**: instead of hard top-k pruning, maintain a soft
distribution over candidates and re-normalize at each step. Measure whether
this reduces catastrophic branch loss.
---
## 3. Fiber bundles and quotient by consequence
The fiber-bundle picture: a base point (the observable fingerprint) with many
possible meanings above it (the fiber). A flat model tries to resolve the
fiber, which is expensive and often impossible. A geometric model collapses the
fiber by *consequence*: different wordings that trigger the same action are
quotiented together.
What we already have:
- The graph already encodes a discrete fiber: `node.id = token_id * 1000 +
sense_index`. Many senses (fiber points) map to one token (base point).
- `corpus_native_graph_wsd.rs` performs word-sense disambiguation by picking a
sense from the fiber using context.
### Proposed experiments
- **Quotient walk by action**: add action labels to nodes (e.g., `payout`,
`audit`, `alert`) and train the walker to converge on the action, not the
exact wording. This is a discrete version of "quotient by consequence."
- **Sense-fiber visualization**: project all senses of a single token onto the
manifold and measure their spread. High spread = high polysemy = fat fiber.
- **Fiber-aware retrieval**: when a prompt matches multiple senses, do not
collapse to one centroid. Run parallel walkers, one per sense, and select the
one whose trajectory stays coherent longest.
---
## 4. Fisher-Rao metric and belief as geometry
Crespo emphasizes that Fisher and Rao proved there is a canonical metric on the
space of probability distributions. Belief is not a sentence; it is a point on
a probability simplex that moves as evidence arrives.
What we already have:
- `algorithms::natural_grad` implements softmax, diagonal Fisher information,
KL divergence, and Fisher-Rao distance.
- `algorithms::infogeo` has information-geometry primitives.
### Proposed experiments
- **Belief-simplex walker**: represent the walker state as a point on the
simplex over candidate next nodes. Update it with Bayesian evidence (edge
weight, curvature, transport score) instead of sampling from a softmax.
Visualize the trajectory on the 2-simplex for small candidate sets.
- **Fisher-Rao step sizing**: use `fisher_rao_dist` to measure how far a
proposed step moves the walker's belief distribution. Reject steps that move
too far in information geometry even if they look good in Euclidean space.
- **Natural-gradient coordinate branch**: replace SGD/Adam in
`CoordinateBranch` and `MlpClassifier` with the natural gradient
preconditioner from `natural_grad.rs`. Compare convergence on the same graph.
---
## 5. Torus and solenoid memory
The "tiny thinking engine" proposal uses a torus as a wrapped address space and
a solenoid as a nested hierarchy. The idea is that memory and hierarchy should
be topology, not a hand-built ontology.
What we already have:
- The graph is a discrete manifold, not a flat list. Closed paths and
neighborhoods are native.
- `storage::sectioned_graph` and domain labels provide a crude hierarchy.
### Proposed experiments
- **Torus embedding baseline**: train a small coordinate branch to map token
positions onto a 3D torus instead of R³. Measure whether periodic boundaries
improve recall of cyclic concepts (days of the week, months, code loops).
- **Solenoid hierarchy**: add multi-scale edges to the graph: short edges for
fine-grained neighborhoods, long edges for category-level jumps. Test whether
the walker can navigate both levels without explicit category labels.
- **Return-friendly metric**: implement a distance function that rewards
returning to previously visited high-value regions, making "memory" a
geometric property of the space rather than a separate buffer.
---
## 6. Triangular attention and diffusive decoding
AlphaFold 3 uses triangular attention and diffusion in 3D coordinate space. The
lesson is that triples, not pairs, can constrain geometric structure more
strongly.
What we already have:
- `parallel_transport` uses triples `(prev, curr, next)` to compute transport
angles.
- The walker scores candidates one at a time; there is no explicit triangular
constraint.
### Proposed experiments
- **Triangular transport score**: score a candidate not just by the angle
`prev → curr → next`, but by how well the triangle `(prev, curr, next)`
matches the local curvature (via Ollivier-Ricci). Reject candidates that
create degenerate or over-curved triangles.
- **Diffusion-style refinement**: start the walker at a noisy position and
iteratively project it onto the manifold defined by the graph. Compare
convergence to the current greedy approach.
- **3-coordinate prediction**: extend `CoordinateBranch` to predict not just
(x, y, z) but also a local frame or curvature estimate, so the model outputs
more geometric structure per token.
---
## 7. Active inference and evidence coupling
VERSES-style active inference treats AI as belief updating under uncertainty.
The walker already has uncertainty (temperature, softmax), but it does not
maintain a explicit posterior over hypotheses.
### Proposed experiments
- **Bayesian walker**: maintain a posterior over a small set of hypotheses
(e.g., "the user wants a tool call", "the user wants a definition"). Each
step updates the posterior using geometric evidence. The walker's behavior
switches only when the posterior crosses a threshold.
- **Uncertainty-aware planning**: use the walker's entropy as a signal to
trigger A* planning or context retrieval. High entropy = the metric is
locally flat or ambiguous = fetch more context before deciding.
- **Free-energy path selection**: prefer paths that reduce the surprise between
expected and observed neighbor distributions, approximating active inference
on the graph.
---
## 8. Tropical geometry of ReLU networks (Zhang, Naitzat & Lim, 2018)
The paper *Tropical Geometry of Deep Neural Networks* (ICML 2018) proves that a
feedforward ReLU network with integer weights is exactly a **tropical rational
map**: a difference of two tropical polynomials, or equivalently a tropical
quotient `f ⊘ g` in the max-plus semiring.
Key takeaways for this project:
- **ReLU = max-plus algebra.** Tropical addition is `x ⊕ y = max(x, y)` and
tropical multiplication is `x ⊙ y = x + y`. A ReLU layer `max(Ax + b, 0)` is
already a tropical operation. This is the formal backbone of Crespo's claim
that current AI "cannot subtract": tropical addition discards the loser, and
the operation has no algebraic inverse.
- **Linear regions = vertices of polytopes.** The number of linear regions of a
ReLU network is bounded by the number of vertices of the polytopes associated
with its tropical rational representation. Deeper networks produce more
linear regions through Minkowski sums of layer polytopes, which is why depth
is exponentially expressive.
- **Decision boundaries = tropical hypersurfaces.** The set where two monomials
of a tropical polynomial attain the same maximum forms a tropical
hypersurface. Classifier boundaries therefore live in a well-studied geometric
object, not an opaque high-dimensional soup.
- **Zonotopes for shallow networks.** A one-hidden-layer ReLU network is
characterized by a zonotope. For deeper networks, Minkowski sums of those
zonotopes produce the richer polytope structure.
### Why this matters for GeoGraphDB
Our `MlpClassifier` is a small ReLU network. The tropical picture says that
every `max` and `argmax` inside it, and every `softmax` temperature-collapse in
the walker, is a point where information is discarded without an algebraic undo.
The geometric graph side of the project is an attempt to move some of that lost
structure out of the network and into the data structure (curvature, transport,
holonomy), so the model does not have to pay the tropical tax at every step.
### Proposed experiments
- **Tropical decomposition of `MlpClassifier`**: given a trained `MlpClassifier`,
decompose each ReLU layer into tropical polynomials `F` and `G` and visualize
the associated Newton polygon / dual subdivision for the 2D toy problems
(e.g., XOR). Count linear regions and compare with the network's actual
behavior.
- **Linear-region growth vs. graph curvature**: train `MlpClassifier` heads of
varying depth on the same corpus graph and correlate the empirical number of
linear regions (estimated on a grid) with the Ollivier-Ricci curvature of the
training graph. Does more curved training data demand more linear regions?
- **Tropical decision boundary visualization**: for the 2-community geometric
classifier in `train_geometric.rs`, plot the tropical hypersurface that
separates the classes and compare it to a flat linear classifier.
- **Zonotope probe for shallow heads**: implement a small tool that, given a
one-hidden-layer `MlpClassifier`, extracts its zonotope generators and reports
the number of vertices on the upper envelope. This gives a rigorous
complexity measure instead of parameter count.
---
## 9. Summary: what to build next
The most immediately testable ideas, ordered by how little new infrastructure
they require:
1. **Natural-gradient training** for `CoordinateBranch` / `MlpClassifier`.
2. **Holonomy measurement** on closed loops in real corpus graphs.
3. **Branch-preserving beam decoder** with full path memory.
4. **Belief-simplex walker** for small candidate sets.
5. **Tropical flat baseline** to quantify the cost of `argmax`-only routing.
6. **Quotient-by-action walk** using labeled tool-call outcomes.
7. **Triangular transport score** using Ricci curvature.
8. **Tropical decomposition / linear-region count** for `MlpClassifier`, using
the Zhang et al. framework.
Each of these produces a concrete number: Fisher-Rao distance, holonomy angle,
beam-recall gain, entropy reduction, or action accuracy. That keeps the
project empirical and avoids the trap of building geometry as philosophy
instead of as a testable mechanism.