Data field theory: a geometric framework for learning on Riemannian manifolds with synthetic validation and limitation analysis

Abstract

Introduction:

Conventional machine learning treats learning as parameter optimization, lacking a first-principles framework for phenomena like criticality, generalization, and causal structure. We introduce Data Field Theory (DFT), a mathematical framework modelling learning as the evolution of a data field governed by stochastic partial differential equations on Riemannian manifolds. This work aims to validate DFT's core predictions in settings where its geometric assumptions hold, while honestly assessing its empirical limitations.

Methods:

We formulate learning as a field evolving on a spherical manifold. To test DFT, we implement a hierarchical classification task using synthetic data drawn from von Mises-Fisher distributions, ensuring match with the manifold geometry. We derive four key predictions: (1) critical exponents near concept formation, (2) a spectral robustness law linking Eigen gaps to out-of-distribution (OOD) error, (3) finite-speed causal propagation from hyperbolic regularization, and (4) approximate rotational equivariance via a Ward identity. We also conduct a preliminary real-data experiment projecting MNIST digits onto the sphere.

Results:

Synthetic experiments validate all four predictions: (1) Correlation length diverges as with ν = 0.63 ± 0.04, accompanied by 1/f fluctuations; (2) OOD generalization error scales as (ρ = −0.78, p < 10⁻⁶); (3) Causal propagation speed c_eff = 0.98 ± 0.03 (theory maximum c_max = 1.0) under hyperbolic regularization; and (4) Ward identity residual R = 0.0032 ± 0.0008 converging as R∝h^1.02. However, on real-world MNIST-sphere data, DFT achieves only 15.7% accuracy versus 51.7% for k-NN, revealing critical limitations.

Discussion:

DFT successfully predicts emergent phenomena criticality, spectral robustness, bounded causality, and approximate equivariance under ideal geometric conditions, supporting its theoretical validity. The poor real-data performance highlights key gaps: the current framework lacks adaptive metric learning, noise robustness, and hierarchical feature extraction present in real images. These results establish DFT as a principled mathematical foundation for learning as field dynamics while clearly delineating necessary extensions for practical applicability.

1 Introduction

Modern artificial intelligence has achieved remarkable empirical success through deep learning architectures with billions of parameters. However, this success has largely been heuristic, characterized by extensive trial-and-error rather than first principles. The dominant paradigm formulates learning as high-dimensional parameter optimization: , which, while computationally powerful, lacks the theoretical coherence and predictive laws characteristic of mature scientific theories.

1.1 Related work and novel contributions

Our work builds upon several research traditions while making distinct contributions. Geometric deep learning explores neural architectures that respect underlying data symmetries (Bronstein et al., 2021). Neural field theories model cortical dynamics as continuous fields (Coombes et al., 2014), while the critical brain hypothesis suggests neural systems operate near phase transitions (Chialvo, 2010; Beggs and Plenz, 2003). Hyperbolic embeddings leverage negative curvature for hierarchical representations (Nickel and Kiela, 2017). Our contribution synthesizes these approaches within a unified field-theoretic framework with explicit mathematical foundations and reproducible synthetic validation. Unlike previous work, DFT provides a complete mathematical derivation from variational principles, explicit testable predictions derived from first principles, a reproducible experimental framework with a concrete learning task, and validation against multiple alternative dynamical systems to isolate DFT-specific effects.

1.2 Paper structure

Section 2 presents the mathematical foundations of DFT. Section 3 details the experimental framework, including the learning task, computational implementation, and statistical methodology. Section 4 presents experimental validation of the four predictions on synthetic data. Section 5 discusses implications, limitations (including the real-data limitation analysis), and future directions. Supplementary derivations and implementation details are provided in the Appendices A1–A4.

2 Mathematical foundations of data field theory

2.1 Data field formulation

Definition 2.1 (Data Field). Given a compact Riemannian manifold (, g) of dimension d, a data field is a time-dependent vector-valued function: , where k represents the dimensionality of the representation space. The field ϕ(p, t) encodes the state of the learning system at position p ∈ and time t.

The dynamics of the data field follow from variational principles. We define the Ginzburg-Landau free energy functional as Equation 1:

where the potential V(ϕ) is with α,β > 0. This is the standard ϕ⁴ theory that exhibits spontaneous symmetry breaking.

Assumption 2.2 (Learning as Energy Minimization). Intelligent behavior corresponds to the minimization ofF[ϕ] under constraints imposed by dataJ(p, t) and noise ξ(p, t).

The field evolution follows gradient descent with stochastic fluctuations as Equation 2:

where Γ > 0 is a mobility coefficient, J(p, t) represents external input signals (data), ξ(p, t) is space-time white noise, and σ controls noise amplitude.

2.2 Hyperbolic regularization for causal structure

To ensure finite propagation speed and numerical stability, we introduce hyperbolic regularization via the telegraph equation:

where τ > 0 controls inertia.

Proposition 2.3 (Finite Propagation Speed). Solutions of Equation 3satisfy finite propagation speed: if initial data ϕ(p, 0) and ∂_tϕ(p, 0) have compact support withinK ⊂ , then fort > 0, ϕ(p, t) has support within thecmaxt-neighborhood ofK.

2.3 Spectral analysis and stability

The Hessian of the free energy plays a crucial role in stability analysis: .

Definition 2.4 (Spectral Gap). For an equilibrium field configuration ϕ^*, the spectral gap is, where λ₂ denotes the smallest positive eigenvalue (excluding zero modes from continuous symmetries). The spectral gap depends on the field configuration and therefore varies with the potential parameter α (Binder, 1981).

Theorem 2.5 (Generalization Bound). For a data field ϕ with spectral gap m_gap, the out-of-distribution generalization error is bounded by:, where L is the Lipschitz constant of the decision function, Δ_shift measures distribution shift,D is distance to domain boundary, andC, C′ > 0 are constants depending only on manifold geometry.

3 Experimental framework and methods

3.1 Learning task specification

To ground DFT in a concrete learning problem, we define a hierarchical classification task on the manifold = S². Consider a taxonomy with C = 8 categories organized in a binary tree of depth L = 3. Each category c corresponds to a region centered at prototype .

Definition 3.1 (Data generation). Data points (x, y) are generated as: (1) Sample categoryc~Uniform{1, …, C}; (2) Sample x~vMF(μ_c, κ), the von Mises-Fisher distribution with mean μ_cand concentration κ; (3) Set labely = c. The von Mises-Fisher distribution onS²has densityp(x|μ, κ) ∝ exp(κμ^⊤x).

3.2 Operational definition of concept formation

To avoid circularity in critical phenomena analysis, we define concept formation time tc independently of correlation statistics:

Definition 3.2 (Concept formation time). For a field trajectory ϕ(t), define, where I(ϕ(t);Y) is the mutual information between field state and true labels, computed viak-nearest neighbors estimation (Kraskov et al., 2004) withk = 5, and η = 0.9.

3.3 Out-of-distribution definition

Out-of-distribution data are defined as follows. For a given set of class centers on the sphere, we generate OOD data by applying a random rotation matrix R ∈ SO(3) to all class centers: . We then sample test points from von Mises-Fisher distributions centered at these rotated centers with reduced concentration parameter κ_OOD = κ/(1+5Δ_shift), where Δ_shift ∈ [0, 1] controls the shift magnitude. This operationalization captures two common forms of distribution shift: change in the underlying concept locations (via rotation) and increased input noise (via reduced concentration).

3.4 Statistical methodology

All experiments follow rigorous statistical protocols. Sample sizes of n = 100 independent runs with different random seeds are used. Multiple testing correction employs Bonferroni correction with family-wise error rate α = 0.05. Confidence intervals are computed via 95% bootstrap with 1,000 resamples. Effect sizes are reported as Cohen's d alongside p-values. Model comparison uses AIC/BIC for alternative functional forms.

4 Experimental results

4.1 Prediction 1: critical phenomena in concept formation

We first tested the prediction that concept formation should exhibit critical phenomena characteristic of continuous phase transitions. Following Definition 3.2, we operationalized concept formation time tc as the time at which the mutual information between the field state and true labels reaches 90% of its maximum value. This information-theoretic definition is independent of correlation length measurements, ensuring non-circular validation.

Figure 1 presents our experimental results. Figure 1a shows the correlation length ξ(t) as a function of time, revealing a sharp divergence near the concept formation time t_c ≈ 100. Figure 1b confirms power-law scaling on a log-log plot, with linear regression yielding ν = 0.63 ± 0.04 (R² = 0.97, p < 10⁻⁶). Figure 1c displays the power spectral density of the field fluctuations, exhibiting 1/f scaling with exponent α = 1.05 ± 0.08, significantly different from white noise (α = 0, p < 10⁻⁸) or Brownian motion (α = 2, p < 10⁻¹⁰). Figure 1d shows the mutual information trajectory used to determine t_c independently of correlation length measurements.

Figure 1

A critical question arises regarding the interpretation of these exponents. Our system is defined on the 2-sphere S², a two-dimensional manifold. The expected universality class for a ϕ⁴ theory in two dimensions is the 2D Ising model, with exponents ν = 1.00, β = 0.125, γ = 1.75, η = 0.25. However, our measured exponents (ν = 0.63 ± 0.04, β = 0.33 ± 0.02, γ = 1.24 ± 0.05, η = 0.036 ± 0.008) consistently match the 3D Ising universality class rather than the 2D class. This discrepancy can be resolved by recognizing that our system includes temporal dynamics, which effectively increases the dimensionality. The stochastic partial differential equation governing field evolution includes both spatial and temporal derivatives, and the noise term introduces fluctuations in both space and time. Consequently, the effective dimension of the system is d_eff = d_spatial+d_temporal = 2+1 = 3, where the temporal dimension contributes with dynamical critical exponent z. This interpretation is consistent with the theory of dynamic critical phenomena.

Table 1 summarizes the measured exponents alongside theoretical predictions. Four out of five exponents agree with the 3D Ising class within statistical uncertainties. The specific heat exponent α deviates from the 3D Ising prediction (0.11) and instead aligns with the 1/f noise exponent, reflecting the fact that our system is driven and dissipative rather than in thermal equilibrium.

4.2 Prediction 2: mass-robustness law

Theorem 2.5 predicts that out-of-distribution generalization error should scale inversely with the square of the spectral gap: . To test this prediction, we systematically varied the spectral gap by adjusting the parameter α in the Ginzburg-Landau potential. The spectral gap was computed from the field-dependent Hessian H[ϕ] = −Δ+V″(ϕ), which varies with α and the field configuration.

Figure 2 presents our results. Figure 2a shows the relationship between OOD error and the inverse squared spectral gap, revealing a strong linear correlation. The best fit to ϵ = a·m⁻²+b yields a = 0.324 ± 0.028, b = 0.042 ± 0.011 with R² = 0.94. The Pearson correlation between ϵ and m_gap is ρ = −0.78 (p = 8.4 × 10⁻⁷), confirming the predicted inverse relationship. Figure 2b compares three candidate functional forms using the Akaike information criterion; the inverse-square law is strongly favored (ΔAIC > 7 relative to the next-best model). Figure 2c demonstrates the scaling of OOD error with shift magnitude, showing quadratic growth as predicted by the bound. Figures 2d–f present robustness checks including partial correlations controlling for training time and noise level, 10-fold cross-validation showing consistent performance (mean accuracy = 0.889 ± 0.003), and correlation significance tests confirming the statistical validity of our findings.

Figure 2

4.3 Prediction 3: emergent causal structure

We validated Proposition 0.0.3 by measuring signal propagation speed from localized perturbations. Using parameters Γ = 1.0 and τ = 1.0, the theoretical maximum speed is . This corrects a factor-10 error present in earlier versions of this work.

Figure 3 shows our results. Figure 3a displays wave propagation at different times following a localized Gaussian perturbation. Figure 3b shows the linear relationship between distance and arrival time. Linear regression yields ceff = 0.98 ± 0.03 (95% CI from 100 runs), matching the theoretical prediction c_max = 1.0 within error margins. Figure 3c verifies the light cone structure: for d > cefft, the correlation C(p, q; t) remains below noise floor (p>0.05 for all such pairs, permutation test), confirming that finite propagation speed is not merely a numerical artifact. Figure 3d shows the bootstrap distribution of measured speeds, while Figure 3e demonstrates the predicted parameter dependence . Figure 3f compares DFT with alternative dynamics, showing that only hyperbolic dynamics produce finite propagation speed.

Figure 3

4.4 Prediction 4: approximate rotational equivariance

Symmetry preservation was quantified via the Ward identity residual R, which measures violation of rotational symmetry. For a rotation g ∈ SO(3), the residual is defined as , where is the discrete Laplacian operator.

Figure 4 presents our results. Figure 4a demonstrates field transformations under rotations, showing that rotated fields remain consistent with the original after transformation. Figure 4b shows the distribution of Ward residuals across 1000 random rotations and field configurations, yielding mean residual R = 0.0032 ± 0.0008, significantly below our threshold of 0.05 (p < 10⁻¹⁰, one-sample t-test). Figure 4c shows systematic convergence with mesh refinement for N ∈ {162, 642, 2, 562, 10, 242}. Figure 4d confirms first-order convergence R ∝ h^1.02 (95% CI [0.98, 1.06]), where h ~ N^−1/2 is the mesh spacing. Figure 4e compares the cotangent Laplacian used in DFT with alternative discretizations, showing superior equivariance properties. Figure 4f provides continuum extrapolation to h → 0, with limiting residual R_continuum = 0.00018 ± 0.00004.

Figure 4

4.5 Comparison with baseline methods

To assess DFT's practical utility on synthetic data where its assumptions hold, we compared its performance against standard machine learning methods on the same hierarchical classification task.

Table 2 summarizes the key findings. DFT achieves 89.2%±1.8% accuracy, outperforming all baseline methods including k-NN (76.3%), SVM (79.1%), MLP (81.8%), and Random Forest (78.4%). In terms of calibration, DFT has the lowest Expected Calibration Error (ECE) of 0.048, indicating well-calibrated probability estimates. An ablation study reveals that each component of DFT contributes to performance: removing hyperbolic regularization reduces accuracy to 76.1%, removing the critical potential reduces accuracy to 68.3%, and removing field dynamics entirely reduces accuracy to 45.2%. These results demonstrate that DFT's unique combination of geometric structure, critical dynamics, and hyperbolic regularization is necessary for achieving state-of-the-art performance on hierarchical classification tasks where the geometric assumptions hold.

4.6 Limitation analysis: validation on real-world data

To assess DFT's applicability beyond synthetic data where its geometric assumptions are satisfied, we conducted a limitation analysis using the MNIST dataset of handwritten digits projected onto the sphere (Evans, 2010). The projection procedure consisted of standardizing the 784-dimensional pixel vectors, reducing dimensionality to three components using principal component analysis, and normalizing each point to unit norm, thereby projecting onto S². We restricted the analysis to digits 0 through 7 to maintain an 8-class problem comparable to the synthetic experiments.

Figure 5 summarizes the results. DFT achieved a classification accuracy of 15.7%±2.1% on this real-world dataset, which is above random chance (12.5% for eight classes) but significantly below the k-NN baseline (51.7%±2.5%, p < 0.01). The confusion matrix (Figure 5c) shows that DFT's errors are distributed across all classes without clear structure, suggesting that the spherical projection of MNIST does not satisfy the geometric assumptions underlying DFT.

Figure 5

4.6.0.1 Limitation acknowledgment

This result reveals an important limitation of the current DFT framework. DFT assumes that: (1) the intrinsic geometry of the data manifold is known a priori (here, S²); (2) class-conditional distributions are isotropic von Mises-Fisher on that manifold; and (3) the hierarchical category structure respects the manifold's geodesic distances. For projected MNIST, all three assumptions are violated. The data do not naturally lie on a sphere after PCA projection, the class distributions are not isotropic, and the digit categories do not follow a simple hierarchical structure on S². Consequently, DFT's strong inductive biases become liabilities rather than assets when applied to data that do not satisfy its assumptions.

4.6.0.2 Implications for future work

This limitation highlights a critical direction for future research: extending DFT to learn the manifold geometry from data rather than assuming it a priori. Potential approaches include (1) Riemannian metric learning from data; (2) neural parametrization of the manifold; (3) anisotropic kernels that adapt to local data density; and (4) end-to-end differentiable DFT layers that can be integrated with deep learning architectures. These extensions are actively being developed and will be reported in future work.

5 Discussion

5.1 Summary of findings

Our experiments validate four predictions derived from the Data Field Theory framework on synthetic data where the geometric assumptions hold. First, we observed critical phenomena during concept formation, with correlation length diverging as and temporal fluctuations exhibiting 1/f noise with exponent α = 1.05 ± 0.08. Second, we confirmed the mass-robustness law with Pearson's correlation ρ = −0.78 (p < 10⁻⁶), providing a quantitative relationship between spectral properties and generalization. Third, we demonstrated finite-speed causal propagation with ceff = 0.98 ± 0.03, matching the theoretical prediction cmax = 1.0 after correcting the parameter values to Γ = 1.0 and τ = 1.0. Fourth, we showed approximate rotational equivariance with Ward identity residual R = 0.0032 ± 0.0008, converging to exact symmetry in the continuum limit with rate R ∝ h^1.02.

5.2 Interpretation of critical exponents

A notable finding is that the measured critical exponents (ν = 0.63, β = 0.33, γ = 1.24, η = 0.036) match the 3D Ising universality class rather than the 2D Ising class expected for a system defined on S². We interpret this as evidence for an effective increase in dimensionality due to temporal dynamics. In dynamic critical phenomena, the addition of a temporal dimension with dynamical exponent z can shift universality classes. For our system, the stochastic partial differential equation couples spatial and temporal fluctuations, effectively realizing a (2+1)-dimensional system. This interpretation is supported by finite-size scaling analysis and is consistent with the theory of critical dynamics. The specific heat exponent α deviates from the 3D Ising prediction because our system is driven and dissipative rather than in thermal equilibrium; the 1/f noise (α = 1.05) reflects the non-equilibrium nature of the learning dynamics.

5.3 Relation to existing work

DFT builds upon several research traditions while making distinct contributions. Geometric deep learning (Bronstein et al., 2021) exploits manifold structure for representation learning but typically lacks dynamical equations and predictive laws. Neural field theories (Coombes et al., 2014) model cortical dynamics as continuous fields but focus on biological neural networks rather than abstract learning. The critical brain hypothesis (Chialvo, 2010; Beggs and Plenz, 2003) suggests neural systems operate near phase transitions, but our work provides a specific mechanism (Ginzburg-Landau dynamics with data coupling) that generates criticality from learning. Hyperbolic embeddings (Nickel and Kiela, 2017) leverage negative curvature for hierarchical data, complementing our spherical geometry experiments. The mass-robustness law relates to sharpness-based generalization bounds in deep learning (Keskar et al., 2017), but our bound is derived from first principles and makes a specific quantitative prediction that we have experimentally validated. Recent work on equivariant neural networks (Satorras et al., 2021) and implicit neural representations (Sitzmann et al., 2020) shares our interest in symmetry and continuous representations, but DFT provides a unified dynamical framework that integrates these ideas.

5.4 Limitations and future work

Several limitations of this study should be honestly acknowledged.

First, and most importantly, while we validated DFT's four theoretical predictions on synthetic data where its geometric assumptions hold, the real-data limitation analysis on MNIST revealed that DFT's geometric assumptions are violated by this dataset, resulting in poor performance (15.7% accuracy vs. 51.7% for k-NN). This indicates that the current framework requires significant extension to handle real-world data where the manifold geometry is not known a priori. DFT is therefore best understood as a theoretical framework for learning on known Riemannian manifolds, with extensions to unknown manifolds left as an open challenge.

Second, our theoretical analysis assumes a compact Riemannian manifold without boundary; extensions to non-compact or bounded domains would require additional boundary conditions.

Third, the computational cost scales as O(N²) with mesh size, limiting the resolution for high-dimensional manifolds.

Future work should explore several directions. The most pressing is learning the manifold geometry from data rather than assuming it a priori. This could involve Riemannian metric learning, neural manifold parametrization, or end-to-end differentiable DFT layers. Application to problems where the manifold is known (e.g., spherical data in geophysics, astronomy, or protein structure analysis) would test DFT's practical utility in appropriate domains. Extensions to more general Riemannian manifolds, including those with mixed curvature or non-trivial topology, could capture more complex data structures. From a theoretical perspective, a path integral formulation of DFT could enable systematic perturbation theory and renormalization group analysis, potentially yielding scaling laws for learning curves. These extensions remain conjectural at this stage and would require careful experimental validation.

6 Conclusion

We have presented Data Field Theory as a mathematical framework for learning grounded in differential geometry and field theory. Through rigorous experiments on a hierarchical classification task using synthetic data where the geometric assumptions hold, we validated four key predictions derived from first principles: critical phenomena during concept formation with universal scaling exponents (noting the interesting emergence of 3D exponents from a 2D geometry with temporal dynamics), a mass-robustness law relating spectral properties to generalization (, ρ = −0.78), finite-speed causal propagation from hyperbolic regularization (c_eff = 0.98 ± 0.03), and approximate rotational equivariance converging to exact symmetry (R = 0.0032 ± 0.0008). All findings are supported by comprehensive statistical validation, sensitivity analyses, comparisons with alternative dynamical systems and standard machine learning methods, and full reproducibility documentation.

6.1 Limitations and future directions

A limitation analysis on real-world data (MNIST projected onto a sphere) revealed that DFT's geometric assumptions (known manifold, isotropic class-conditional distributions) are violated by this dataset, resulting in poor performance (15.7% accuracy vs. 51.7% for k-NN). This limitation highlights a key direction for future work: extending DFT to learn the manifold geometry from data rather than assuming it a priori. Additional future directions include application to problems where the manifold is known (e.g., spherical data in geophysics or astronomy), extensions to more general Riemannian manifolds, and a path integral formulation for systematic perturbation theory.

The mathematical structure of DFT provides a principled alternative to heuristic approaches, connecting learning phenomena to established physical principles. By grounding learning in geometric dynamics rather than parameter optimization, DFT offers a path toward more interpretable and theoretically coherent learning systems, while honestly acknowledging that substantial work remains to bridge the gap to real-world applications where the manifold geometry is unknown.

References

• 1

  BeggsJ. M.PlenzD. (2003). Neuronal avalanches in neocortical circuits. J. Neurosci. 23, 11167–11177. doi: 10.1523/JNEUROSCI.23-35-11167.2003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 2

  BinderK. (1981). Finite size scaling analysis of ising model block distribution functions. Z. Phys. B: Condens. Matter43, 119–140. doi: 10.1007/BF01293604

  • CrossRef
  • Google Scholar
  • View reference in article

• 3

  BronsteinM. M.BrunaJ.CohenT.VelickovicP. (2021). Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv [preprint]. arXiv:2104.13478.

  • Google Scholar
  • View reference in article

• 4

  ChialvoD. R. (2010). Emergent complex neural dynamics. Nat. Phys. 6, 744–750. doi: 10.1038/nphys1803

  • CrossRef
  • Google Scholar
  • View reference in article

• 5

  CoombesS.beim GrabenP.PotthastR.WrightJ. (Eds.). (2014). Neural Fields: Theory and Applications. Cham: Springer. doi: 10.1007/978-3-642-54593-1

  • CrossRef
  • Google Scholar
  • View reference in article

• 6

  EvansL. C. (2010). Partial Differential Equations, 2nd Edn. Providence, RI: American Mathematical Society.

  • Google Scholar
  • View reference in article

• 7

  KeskarN. S.MudigereD.NocedalJ.SmelyanskiyM.TangP. T. P. (2017). “On large-batch training for deep learning: generalization gap and sharp minima,” in International conference on learning representations (Ithaca, NY).

  • Google Scholar
  • View reference in article

• 8

  KraskovA.StögbauerH.GrassbergerP. (2004). Estimating mutual information. Phys. Rev. E69:066138. doi: 10.1103/PhysRevE.69.066138

  • CrossRef
  • Google Scholar
  • View reference in article

• 9

  NickelM.KielaD. (2017). “Poincaré embeddings for learning hierarchical representations,” in Advances in Neural Information Processing Systems, 30 (Long Beach, CA).

  • Google Scholar
  • View reference in article

• 10

  SatorrasV. G.HoogeboomE.WellingM. (2021). “E(n) equivariant graph neural networks,” in International conference on machine learning (Vienna), 9323–9332.

  • Google Scholar
  • View reference in article

• 11

  SitzmannV.MartelJ. N.BergmanA. W.LindellD. B.WetzsteinG. (2020). Implicit neural representations with periodic activation functions. Adv. Neural Inf. Process. Syst. 33, 7462–7473. doi: 10.48550/arXiv.2006.09661

  • CrossRef
  • Google Scholar
  • View reference in article