Continuous Quasi-Unitary Lie-Group State Space Models with Sparse Mixture-of-Experts. Eliminates dissipative memory decay while delivering strictly constant O(1) inference memory.
Head-to-head empirical evaluation loading real weights directly from Hugging Face Hub under unpadded live execution:
| Model Architecture | Total Parameters | Active Params / Token | Generation Speed | Step Latency | RAM at L=8,192 |
|---|---|---|---|---|---|
| SmolLM-135M (Hugging Face) | 134.52M | 134.52M (Dense) | 1.67 tok/s | 597.86 ms/tok | 360.00 MB (Expanding) |
| Mamba-130M-HF (Albert Gu et al.) | 129.14M | 129.14M (Dense) | 1.98 tok/s | 506.18 ms/tok | 0.19 MB (Constant) |
| QU-SSM-130M-MoE (Our Model) | 134.89M | 78.27M (Sparse Top-2) | 5.55 tok/s (🥇 3.32x SOTA) | 180.16 ms/tok | 0.19 MB (🥇 1,894x Smaller) |
Visualizing pure 2D orthogonal matrix rotation on SO(2). Notice the radius norm is strictly preserved (‖R(θ)‖₂ ≡ 1.00000):
The state evolves as an exact dual-component log-prefix scan without complex numbers:
# Exact Real Dual-Component Recurrence
theta = clamp(W_theta(x) + theta_base, -pi, pi)
log_g = logsigmoid(W_gamma(x))
S = cumsum(log_g, dim=1).clamp(min=-12, max=0)
Phi = cumsum(theta, dim=1)
u_real = x * exp(-S) * cos(Phi)
u_imag = -x * exp(-S) * sin(Phi)
# Reconstruct State (Strictly O(1) RAM)
h_t = exp(S) * (cos(Phi) * cumsum(u_real)
- sin(Phi) * cumsum(u_imag))Autoregressive causal token generation with Top-2 SwiGLU Mixture-of-Experts routing.
QUSSMForCausalLM1D acoustic pressure wave modeling. Preserves vocal formants without low-pass decay.
QUSSMForAudio77,288 ticks/s throughput with lowest spectral phase error (1.9648 rad) on live LOB streams.
QUSSMForSensorTelemetryContinuous 16x16 patch scanning with O(1) state space memory.
VisionQUSSM134.89M Total Params | 78.27M Active Params | 8 SwiGLU Experts | Top-2 Routing
View Flagship 130M Model ➔64.30M Total Params | 44.64M Active Params | 4 SwiGLU Experts | Top-2 Routing
View Mid-Tier 60M Model ➔29.80M Foundation Params | Dense SwiGLU Backbone | Embedded Edge Ready
View Foundation 15M Model ➔