mHC — Manifold-Constrained Hyper-Connections — restores what unconstrained hyper-connections break: the mixing matrix is projected onto the Birkhoff polytope, so signal propagation stays bounded by construction. Measured at 27B scale: −0.021 training loss, +2.3 pp BBH, 6.7% end-to-end overhead. We make it run on your cluster.
Public mathematics. Private engineering. · arXiv:2512.24880
Expanding the residual stream to n parallel copies with learnable mixing lifts quality. Unconstrained, it also breaks the two properties that made residual networks trainable in the first place.
In a residual network, xl+1 = xl + F(xl) guarantees an unobstructed gradient route through depth. Arbitrary mixing matrices destroy that guarantee — products across layers drift away from identity.
Composed unconstrained mixing amplifies norms multiplicatively. In the published measurements, HC training exhibits gradient-norm spikes on the order of 3×10³ — the kind of instability that ends a multi-week run.
An n-wide stream means n× residual activations. Without kernel-level engineering, that alone prices the method out of production — which is precisely where most teams stop.
These are published, checkable facts — not our claims. We cite them because the audience we work with checks.
Sinkhorn–Knopp (~20 alternating normalizations) projects the residual mixing matrix onto the Birkhoff polytope: non-negative entries, every row and column summing to 1.
Doubly stochastic matrices are closed under composition and non-expansive in ℓ₂ — a convex combination of permutations cannot amplify. Depth-wise products stay bounded; the identity-mapping property is restored, not approximated.
With fused kernels, selective recomputation and pipeline-parallel scheduling, the published end-to-end training overhead is 6.7% at n=4 — the difference between a paper result and a production decision.
Sources: Xie, Wei, Cao et al., arXiv:2512.24880 (mHC) · Zhu et al., arXiv:2409.19606 (Hyper-Connections) · He et al., CVPR 2016 (residual learning).
Illustrative visualization of the published stability behaviour — HC spikes to ~3×10³; mHC remains bounded. Not raw paper data.
Lower final training loss versus baseline at 27B — at a scale where a third decimal is real money.
Downstream reasoning gain (BBH) reported alongside improvements on further benchmarks.
End-to-end training overhead with the full engineering stack applied — the number that turns research into a line item.
A random positive 4×4 matrix is alternately row- and column-normalized until it lands on the Birkhoff polytope. Watch the sums converge to 1 — this is the projection that keeps an n-wide residual stream bounded. In production it runs in ~20 iterations, fused, at negligible cost.
Row / column sums: far → close → = 1.000
Free and technical. Your stack, scale, precision policy, parallelism. We tell you plainly whether mHC is worth your time.
The flagship entry point. mHC running on a proxy of your configuration, instrumented side-by-side with your baseline. Go/no-go report.
Integration in one production training pipeline, A/B at an agreed token budget, full telemetry. Code IP transfers to you.
Multi-node rollout, monitoring, regression suite, team enablement. 90 days of engineering care.
This page won't teach you what doubly stochastic mixing is — you know. It tells you exactly what you get, in what time, for what price. Fixed scope, fixed price, three currencies.
You get the numbers on your workload and the code you paid for. Our kernels, integration playbook and tuning heuristics are not published anywhere — including this page. The mathematics is public; the engineering is the product. NDA as standard, in both directions.
One day, on-site or remote: the mathematics, the failure modes, the operational practice of n-wide residual streams. For research and infra teams.
£1,800 · €2,100 · 9 300 zł · net
| Scope | Demo | Pilot | Production |
|---|---|---|---|
| Live run on a proxy of your config | ✓ | ✓ | ✓ |
| Instrumented A/B at token budget | — | ✓ | ✓ |
| Production pipeline integration | — | ✓ | ✓ |
| Fused kernel path | — | ✓ | ✓ |
| Multi-node · pipeline parallel | — | — | ✓ |
| Monitoring & regression suite | — | — | ✓ |
| Written report & debrief | ✓ | ✓ | ✓ |
| Code IP transfer | — | ✓ | ✓ |
| Support window | Q&A 14 d | 30 d | 90 d |
Net prices. Invoicing from a UK Ltd (Companies House № 16019829) in GBP, EUR or PLN — your choice, fixed at signature. Staged payments. Kickoff within 10 business days. We run a limited number of concurrent engagements. First call is free and commits you to nothing.
The mathematics is published research by DeepSeek-AI (arXiv:2512.24880) — we cite it because our audience checks. What we sell is the engineering that makes it hold on real clusters: kernels, integration, telemetry, operational practice. That part is ours, and it stays ours.
Because the Birkhoff polytope is closed under multiplication and non-expansive in ℓ₂: a product of doubly stochastic matrices is doubly stochastic and cannot amplify a signal. That restores the identity-mapping property residual networks depend on — as a structural guarantee, not a regularizer you hope holds.
An NDA (ours or yours), your model configuration (dims, depth, precision policy, parallelism), and a technical contact. Compute can be yours or ours — agreed on the first call. No weights, no data leave your perimeter.
The published results were obtained in a pipeline-parallel training stack, and the method is compatible with standard tensor/pipeline schemes. Whether it holds in your exact topology is precisely what the Demo measures before you commit to more.
Then you've spent a fixed, known amount to avoid a much larger mistake — and you keep the report and the numbers. If it says “go”, 100% of the Demo fee is credited against the Pilot. Either way you decide on data, not on a deck.
CyberSentinel Solutions Ltd, a UK company (№ 16019829). Fixed-scope, fixed-price statements of work, staged payments, invoicing in GBP, EUR or PLN. NDA as standard, both directions.
The first call is technical, free, and honest — if mHC isn't worth it at your scale, we'll say so and save us both the paperwork.
Reply within 1 business day.