Federated learning: a demo on real public data
Mesh (Mesh Universe) runs a confidential federated learning demo on real public data. Several actors train a shared model without ever sharing their data: each trains locally, and only the model's parameters travel, encrypted. The federated model gains 6 to 8 AUC points over the average of the standalone actors, in all twenty-one runs of the bench. And federating rather than gathering all the data in one place costs nothing measurable: 0.3 of an AUC point at most, against a measurement uncertainty at least six times larger. Keeping the data at home therefore gives up nothing.
The same model, on real data: the model travels, your data stays home.
What does the demo show?
Much of the talk about federated learning stays theoretical. This demo does the opposite: it actually runs a federated training, round after round, on real, public datasets, and shows at every step what the collective gains versus each isolated actor.
It runs the mechanism itself: each participant trains the model on its own data, computes an update (parameters, never raw data), and returns only that update, end-to-end encrypted. A coordinator aggregates the updates (FedAvg) to produce an improved global model. → see Federated learning: definition and how it works.
Which public datasets are used?
The demo runs on open, real, cited datasets, each with its original source and licence:
- Predictive maintenance: AI4I 2020 Predictive Maintenance Dataset. UCI Machine Learning Repository, CC BY 4.0. Machine-failure detection from process readings.
- Molecular toxicology: Tox21. Public program (MoleculeNet / tox21.gov). Predicting molecular toxicological activity.
- Wind: EDP Wind Farm Open Data. SCADA 2016 + failure log (research/education use). Anticipating turbine failure.
An additional leather quality-control scenario illustrates a cross-workshop case: that one is a calibrated synthesis (not a public dataset), and is flagged as such in the demo.
What is the observed result?
Across these datasets, the pattern repeats: a lone actor has a blind spot: it only observes part of reality. By federating, each one gains access to what it can't see, without exposing what it can. Three markers frame the result, each computed exactly rather than approximated:
- Against the average of the standalone actors: the federated model gains 6 to 8 AUC points, and the gain is positive in all twenty-one runs (seven seeds, three datasets). That is three to four times the uncertainty of such a measurement.
- Against the model that would have gathered everything in one place: the federated model sits 0.3 of an AUC point at most from it, while the measurement uncertainty is at least six times larger. The gap cannot be told apart from statistical noise: giving up centralization gives up nothing.
- Against the best of the standalone actors: the federated model is ahead on AI4I and EDP, on all seven seeds. On Tox21 it matches it: the 0.1 point gap is far below the measurement uncertainty, which makes it a tie and not a lead.
Runs are reproducible (fixed seed): re-run, the demo returns the same figures. That's deliberate: we show a verifiable mechanism, not a hand-waved performance.
What the demo proves, and what it doesn't
- What it proves. That confidential federated learning genuinely runs on real public data, that nothing sensitive travels (only encrypted parameters), and that the collective does better than the average of the isolated actors in all twenty-one runs.
- What it doesn't prove. A guaranteed performance figure on your data: that is measured on a pilot, on your datasets, at your site. The demo shows the mechanism and the order of magnitude of the gain, not a contractual promise.
- Its technical scope. The participants are simulated inside a single process: the training, the aggregation rounds and the encryption of the exchanges are real, but there is no agent installed on a remote site and no network protocol between organisations. Mesh is at launch stage and no production deployment exists to date.
How to access the demo?
The interactive demo is provided on request, with named access: we'd rather open it for you and walk you through the results than leave it self-service. Email us with your sector; we'll tailor the example to your case.
FAQ: Federated learning demo on real data
Does Mesh have a demo on real data?
Yes. Mesh (Mesh Universe) runs a confidential federated training on real public datasets: AI4I 2020 (maintenance, UCI, CC BY 4.0), Tox21 (toxicology) and EDP Wind. The federated model gains 6 to 8 AUC points over the average of the standalone actors, without sharing data.
What do you lose by not centralizing the data?
Nothing measurable. On the three datasets, the federated model sits 0.3 of an AUC point at most from the model you would get by gathering all the data in one place, against a measurement uncertainty at least six times larger: the gap cannot be told apart from statistical noise. The collective's gain, by contrast, can.
Is the data real or synthetic?
The AI4I 2020, Tox21 and EDP datasets are real public data (source and licence shown). The leather scenario is a calibrated synthesis, flagged as such. It's a mechanism demonstration, not a customer result.
Are the results reproducible?
Yes: runs use a fixed seed. Re-run, the demo returns exactly the same figures: a verifiable mechanism.
Does that mean I'll get the same gain on my data?
Not necessarily. The demo shows the direction and order of magnitude of the gain; performance on your data is measured on a pilot, at your site, without your data leaving home. → see The Collective.
How do I access it?
On request, with named access: use the contact form and tell us your sector.
See the demo, then let's talk about your data
Named access on request: we open the demo and walk you through it.