Failures are rare, and that changes everything
A well-maintained pump might produce 50,000 healthy measurement windows for every few hundred faulty ones. That imbalance breaks the intuitions people bring from balanced textbook examples, starting with accuracy.
Handling imbalance
- Use the right metrics: precision, recall, F1, the precision-recall curve and false alarms per asset per month.
- Reweight the loss so mistakes on rare classes cost more, rather than discarding healthy data.
- Resample carefully. Oversampling or SMOTE can help, but only on the training split, never before splitting.
- Reframe as anomaly detection when faults are too rare to learn directly, then classify only flagged windows.
Few labels
Even when faults occur, they are rarely labelled well. The CMMS says "replaced bearing" but not which defect, when it started, or which sensor saw it. Practical ways forward:
| Approach | Idea | When it helps |
|---|---|---|
| Weak labels from work orders | label the weeks before a repair as "degrading" | you have maintenance history with dates |
| Active learning | the model asks an analyst to label the windows it is least sure about | analyst time is the bottleneck |
| Semi-supervised learning | learn structure from unlabeled data, fine-tune on few labels | plenty of unlabeled data |
| Self-supervised pre-training | train a network to predict masked or future parts of the signal, then fine-tune | large fleets of similar assets |
| Expert rules as labellers | use reliable rules (for example envelope peaks at BPFO) to generate training labels | you have trusted diagnostic rules |
Few run-to-failure histories
Prognosis models want complete degradation paths, and good maintenance prevents them. Options include pooling histories across similar assets with per-unit normalisation, accelerated life tests on a test rig, survival models that also learn from units that have not failed, and physics-based degradation laws with Bayesian updating. See remaining useful life.
Synthetic data and simulation
Simulated signals with known faults, like the ones used in every explainer on this site, are valuable for three jobs: testing that a pipeline works end to end, pre-training models, and teaching. They are risky as the only training data, because a simulator only produces the effects its author thought of. A model trained purely on simulation may learn the simulator. Use simulation to bootstrap and verify; validate on real data.
Classical baseline · seeded-fault rigs
Laboratory rigs with drilled or machined defects, such as the CWRU bearing dataset, made fault diagnosis research possible. Their faults are cleaner and more uniform than natural spalls, which is why models that score near-perfect on them often disappoint in plants.
Domain shift
A model learns the conditions of its training data. Change the machine, the sensor position, the speed range or the season, and the input distribution shifts. Typical sources:
- Different machine of the "same" type: other foundation, other resonance frequencies.
- Sensor moved, re-mounted or replaced with a different sensitivity.
- New operating regime: a new product, a new setpoint, a variable-speed drive retrofit.
- Seasonal temperature changes that alter clearances and lubricant viscosity.
Defences include features normalised per asset and per operating state, order-based features for variable speed, domain adaptation methods that align feature distributions between source and target machines, and drift monitoring in production (see MLOps and drift).
Key takeaways
- With rare faults, accuracy is misleading. Track precision, recall and false alarms per month.
- Labels come from maintenance records, analysts and trusted rules; invest in capturing them.
- Synthetic data is excellent for testing and pre-training, risky as the only source.
- Expect domain shift between machines, sensors and seasons, and design features to resist it.