Rockfall Prediction System
Sole developer · team of 3 · Smart India Hackathon 2025- XGBoost
- FastAPI
- React
Problem
Estimate rockfall risk for open-pit mine slopes from synthetic, sensor-style readings, and alert operators by SMS when the risk is critical.
Data
A 100,000-row synthetic slope-stability dataset: slope angle, 24-hour rainfall, displacement rate, pore pressure, and an image-derived crack score, plus an engineered rainfall × slope interaction term.
What I built
A SMOTE + StandardScaler + XGBoost pipeline for binary rockfall-risk classification, a FastAPI service that maps predicted probability to risk levels (GUARDED ≥ 0.45, ELEVATED ≥ 0.70, CRITICAL ≥ 0.95) and sends a Twilio SMS on CRITICAL, and a single-page React dashboard (sliders → live prediction). Sole developer on a team of 3 at Smart India Hackathon 2025.
How I evaluated it
Stratified 60/20/20 train / validation / test split (seed 42). Every number below is from the 20,000-row test set, for the exact XGBoost model the API serves, at the API's own risk cutoffs.
Results
When the system sends an SMS, it is almost always right: 98% of alerts are real events. But it stays silent for 83% of real rockfall events. On synthetic data where 65% of rows are events, the model separates the classes only moderately (ROC-AUC 0.75).
| ROC-AUC (test) | 0.75 |
| Baseline: always predict "rockfall" | 64.75% accuracy |
| GUARDED (≥ 0.45): precision / recall | 78.3% / 71.7% |
| ELEVATED (≥ 0.70): precision / recall | 88.2% / 42.3% |
| CRITICAL (≥ 0.95, sends SMS): precision / recall | 98.3% / 17.4% |
What broke
- My training script tuned its decision threshold on the same test set it reported results on. I moved threshold selection to a separate validation split.
- Training evaluated an average of XGBoost and a random forest, but the API served XGBoost alone, so none of the reported numbers described the deployed model. I removed the random forest and re-measured the served model.
- The threshold from training was never connected to the app. The API uses hand-set cutoffs, and at the one that sends alerts (0.95) the system misses 5 of every 6 real rockfall events.
- Metrics were printed to the console and never saved, and the training data was later deleted from the repo. The dataset is restored, and every training run now writes results/metrics.json.
What I'd do next
Choose the CRITICAL cutoff on the validation set from an explicit trade-off (how many false alarms are acceptable per missed rockfall) instead of a hand-set 0.95, and validate on real sensor data before anything like this is trusted.