Rolling SHAP Analysis: Catching Feature Decay Before It Catches You
Rolling SHAP value analysis that tracks which features drive L1 decisions over time, detecting feature decay and concept drift.
Why Static SHAP Is Not Enough
Training-time SHAP values tell you which features mattered historically. But features decay. A feature that was the top predictor in 2020 might be noise by 2024. S11 computes SHAP values on a rolling 500-trade window, tracking how each feature's importance evolves over time.
When a feature's SHAP contribution drops below 50% of its training-time importance for three consecutive windows, S11 flags it for review. This early detection of concept drift prevents the model from relying on features that no longer carry signal.
What We Found
Two features showed meaningful decay over the 7.5-year backtest: bid_ask_imbalance and a custom momentum divergence indicator. Both lost predictive power gradually rather than suddenly, consistent with market microstructure evolution rather than a single structural break.
The good news is that core features like ADX, RSI, Hurst, and ATR-based measures showed stable or increasing importance over time. Classical indicators based on price and volume are harder to arbitrage away than microstructure features because they capture fundamental market dynamics.
The Monitoring Mindset
S11 is a diagnostic module, not a trading module. It does not directly affect entries, exits, or sizing. Its value is in preventing slow model degradation that you would not notice until monthly returns start declining. Catching a decaying feature at month 6 instead of month 18 lets you intervene (retrain, replace, or remove) before performance degrades meaningfully. For a frozen production system like V7, S11 provides the confidence that the current model weights are still valid. When it eventually flags serious decay, that will be the signal that a controlled retraining cycle is needed.