In every day work as data scientist & ML engineer, hyperparameter tuning often consumes a disproportionate amount of experimentation time, yet many tuning failures stem from recurring structural issues rather than random chance. These issues are typically identifiable by experienced practitioners but remain inaccessible to automated optimization systems due to their reliance on scalar objective functions alone.
This work reframes hyperparameter optimization as an iterative reasoning process rather than a pure search problem. The key insight is that intermediate explanation artifacts—specifically SHAP value distributions—can be treated as first-class signals that guide subsequent optimization decisions. Encoding this reasoning explicitly via agents enables systematic reuse of expert heuristics that are otherwise applied informally.
The proposed system decomposes the optimization process into agent roles with clearly defined responsibilities, such as diagnostic reasoning, parameter constraint validation, and experiment coordination. These agents interact through a controlled workflow that preserves reproducibility and auditability while leveraging Retrieval-Augmented Generation to reason over model documentation and prior experiment context.
The case study focuses on gradient boosted tree models, with XGBoost used as the primary example. While the approach generalizes conceptually, it is most effective in settings where model interpretability and parameter interactions dominate performance outcomes. The talk explicitly discusses scenarios where agent-based optimization adds limited value or introduces unnecessary complexity.
Attendees will gain:
The presentation is a technical case study supported by architecture diagrams and experiment traces. All code, configurations, and artifacts will be made available as open source to ensure reproducibility and facilitate adaptation.
