Hyperparameter Tuning Examples¶
Practical examples demonstrating Optuna-based hyperparameter tuning in DoubleML PySpark — from a guided end-to-end workflow to advanced edge cases and tuning strategies.
1. Interactive Notebooks¶
A. Core Workflow¶
- Tuning Walkthrough in which we cover the full
tune_ml_modelsworkflow for IRM and PLR estimators, parameter space definition via Optuna trial callables (suggest_int,suggest_float,suggest_categorical), global and learner-specific Optuna settings (trial counts, direction, samplers, pruners, callbacks), scoring method selection for classifiers (areaUnderROC,logLoss,f1) and regressors (rmse,mae,r2), tuning modes (full, partial, no-tuning, explicitNoneskip), theDMLOptunaResultobject with access to full OptunaStudyhistories, and best practices for reproducibility, convergence, and small-dataset regimes.
B. Edge Cases & Advanced Patterns¶
- Tuning Edge Cases in which we explore partial tuning (subset of learners), no-tuning baselines, explicit
Noneskipping, model-specific Optuna overrides (different trial counts, directions, and scoring metrics per learner), memory-constrained sequential tuning, and default-settings fallback behavior.
2. Key Concepts¶
Each notebook builds on the same core workflow:
- Parameter space definition: callable functions that map an Optuna
Trialto a dict of sampled hyperparameters (e.g.,regParamon a log scale,elasticNetParam,maxIter). - Optuna settings: global defaults (
n_trials,direction,gc_after_trial) with optional per-learner overrides for fine-grained control. - Scoring alignment: matching the optimization direction (
minimize/maximize) to the chosen metric (rmse,areaUnderROC,mae,r2). - Result inspection: accessing
best_params,best_score, and the full OptunaStudyobject for convergence analysis and parameter importance ranking.
3. Supported Estimators¶
Both notebooks use DoubleMLIRM as the running example. The same tuning API applies identically to DoubleMLPLR where only the learner names change (ml_m + ml_l instead of ml_m + ml_g0 + ml_g1).