Skip to content

Hyperparameter Tuning Examples

Practical examples demonstrating Optuna-based hyperparameter tuning in DoubleML PySpark — from a guided end-to-end workflow to advanced edge cases and tuning strategies.

1. Interactive Notebooks

A. Core Workflow

  • Tuning Walkthrough in which we cover the full tune_ml_models workflow for IRM and PLR estimators, parameter space definition via Optuna trial callables (suggest_int, suggest_float, suggest_categorical), global and learner-specific Optuna settings (trial counts, direction, samplers, pruners, callbacks), scoring method selection for classifiers (areaUnderROC, logLoss, f1) and regressors (rmse, mae, r2), tuning modes (full, partial, no-tuning, explicit None skip), the DMLOptunaResult object with access to full Optuna Study histories, and best practices for reproducibility, convergence, and small-dataset regimes.

B. Edge Cases & Advanced Patterns

  • Tuning Edge Cases in which we explore partial tuning (subset of learners), no-tuning baselines, explicit None skipping, model-specific Optuna overrides (different trial counts, directions, and scoring metrics per learner), memory-constrained sequential tuning, and default-settings fallback behavior.

2. Key Concepts

Each notebook builds on the same core workflow:

  1. Parameter space definition: callable functions that map an Optuna Trial to a dict of sampled hyperparameters (e.g., regParam on a log scale, elasticNetParam, maxIter).
  2. Optuna settings: global defaults (n_trials, direction, gc_after_trial) with optional per-learner overrides for fine-grained control.
  3. Scoring alignment: matching the optimization direction (minimize / maximize) to the chosen metric (rmse, areaUnderROC, mae, r2).
  4. Result inspection: accessing best_params, best_score, and the full Optuna Study object for convergence analysis and parameter importance ranking.

3. Supported Estimators

Both notebooks use DoubleMLIRM as the running example. The same tuning API applies identically to DoubleMLPLR where only the learner names change (ml_m + ml_l instead of ml_m + ml_g0 + ml_g1).