Skip to content

DoubleMLBLP

doubleml_pyspark.utils.blp.DoubleMLBLP

Best Linear Predictor (BLP) for DoubleML models.

This class fits and summarizes a best linear predictor for orthogonal signals and basis functions, typically used in Double Machine Learning workflows.

Parameters:

Name Type Description Default
basis_columns list of str

Names of the basis columns used as predictors.

required
target_column str

Name of the orthogonal signal column. Default is 'orth_signal'.

'orth_signal'
is_gate bool

Set to True when the basis is a dummy (indicator) matrix built for GATE estimation. Affects the confint default behaviour when basis=None. Default is False.

False

Methods:

Name Description
confint

Pointwise or joint confidence intervals for predicted CATE and GATE values.

fit

Fit the BLP.

summary

Summary for the best linear predictor effect after calling fit.

Attributes:

Name Type Description
basis_columns list[str]

The basis columns.

blp_model Any

The BLP model.

blp_omega ndarray | None

Coefficient covariance matrix of shape (d, d), populated after :meth:fit.

cov_type str | None

Covariance estimator used by the latest call to :meth:fit.

is_gate bool

Whether this BLP was built for GATE (dummy-basis) estimation.

target_column str

The target/outcome column.

Attributes

doubleml_pyspark.utils.blp.DoubleMLBLP.basis_columns property

basis_columns: list[str]

The basis columns.

Returns:

Type Description
list of str

The names of the basis columns.

doubleml_pyspark.utils.blp.DoubleMLBLP.blp_model property

blp_model: Any

The BLP model.

Returns:

Type Description
LinearRegression or None

The fitted PySpark LinearRegression model, or None if not fitted.

doubleml_pyspark.utils.blp.DoubleMLBLP.blp_omega property

blp_omega: ndarray | None

Coefficient covariance matrix of shape (d, d), populated after :meth:fit.

For cov_type='nonrobust' this is σ² · (XᵀX)⁻¹. For the HC estimators it is the corresponding heteroskedasticity-consistent sandwich covariance. The inversion uses the Moore-Penrose pseudoinverse for rank-deficient or nearly collinear basis matrices.

doubleml_pyspark.utils.blp.DoubleMLBLP.cov_type property

cov_type: str | None

Covariance estimator used by the latest call to :meth:fit.

doubleml_pyspark.utils.blp.DoubleMLBLP.is_gate property

is_gate: bool

Whether this BLP was built for GATE (dummy-basis) estimation.

doubleml_pyspark.utils.blp.DoubleMLBLP.target_column property

target_column: str

The target/outcome column.

Returns:

Type Description
str

The name of the target column.

Functions

doubleml_pyspark.utils.blp.DoubleMLBLP.confint

confint(spark_session: SparkSessionType, basis: DataFrame | None = None, level: float = 0.95, joint: bool = False, n_rep_boot: int = 500) -> DataFrame

Pointwise or joint confidence intervals for predicted CATE and GATE values.

Parameters:

Name Type Description Default
spark_session SparkSession

Active Spark session used to create the output DataFrame.

required
basis DataFrame or None

Evaluation grid as a Spark DataFrame. Must contain all columns listed in self.basis_columns.

  • If None and is_gate=True: returns GATEs by evaluating at the identity basis (one row per group).
  • If None and is_gate=False: returns CIs for the raw basis coefficients (equivalent to the summary confidence columns, but as a tidy DataFrame).
  • If a Spark DataFrame is passed: predicts basis @ coef on the supplied grid and returns pointwise CIs.
None
level float

Confidence level in (0, 1). Default is 0.95.

0.95
joint bool

If True, return simultaneous (joint) confidence intervals, using a multiplier-bootstrap max-t critical value that is shared across all evaluation points/groups and computed exactly as in DoubleML-for-py's DoubleMLBLP.confint (doubleml.utils.blp): per evaluation point, the max of the bootstrapped studentised statistic over the n_rep_boot draws, then the level-quantile over evaluation points. If False (default), return pointwise intervals.

False
n_rep_boot int

Number of bootstrap repetitions used to estimate the joint critical value. Only relevant when joint=True. Default is 500.

500

Returns:

Type Description
DataFrame

plus a 'group' column (basis column name) when basis=None.

Raises:

Type Description
ValueError

If :meth:fit has not been called yet.

TypeError

If basis is not a Spark DataFrame (or None).

ValueError

If basis is missing any of self.basis_columns.

doubleml_pyspark.utils.blp.DoubleMLBLP.fit

fit(df: DataFrame, cov_type: str = 'HC0', use_persist: bool = False) -> DoubleMLBLP

Fit the BLP.

Parameters:

Name Type Description Default
df DataFrame

Input DataFrame containing the basis and target columns.

required
cov_type str

Covariance estimator. Supported values are 'nonrobust', 'HC0', 'HC1', 'HC2', and 'HC3'. Default is 'HC0', matching DoubleML-for-py.

'HC0'
use_persist bool

Whether to cache the intermediate BLP DataFrame. Not supported on Databricks serverless compute. Defaults to False.

False

Returns:

Type Description
DoubleMLBLP

The fitted instance.

Raises:

Type Description
NotImplementedError

If an unsupported covariance type is specified.

doubleml_pyspark.utils.blp.DoubleMLBLP.summary

summary() -> pd.DataFrame

Summary for the best linear predictor effect after calling fit.

Returns:

Type Description
DataFrame

DataFrame with coefficients, standard errors, t-values, p-values, and confidence intervals for each basis column and the intercept.