Method library WORKFLOW / 006

Research workflow/6 steps

Assessing heterogeneity with RATE (AUTOC & Qini)

@grf ✓ P D · Jun 29, 2026 · 1.1k views

@misc{grf,
  title        = {grf},
  author       = {Athey and Tibshirani and Wager},
  howpublished = {\url{https://grf-labs.github.io/grf/}},
  note         = {Software / documentation}
}

In the margins · Peer notes@kfold_kenji · The independent train/eval split is non-negotiable here. Priorities have to be out-of-sample or RATE is optimistic. 2

The method workspace

Follow the analysis.

Select a node to explore the method

Method mapInput → 6 steps → output

START WITHInput data

ARRIVE ATResults

Read step 01

Inside the methodReasoning, equations & implementation

Data prep

[GRF] Regression forest

Step notesField note & discussion1

Data preparation — shapes the raw inputs into what the estimator expects.

What happens here

Cross-fit nuisances.

Formula

\hat\mu(x)=\mathbb{E}\!\left[\,Y\mid X=x\,\right]

The estimator

Regression forest — Honest non-parametric regression for E[Y|X], with out-of-bag predictions and pointwise CIs.

OTHER Code @ v2.6.1 ↗ Paper ↗

Reads fromNot specified Feeds into#2

Key code

rf <- regression_forest(X, Y)
Y.hat <- predict(rf)$predictions

Discussion on this step (0)

StatsOtterDiscussion starter · Editorial note

Before using this step, make its assumptions and its output explicit. A useful peer note explains what you would check, what could go wrong, or what would change your interpretation.

Questions to explore

Which assumption matters most here, and what evidence could challenge it?
What would you check before passing this output to the next step?

Peer discussion

Bring your experience, a question, or a different perspective.

Estimation

[GRF] Causal forest

Step notesField note & discussion1

The core estimate — where the causal quantity itself is computed.

What happens here

Causal forest on the training half; produce CATE priorities.

Formula

\tau(x)=\mathbb{E}\!\left[\,Y(1)-Y(0)\mid X=x\,\right]

The estimator

Causal forest — Honest random forest for heterogeneous treatment effects — CATE for a binary treatment via GRF moment conditions.

CATE Code @ v2.6.1 ↗ Paper ↗

Reads from#1 Feeds into#3

Key code

cf <- causal_forest(X, Y, W)          # Y.hat, W.hat cross-fit
tau.hat <- predict(cf)$predictions    # OOB CATEs

Discussion on this step (0)

StatsOtterDiscussion starter · Editorial note

Before using this step, make its assumptions and its output explicit. A useful peer note explains what you would check, what could go wrong, or what would change your interpretation.

Questions to explore

Which assumption matters most here, and what evidence could challenge it?
What would you check before passing this output to the next step?

Peer discussion

Bring your experience, a question, or a different perspective.

Data prep

Train / evaluation split

Step notesField note & discussion1

Data preparation — shapes the raw inputs into what the estimator expects.

What happens here

Hold out an independent half to fit the evaluation forest — RATE needs out-of-sample priorities.

Reads from#2 Feeds into#4#5

Key code

idx <- sample(n, n / 2)            # independent train / eval split
train <- data[idx, ]; eval <- data[-idx, ]

Discussion on this step (0)

StatsOtterDiscussion starter · Editorial note

Before using this step, make its assumptions and its output explicit. A useful peer note explains what you would check, what could go wrong, or what would change your interpretation.

Questions to explore

Which assumption matters most here, and what evidence could challenge it?
What would you check before passing this output to the next step?

Peer discussion

Bring your experience, a question, or a different perspective.

Heterogeneity

[GRF] Rank-weighted ATE — RATE / AUTOC / Qini

Step notesField note & discussion1

Heterogeneity — who is affected, and by how much, not just on average.

What happens here

AUTOC: area under the TOC, sensitive to concentrated benefit.

Formula

\mathrm{AUTOC}=\int_0^1\!\mathrm{TOC}(q)\,dq,\quad \mathrm{TOC}(q)=\mathbb{E}\!\left[\tau(X)\mid \hat S(X)\ge \hat F^{-1}(1-q)\right]-\mathbb{E}[\tau(X)]

The estimator

Rank-weighted ATE — RATE / AUTOC / Qini — Evaluate how well a CATE estimate prioritizes treatment: TOC curve, AUTOC and Qini with confidence intervals.

OTHER Code @ v2.6.1 ↗ Paper ↗

Reads from#3 Feeds into#6

Key code

rate <- rank_average_treatment_effect(eval.forest, priorities = tau.hat)
plot(rate)                          # TOC curve
rate$estimate / rate$std.err        # AUTOC z-stat

Discussion on this step (0)

StatsOtterDiscussion starter · Editorial note

Before using this step, make its assumptions and its output explicit. A useful peer note explains what you would check, what could go wrong, or what would change your interpretation.

Questions to explore

Which assumption matters most here, and what evidence could challenge it?
What would you check before passing this output to the next step?

Peer discussion

Bring your experience, a question, or a different perspective.

Heterogeneity

[GRF] Rank-weighted ATE — RATE / AUTOC / Qini

Step notesField note & discussion1

Heterogeneity — who is affected, and by how much, not just on average.

What happens here

Qini: weights by treated fraction, sensitive to broad benefit.

Formula

\mathrm{AUTOC}=\int_0^1\!\mathrm{TOC}(q)\,dq,\quad \mathrm{TOC}(q)=\mathbb{E}\!\left[\tau(X)\mid \hat S(X)\ge \hat F^{-1}(1-q)\right]-\mathbb{E}[\tau(X)]

The estimator

Rank-weighted ATE — RATE / AUTOC / Qini — Evaluate how well a CATE estimate prioritizes treatment: TOC curve, AUTOC and Qini with confidence intervals.

OTHER Code @ v2.6.1 ↗ Paper ↗

Reads from#3 Feeds into#6

Key code

rate <- rank_average_treatment_effect(eval.forest, priorities = tau.hat)
plot(rate)                          # TOC curve
rate$estimate / rate$std.err        # AUTOC z-stat

Discussion on this step (0)

StatsOtterDiscussion starter · Editorial note

Before using this step, make its assumptions and its output explicit. A useful peer note explains what you would check, what could go wrong, or what would change your interpretation.

Questions to explore

Which assumption matters most here, and what evidence could challenge it?
What would you check before passing this output to the next step?

Peer discussion

Bring your experience, a question, or a different perspective.

Data contract

Inputs & data

A fitted causal forest plus a held-out evaluation sample (X, W, Y).

Show data format & exampleHide example

Format — one row per unit. A covariate matrix X (numeric), a binary treatment W ∈ {0,1}, and an outcome Y.

  X1     X2    X3    W    Y
 0.42  -1.1   0     1   3.10
-0.07   0.6   1     0   1.85
 1.20   0.3   0     1   4.02

Read the evidence

Results & interpretation

4 figures Workflow discussion 2

01FigureA sample TOC curve: the average benefit as you treat the highest-ranked fraction of units.

02FigureAUTOC is the shaded area under the TOC — one number for how much heterogeneity you can exploit.

03FigureEvent times in the SPRINT/ACCORD trials, showing the right-censoring the estimator has to handle.

04FigureTOC curves estimated across trials — does the same prioritisation rule transport?

Figures reproduced from grf — Athey, Tibshirani & Wager — unofficial community showcase; all credit to the original authors.

Context & interpretation

The GRF 'Assessing heterogeneity with RATE' tutorial. Fit on one half, evaluate prioritization on the other; report AUTOC (rewards concentrated heterogeneity) and Qini (rewards broad effects) with CIs. Unofficial summary.

Peer notes

Discuss the workflow (2)

3

@rate_rosa · Jun 29, 2026

Running both AUTOC and Qini side by side is the right move — they answer different targeting questions and disagreeing is informative.

2

@qini_quinn · Jun 29, 2026

Yep. If AUTOC fires but Qini doesn't, your benefit is concentrated in a small group.
3

@kfold_kenji · Jun 29, 2026

The independent train/eval split is non-negotiable here. Priorities have to be out-of-sample or RATE is optimistic.