Andrew Gelman started a great discussion about interaction testing in randomized controlled trials (RCTs), i.e. the search for subgroup effects of treatment.
I commented as follows: "Did anyone really simulate the case study as suggested? Admittedly, estimates of effect sizes are key; and these impact on the estimated SE in a model, correct?"
Under the Null, main effect has SE 0.63; interaction has SE of 1.26, with N=1000, sigma 10.
With the given example, main effect 2.8*sigma; for x2== -.5, 2.1*sigma, and for x2== .5, 3.5*sigma. This is the implication of the interaction being half the size of the main effect (1.4; -.7 and +.7 effect).
This results is 3 interesting findings:
SE of main effect is estimated larger (as 0.70)
SE of main effect is estimated as 0.66 if we adjust for x2
SE of interaction is estimated correctly as 1.26.
My reflections:
The only correct model is the model with interaction; and there the SE is identical to what was derived under the Null (SE 1.26).
In practice, we will start with the main effect model, and some variance is explained by adjusting for other covariates that are associated with the outcome. This is indeed what we observed for x2, even if x2 is interacting with x1. So, this confirms the recommendation to include covariates that are associated with the outcome, more than searching for subgroup effects.
The inclusion of prognostic covariates is beneficial in linear models as well as in generalized linear models such as logistic or Cox regression, for example for mortality after an acute MI