The recovery protocol looks good because the drop-outs left

Adherence and outcome are tangled, so a programme measured on its finishers is measuring the finishers, not the programme.

Twenty-six players started the protocol. Nineteen completed it, and the nineteen had a good season. The protocol goes into next year's plan.

What separated the nineteen from the seven? Almost never something random. The finishers tend to be the ones who were not repeatedly injured, not dropped, not away with a national squad, not in a run of games so congested that the extra sessions became impossible. In other words, the people who completed the recovery programme are disproportionately the people whose seasons were already going well, and their good outcomes were partly the cause of their adherence rather than the result of it.

The arrow points backwards and it is very hard to see from inside. Somebody who feels good turns up to the optional bike session. Somebody carrying a sore calf skips it, then skips the next one, and by week five has left the analysis. Strip out the strugglers and any programme in the world will look effective.

There is a second layer here that I think is underrated. Adherence is itself a personality measurement. The athletes who complete voluntary protocols are the ones who sleep on schedule, eat consistently and go to bed instead of out. All of those habits affect the outcome you are measuring, and none of them were caused by your ice baths. You have built a filter that selects conscientious professionals and then credited the filter with their results.

The way out is uncomfortable and cheap. Analyse everyone who started, including the seven, with their real outcomes attached. That number is the one that answers the question a director actually asked, which was what happens if we roll this out, not what happens to people who complete it. If the effect survives the seven, it is probably real.

And record why each drop-out left, in a sentence, at the time. Reconstructed reasons collected in June are stories rather than data.

I would go further and treat a low completion rate as a finding about the protocol rather than a nuisance in the data. If a third of the squad cannot fit it into their week, that is information about the design of the thing, and it will not be improved by an analysis that quietly removes the people who told you so.