How to read a small trial

Most claims about reflexology trace back to a small trial, and most small trials are read badly by everyone who cites them. The register keeps its reading method on the page so that its own citations can be checked against it.

The four checks

Size first. A trial of forty women split into two arms can show that something interesting happened to twenty people; it cannot carry a claim that reflexology works. Small samples produce dramatic results by chance, which is why the register flinches at percentages built on tiny groups.

Blinding second. A hands-on therapy cannot be properly blinded: the client knows she is being touched, and the reflexologist knows she is touching. The honest comparator is not sham reflexology, it is an equal hour of rest or attention, and trials that skip that comparison measure theatre as much as therapy.

Significance third. Statistically significant means unlikely to be pure chance; it does not mean large, important or useful. A result can clear the bar and still be a change a woman would not notice.

Replication last. One trial is a finding; two independent trials finding the same thing is the beginning of evidence. Reflexology literature is mostly singles.

Why abstracts overclaim

Abstracts are written to be read, and the cautious qualifiers live in the body. A trial that reports an improvement in sleep quality on a self-rated scale in thirty-two women becomes, by the time it reaches a practice website, proven to help pregnancy insomnia. The register's habit is to read the limitation paragraph before the conclusion, and to treat any site that cites only abstracts as citing adverts.

The claims that fail the checks

Certain claim-shapes recur so reliably that the register keeps them as patterns: the testimonial wearing a statistic, nine out of ten clients felt better, with no comparison group at all; the borrowed trial, where a study on massage is cited for reflexology as if the therapies were interchangeable; and the review cited for its abstract while its authors write, in the text, that the evidence is insufficient. Each fails a different check, which is why the checks are plural.

The register's own discipline

When this site cites a review, it cites the review's own grading of certainty, low where low is the word. When it cites a dated webpage, the practice's 2013 price list for instance, it cites it as a dated claim, not a fact about today. And where the register cannot verify something it does not print it. That is the whole method, and it is slower than believing things, which is the point.

A last note for readers who want to apply the method themselves: the four checks fit on a postcard, size, blinding, significance, replication, and they work on any therapy's literature, not only this one's. The register lends them freely because a reader who checks is the reader the register was built for.

The strength underneath

It is worth ending on what small trials are good for, because the method is not meant to produce cynics. Small trials are excellent at showing whether a therapy is pleasant, whether it is safe, and whether it is worth studying properly. On all three, reflexology in pregnancy does reasonably well. What they cannot yet show is the thing the marketing wants them to show. The register keeps that sentence underlined, because it is the difference between a therapy that is nice and a therapy that is proven.