Power

The Powerpower  isof a critical concept when planning or evaluatingclinical trial is the probability that the trial will find a radiology studydifference between groups if there is one. Power can be defined as the probability of a true positive trial result and is often written as:

• power = (1 -β)

Conventionallywhere β is the probability of missing a difference between groups. Most clinical trials aim for a power of 80% or an 80% chance of finding a difference if there is one. Using the definition of power above, this means that many clinical trials have a β of 20% or a 20% chance of missing a real difference, i.e., having a false negative trial result.

For Radiologists, a useful analogy is that power is set at 0.80-0.85. For radiologistslike sensitivity: both power and sensitivity are measures of the likelihood of finding something, it may be useful to thinka difference between groups in the case of power as being similar to sensitivity: power isand disease in the abilitycase of a diagnostic test.  

A variety of factors determine the power of a study including the size and variability of the effect under study, the level of alpha (α), and the sample size.

Effect size

The larger the effect under study, the easier it is to detectrecognise, and the higher the power. For example, the effect of a difference between two or more treatments / diagnostic studies if there reallynew chemotherapeutic agent that resulted in a 100% cure rate of a previously incurable cancer would be easy to identify.

 Effect variability

The less variable the effect under study is, the easier it is a differenceto recognise, and diagnostic test sensitivitythe higher the power. For example, the effect of a new weight loss agent that resulted in a uniform loss of 10 kilograms among all study participants would be easier to identify than one that resulted in a range of weight loss with a high standard deviation. For Radiologists, a useful analogy is the ability of a test to detect disease if it is present.

Concept

When reviewing a samplesignal-to-noise ratio: the noisier the data set, the meanless likely one is to recognise the signal.

Level of α

The effect of the value in questionlevel of the experimental populationα is likely different fromthe least intuitive factor affecting power. Recall that α is the overall population. In other words, after you do something to your experimental sample, you expectpre-determined threshold for rejecting the variable you're watching to change.

For instance, imagine you want to evaluate the size of the pancreatic duct after administering secretin. You think secretin will increase the size of the pancreatic duct. Pathologic data is your gold standard and using this technique, it's been shown that an average adult patient has a mean size duct (μnull hypothesis (H0) with a standard deviation (σ). Now you give a sample population a bolus of secretin and start counting...

The mean size of the pancreatic duct from the post-secretin experimental population (X) is greater than expected from the path data... but is this a real effect or is this just due to chance?

If we set the p-value to 0.05, then we know we have only a 5% chance of making a type I error (α)... the error of saying that the increase in size of the pancreatic duct is a real effect, when it really is not. Although not strictly identical, radiologists might find it useful to think of type I errors as false positives.

If our study does not show that the increase is significant to the 0.05 level, then we cannot believe that secretin made a difference, but we run the risk of a type II error (β)... the error of saying that there is no difference whenbetween groups, customarily set at .05. If the p-value of a study is below .05, then H0 is rejected; if the p-value of a study is above .05, the conclusion is that there reallyis insufficient evidence to reject H0. If power is. Again analogous to sensitivity, although not strictly identicalthen α is analogous to 1-specificity or the false positive rate. By setting α at .05, radiologists might findwe accept a false positive rate of 5%. If we decrease α to .01, we decrease our false positive rate to 1%, but in doing so, we increase the β or false negative rate, decreasing power. A decrease in α means that we are less likely to reject H0. This protects us from false positive results but will increase the number of times we fail to reject H0 when it usefulis incorrect, decreasing the number of true positives. On the other hand, increasing α makes it easier to thinkreject H0, increasing the number of type II errors asfalse positives and increasing the number of true positives and thus power.

Consider the 2 extremes. If α=0, H0 will never be rejected and all study results will be negative: there will be no false positives (good) but also no true positives (bad), resulting in a power of 0. If α=1, H0 will always be rejected and all study results will be positive: the only possible outcomes will be false positives (bad) and true positives (good), resulting in a power of 1. Increasing the number of negative results increases both true negatives and false negatives while increasing the number of positive results increases both true positives and false positives. More false negatives decreases power, more true positives increases power. 

A more intuitive analogy compares the level of α to the amount of evidence required to convict a defendant at trial. The question then becomeslower the α, how do we know we have enough peoplethe more evidence is needed to convict, resulting in our experimental groupfewer false positive convictions but also fewer true positive convictions, decreasing power. The higher the α, the less evidence is needed to show a difference if there were one? Thisconvict, resulting in more false positive convictions but also more true positive convictions, increasing power.

Sample size

Power increases as sample size increases. Increasing the sample size decreases the standard deviation which decreases variability. Because effect size and variability are dependent on biology and α is how you powera study effectively. Ifalmost universally set at .05, the sample size is too small (underpowered)the easiest clinical trial parameter for investigators to control.

Again, thenan analogy makes the riskeffect of a type II error increases.

You can imagine two tests for the pancreatic duct: one with 15 post-secretin patients and one with 115 post-secretin patients. Both may failsample size easier to meet the p-valuegrasp. A coin that landed heads on 3 straight consecutive tosses would be unlikely to raise suspicion of an unfair coin, but intuitively we know that the second test did30 or even 300 heads in a better job of trying to show a difference if there were one.  This is what we're trying to capture with the concept of power.

The key variables in powerrow are

• the size convincing evidence of the difference you expect betweencoin’s unfairness.

  Caveat

  It’s important to remember that this discussion of power only considers the two groups

  • the smaller the difference you expect between the experimentalinfluence of random factors and control group, the larger numberexcludes consideration of subjects you need to tease outother factors that small difference

  • an analogy that may help is the children's game "Where's Wally" or "Where's Waldo" where Wally/Waldo is the effect: if Wally were large, he would be easier to see

• the number of patients

• the variability of the data,influence clinical trial results such as captured by the standard deviationbias.

  • the more variable (noisy) the data, the more patients you will need to show a difference

  • again an analogy with the children's game "Where's Wally" or "Where's Waldo" may help: if Wally were all by himself (no variability), he would be easier to recognise

• the alpha cut off (usually 0.05)

Post hoc power analysis

The use of post hoc power analysis (i.e. calculating power after the study has concluded) is controversial as it is thought to be unreliable ².