APPENDIX FIVE
Statistical Considerations

The following observations pertain to confirmatory or pivotal trials. A pivotal trial should demonstrate the effect of a drug when given to a broad spectrum of patients with ALS.

A clinical trial usually requires a collaboration among many different groups of investigators at different centers. Trialists and /or statisticians can elaborate general principles for trial design but clinicians must help mould those principles for use in specific diseases such as ALS. Answers to important questions cannot be assigned to one group or the other but must be reached in collaboration.

What drives a trial is the question being asked. If a methodology is judged to be an efficient way of getting to the answer in a particular trial it may be appropriate in that trial even if is not appropriate in others.

Dropouts

All trials should be structured to keep dropouts to a minimum. All attempts should be made to obtain at least the primary outcome measure from all subjects entered into the trial. Both of these issues should be a consideration in the selection of a primary endpoint. Adjustment for dropouts is always problematic because the relation of dropouts to study medication is unknown. Reasons for dropout should always be noted in the protocol. Comparisons of baseline variables between dropouts and completers should be made to see if dropouts have any pattern and to determine if dropouts were tending in a direction different from those who completed the trial. If dropouts are truly random then all of the current weighting schemes should give similar results. Since these schemes all depend upon assumptions about the structure of the data it’s difficult to recommend one over the other for all situations. If dropouts occur for non-random reasons, it is more important to discover and adjust for these reasons than to provide a single statistic. Careful attention should be given to collecting baseline data for comparison of dropout and non-dropout groups.

As a specific recommendation, last observation carried forward is probably not appropriate for ALS trials. The analysis should be "intent to treat".

Stratification

Since randomization is not perfect it is often necessary to make sure that key covariates are balanced between the arms of a trial by stratifying on these variables and then conducting blocked randomization within strata. Clinical sites are almost always one of the strata. Other possible strata depend upon the disease and the impact of the covariate on the outcome. Covariates which should be considered for strata are those which are highly predictive of the outcome measure being used. The predictiveness of the strata variable should be determined in advance and we recommend that when the results of any trial have been published, the data (particularly from the control group) should be made available for analysis of predictive factors. At this time, site of onset, age, and FVC are all variables in ALS that are worth considering as separate strata in future trials. The number of stratification factors should be small (at most 3 to 4) and it is more efficient to have independent variables as strata indicators. Very predictive variables are those with relative risks of 2 or 3. The strata need to be specified in advance and they are part of the primary analysis. If the analysis is not adjusted to account for stratification, the power may be less than what would be expected from simple randomization.

It is important to recognize the distinction between Predictors and effect modifiers. An effect modifier is a clinical feature that results in different drug effects in different subgroups. When pre-trial evidence indicates that the effect of the experimental treatment is likely to differ across subgroups, separate studies in subpopulations should be considered. If this is not the case and post-hoc explanatory subset analysis is done, these hypothesis-generating results will require validation with independent data sources.

Adjustment for Prognostic Factors or Other Covariates

The basic reason for planned covariate adjustment is to reduce the risk of confounding, to increase precision, and to account for structure imposed by stratification at randomization. While post-hoc adjustment provides useful insights, a significant result that occurs only after adjustment is less compelling than finding a result using a priori design techniques. Any adjustment "discovered" during the trial or on the basis of modeling conducted during the trial is still a discovered result not a pre-planned confirmation. We encourage investigators to engage in post-hoc analysis for the purpose of learning more about disease and its treatment. Results from this experience should be considered as hypothesis generating and not hypothesis confirming.

Adjustment with pre-specified variables can be viewed in the same light as stratification, with similar caveats.

Analytical Methods

The general issue is that an analysis appropriate to the question being asked, the outcome measures chosen, and the adjustments made, must be specified in advance of the trial. Clinical difference to be detected, power, and sample size also must be pre-specified and considered in light of achieving maximum efficiency for the trial. Cox is a perfectly valid analysis procedure and can be used with covariates, subject to the discussion above, in time to event situations where data may be censored.

Use of the Cox model assumes proportional hazards. This should need be examined during the study design phase, not post hoc.

Time to Failure Variables

Almost any clinical variable can be an event, and a time to achieve it can be measured. It should be a clinically meaningful event or surrogate measure, which can be precisely, unambiguously, and reliably measured.

Lead-ins

Lead-ins are a method of specifying inclusion, exclusion or stratification criteria for the trial. They should be judged that way, i.e. do they provide sufficient reduction in variability to overcome the loss of generalizability they produce and the effort to implement them. Further study of lead-ins is needed in ALS Clinical Trials.

Surrogate

A better understanding is needed by ALS clinical trialists about what a surrogate measure is, how it is to be used, and how we can determine whether a given variable is or is not a good surrogate measure. A similar therapeutic response between a putative surrogate and the outcome measure in a therapeutic trial is necessary to establish it as a surrogate.

Back to Airlie '98 Contents