Regulatory Science Virtual Symposium: “Make Informed Decisions: Key Statistical Principles to Clinical Trial Design” Session 2: Basic Statistical Principles: Validity and Sample Size (2022)

1. Outline
   1. Statistical Inference, Hypothesis Testing, Type 1 Error, Power, and Sample Size
   2. Randomization and Blinding
   3. Multiplicity in Clinical Trials
   4. Types of Endpoints
2. Hypothesis testing
   1. Start with a null hypothesis, which usually state that there are no differences between treatment groups
   2. P-value presents that the probability that observed data could have occurred by chance alone i.e., if the null hypothesis were true
      1. Small P-Value suggests that the null hypothesis is false
3. Statistical significance and clinical significance
   1. Statistical significance can simply mean that the difference is greater than zero but that does not necessarily equate to being clinically significant
4. Power and Sample Size
   1. 80 to 90% Power to detect a small but clinically meaningful effect
   2. Consider what treatment effect size do we want to detect
5. Randomization and Blinding
   1. Bias is a systematic error
      1. Blinding is a technique often utilized to reduce bias
   2. Random error is unpredictable i.e., innate differences between patients
6. Estimates and confidence intervals
   1. We want the results of the study to allow use to draw inferences about the population from the sample
   2. Power of a study is the sponsor’s risk, so it is up to the sponsor if they want to do a 90% power vs. with 80% power
7. Multiplicity
   1. Clinical trials often have multiple hypothesis tests to increase likelihood of chances of having significant P-value
   2. Example: PROMISE-1 Key Efficacy Results
      1. One can controlling Type 1 error rate for the experiment by applying a multiplicity correction
   3. More statistical tests will result in increased likelihood of Type 1 errors, which means it is very likely there will be at least one Type 1 error
   4. Clinical trials have numerous statistical tests, providing opportunities for a Type 1 error to occur
   5. Sources of Multiple Tests: multiple endpoints, multiple dose groups, multiple timepoints
8. Controlling the Experiment-Wise Error Rate
   1. Goal in the trials: requiring P < 0.05 ensures that the Type 1 error rate for a given test is <5%
9. Main Approaches for Adjusting for Multiplicity
   1. Bonferroni Method
   2. Improved Bonferroni Methods
   3. Fixed-Sequence Method
   4. Gate Keeping Approaches, Fallback Approaches, Combination Approaches, etc.
10. Regulatory Claims
    1. Draft guidance from FDA in January 2017 states that primary endpoints should be statistically significant before proceeding to the secondary endpoints
       1. This means claims cannot be established for secondary endpoints until statistical significance is shown for the primary endpoints
    2. Requires strong control of the Type 1 error rate to demonstrate additional effects
11. Adjusted P-Values
    1. Adjustments might be applied to statistical analyses when there are multiple P-values
12. Multiple Endpoints vs. Co-Primary Endpoints
    1. No multiplicity adjustment for co-primary endpoints, which are all the endpoints that must be statistically significant
       1. This situation might arise as a requirement from FDA, not necessarily the sponsor
13. Graphical Approach for Adjusting for Multiplicity
    1. Divide alpha into thirds (for each hypothesis test) and compare these values to each other
    2. Arrows inform what to do with that alpha level
14. Types of Endpoints
    1. Continuous
       1. Treatment effect measured as mean difference between groups
       2. Simple Analysis Approach: T-Test
    2. Ordinal
       1. Include rating scales i.e., happy to sad emotion faces in hospital pain rating scales
       2. Model-Based Analysis
    3. Dichotomous/Binary
       1. An example of these type of variables is in oncology studies, the endpoint or variable is the shrinkage of the tumor
       2. Simple Analysis Approach: Chi-Square Test
    4. Survival or Time-To-Event
       1. Most Common Analysis: Cox Regression
    5. Composite Endpoints
       1. Are continuous variables that are calculated into an overall score
       2. Kaplan-Meier Curves
15. Responder Analysis
    1. We want to see small differences that might be clinically significant, especially for continuous variables i.e., blood pressure, ACR 20 scale for rheumatoid arthritis, etc.
16. Clinical and Surrogate Endpoints
    1. Clinical endpoint is a measure of how a patient feels, functions, or survives versus, a surrogate endpoint is believed to predict clinical benefit and is often used in place of a clinical endpoint
17. Missing Data and Bias
    1. Most common reason for missing data is early discontinuation from the study (patients drop out of the study and ideally, would have liked to measure to the end of the study but can no longer do so)
    2. Provides an opportunity for bias to play a role
    3. Statistics techniques might be used to adjust for bias
18. Handling Missing Data
    1. No technique is perfect!
    2. Best as well as ideal scenario is to follow everyone enrolling in a clinical trial from start to finish
19. Thank You!

Accompanying text created by Annie Ly | RKS Project Administrator, SC CTSI  lyannie@usc.edu