1. Agenda
a. Defining evidence
b. Need for evidence
c. Type of evidence
d. Validating evidence
e. Case studies
2. Evidence is the persuasive argument
a. Guide regulatory decision making which leads to action
3. Driver: Research question
a. Research question drives the evidence
4. Digging deeper: generating data
5. Research question
a. Provides the framework for study design, type of data, and analysis plan
6. Finding Data
a. RCT data, publications, case studies, and real-world data make up the analysis plan
7. Research designs that generate evidence
a. Randomized controlled trials (RCT)
i. Sham: pretend to provide the treatment as the control vs. the group that receive treatment
ii. There is some extent of biases with historical data
iii. Propensity scores are utilized to reduce the biases and baseline differences among the individuals
b. Alternative research trail designs
i. Observational study – from real world data over time, no intervention
ii. Registries – within a real-world setting
iii. Pragmatic trial – hybrid of interventional studies and real world evidence by using the intervention as part of real-life routine practice
iv. Meta-analysis of published results
v. Case studies
8. RCT Issues
a. If the gold standard not the right design because it is difficult to blind subject and investigator i.e. devices requiring invasive surgeries
b. Ethics
c. Is there a comparator?
9. Real world data (RWD)
a. Real world data becomes real world evidence because it is more generalizable
10. Pause: Let’s review terms
a. Real world data (RWD)
b. Real world evidence (RWE)
c. Routine clinical data (RCD)
d. Pragmatic design
e. Observational design
f. Propensity score
11. Quality of evidence
a. Quality of evidence depends on quality of data
12. Validity of evidence
a. Does your evidence answer your research question?
b. Internal validation
c. External validation
d. Higher validation = strong evidence
e. RCT decreases bias and generalizability
f. RWD increases bias and generalizability
13. Internal Validity
a. Minimizing bias increases internal validity
b. Selection bias – Randomization, stratification, blinding
c. Performance bias – Blinding
d. Detection bias – Prior analysis plan, monitoring, appropriate analyses of data interpretation of results
e. Attrition bias - ITT (intended to treat), per-protocol,
14. Case study: Mitraclip
15. Case study: Senhance Surgical System
16. Summary Points to Consider
a. Research question determines design/data
b. Quality of data impacts quality of evidence
c. Internal/external validity impacts validity of evidence
d. RWE is different, not inferior to RCT evidence
e. RCT aims to show technology works vs. RW studies show if technology works in a clinical setting
17. Questions
a. If you change your analysis plan, do so prior to closing the studies and conducting the analysis plans
i. Use document control and version control and be ready to provide explanations for these changes
