– Meta-epidemiological assessment of the quality of evidence in clinical practice guidelines.
– https://doi.org/10.1101/2025.08.11.25333132.

– About 60% of decisions adhere to guidelines, 25% of strongest guideline recommendations are based on RCTs, and 40% of these RCTs have serious deficits
– 90% of decisions are not based on solid evidence
– https://doi.org/10.31219/osf.io/cv687_v2.

– Basic competence to search for biomedical evidence
– Population + Intervention search approach
– Cochrane Library, PubMed and Prospero for SR
– PubMed, Google Scholar, or CENTRAL, and WHO ICTRP for trials
– https://osf.io/preprints/osf/bpxfv_v1

– Historical lessons of redundant, wasteful trials
– Protocols and trial reports frequently do not cited SRs as justification
– Dearth of stakeholder requirements
– https://osf.io/xmsjv_v1.

-Traditional systematic reviews are time and resources demanding. Reviews in the planning phase can be done differently.
– Use published reviews as basis and search WHO ICTRP. Analyse only outcomes of interest for conducting new trial
– https://doi.org/10.31219/osf.io/ves7d_v1

– Overview of educational material on how to justify, design, conduct, analyse and share RCTs.
– https://www.medrxiv.org/content/10.1101/2025.05.16.25327604v1

– An introduction to CTU Blended Learning Open Course 1 on randomised clinical trials.
– Highlighting the purpose of the course (to stop research waste and make more informative trials) and that existing open educational resources have gaps.
– Justification of new trials and the thematic use of systematic reviews as foundation for justification and informing the design.

– The conclusiveness of a review is more than statistical significance or not
– Three-step approach: relevant outcome, reliable evidence, and quantitative analysis of the effect estimate
– Multiple methods for quantitative analysis: cumulative meta-analysis, trial sequential analysis, prediction intervals.

– Use systematic reviews to inform the choices of intervention, comparators, and outcomes.
– Anticipated number of events, duration, and sample size
– Methodological and practical limitations

Classical trial designs

– Parallel group or crossover designs
– Special emphasis on the factorial designs (answering multiple questions in one trial)
– Advantages and disadvantages of all three designs

Superiority, equivalence, non-inferiority trials

– Important differences and limitations to the various designs
– Garattini (2007) Lancet viewpoint on non-inferiority (https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(07)61604-3/abstract).

Choose the population

– What is the clinical question?
– Considerations about inclusion/exclusion criteria
– Restricted (mechanistic) versus broad criteria (pragmatic trial)
– Examples from literature, e.g. psychiatry (Zimmerman 2002, https://pubmed.ncbi.nlm.nih.gov/11870014/ and Zimmerman 2016, https://pubmed.ncbi.nlm.nih.gov/27541608/).

Choose the experimental intervention

– Consider intervention, co-interventions, regimes, formulations, generics, dose, duration, etc.

Choose comparators

– What is the best standard of care according to Pre-Trial SRs?
– Careful to not default to assumptions on the comparator
– Examples of subpar comparators, either placebo (in the presence of effective treatments) or subpar treatments

What is a trial protocol

– Overview of the various components of a trial protocol
– What is essential and needs elaboration and what is less important at the pre-trial stage
– Major mistakes in trial protocols

Introducing Blended Learning Open Course2

– – Overview of BLOC 2

Choose ‘must give’ standard treatments

– What is the best standard of care according to Pre-Trial SRs?

Choose outcomes

– Is there a Core Outcome Set for the indication?
– Short introduction to COMET
– Careful to not default to assumptions on the comparator
– Distinction between patient-reported (PROMs), surrogate/biomarker, hard/functional outcomes and compositive outcomes

Unvalidated and validated surrogate outcomes and biomarkers

– Dangers of the abundance of unvalidated surrogate outcomes and biomarkers
– John Ioannidis.

Trial duration and time of outcome measurements

– What are the expected event proportions depending on the primary outcome?
– Gain this information from the pre-trial SR
– Duration should reflect clinical course based on relevant outcomes
– Examples of fields with short trials, lacking long-term evidence, e.g. psychiatry

Historical development of sample size estimation

– Article submitted to James Lind Library.
– Kristian Thorlund.

Sample size formulas for frequentist randomised clinical trials

– Which formulas to use
– Kristian Thorlund and Robin Christensen.

Sample size estimation

– How large should the trial be to demonstrate an effect of the tested intervention
– Depends on the anticipated number of events (gained from pre-trial SR)
– Simple (predictor analysis, Chi2) vs non-simple statistics (SAT – involve stratifiable variables)
– Note that industry and other researchers often enrol surplus of patients based on expected missing data
– Trials has a whole collection of published articles on how to calculate sample size: https://www.biomedcentral.com/collections/randomizedtrialsamplesize

Involvement of patients and the public in trial design

– What is PPI and why does it matter
– Evidence on PPI’s impact on research waste?
– https://ncto.ie/wp-content/uploads/2022/10/Brett-PPI-Impact-on-Research-SystematicReview-2012.pdf
– How to facilitate it? Recruiting patients, engaging them, incorporating feedback.
– See landscape analysis (EUPATY) for resources

Anticipating methodological challenges

– Based on pre-trial SR, what were the main limitations to the evidence
– Problems, e.g. with the blinding (drug’s subjective effects, or logistical problems), low adherence, many dropouts, or low event rates?
– What were the main sources of bias or reduced generalisability in previous trials, e.g. strict inclusion criteria, inferior control intervention, irrelevant outcomes.
– Jane Lindschou and Janus Engstrøm.

Detailed statistical analysis plan

– The prespecification of analytical methods
– Exhaustive considerations of analyses
– Different from trial registration and from the trial protocol itself
– A few examples of published SAPs, e.g. SafeBoosc III

Decentralised trial designs

– See Science paper
https://www.science.org/doi/10.1126/science.adq4994

Advanced trial designs

– Short overview of other more recent (and hyped) designs
– Adaptive design, platform trials, basket, umbrella
– New designs in oncology, e.g.Janiaud (2019)https://pubmed.ncbi.nlm.nih.gov/30572165/.

Platform trial designs

– Principles of a platform trial
– Stampede trial: the world’s first platform trial
– www.mrcctu.ucl.ac.uk/studies/all-studies/s/stampede/
– How does it differ from a regular trial
– What are the advantages
– What are the disadvantages

Introducing Blended Learning Open Course3

– Overview of BLOC 3

The interactions of governance, planning, infrastructure, and organisation of a randomised clinical trial

– The lay of the land – drafted.
– Sanam Safi.

Sources of waste in the planning of new trials

– Barriers of common sources of research waste: unjustified and uninformative trial design, lack of funding, insufficient patient accrual (insufficient sites), legal delays (collaborator contracts and regulatory approvals)
– Barriers to trials. Djursic 2017; https://link.springer.com/article/10.1186/S13063-017-2099-9)
– Obstacles in multinational RCTs (Hout 2025, https://doi.org/10.1001/jamanetworkopen.2025.18503)
– ACTIVE trial as case study: https://pmc.ncbi.nlm.nih.gov/articles/PMC10022634/
– How to create the necessary research infrastructure for successful and valid results of randomised clinical trials.

Funding a clinical trial

– Finances of a clinical trial (pro bono, fully funded, partially funded)
– Public versus industry sponsorship
– Budget templates (see landscape analysis)

Trial IT infrastructure/eCRF

– How is data captured and recorded during the trial?
– The electronic case report form; dos and don’ts.
– The data management plan (see landscape analysis)

Randomisation

– The various types of randomisation generation (simple, block, stratified, covariate) and allocation
– Various overviews: Suresh (2011)https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3136079/; Broglio (2018) https://jamanetwork.com/journals/jama/article-abstract/2683203
– Altman (1999) How to randomize. A classic. Must be used as template. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1116549/
– Altman (1999). https://www.bmj.com/content/318/7192/1209.1.long (another classic on treatment allocations)

Quality assurance

– Three components: SOPs, training, and delegation logs
– Standard Operating Procedures for everything (list a few examples, refer to the Norwegian Methods Handbook)
– Pre-trial training of staff and investigators (SafeBoosC III as example)
– Delegation lists and responsibilities

Assembling trial committees

– Overall trial hierarchy: executive committee, steering committee, site coordinators, other collaborators
– Trial steering committee; plan, purpose, pros and cons.
– Data safety monitoring committee; plan, purpose, pros and cons.

Successful collaboration in multi-centre trials

– What is a collaborator in an RCT= other site enrolling patients.
– Leverage to attract collaborators: publication, prestige, money.
– Problems: different views on topics, contracts, tension.
– Highlight evidence or SWATs to have investigated successful collaborations.

Governance – regulatory and ethics approval

– The purpose of regulatory and ethics approval
– Main roadblocks
– References to templates and ‘how to’ based on landscape analysis
– Pointing out pitfalls of current system and lack of methodological assessments
– Jane Lindshou.

Trial duration and time of outcome measurements

Introducing Blended Learning Open Course4

– Next stage of the trial starts with enrolment of the first participant
– Preparation of the logistical challenges, addressing bureaucratic roadblocks, anticipating problems

The interactions of governance, planning, infrastructure, and organisation of a randomised clinical trial

– What is GCP and why it is needed?
– Highlight evidence assessing its impact on research waste.
– Cite resources to take online courses.

Regular central data monitoring (Quality control part 2)

– The principles of data monitoring: completeness, integrity of data, patterns of missingness
– On-site versus central data monitoring
– Examples of trials employing this principle

Independent data monitoring and safety committee

– What it is, their composition, and what a charter is
– Stopping rules and the risk of early stopping
– Adjudication committees

Interim analyses

– The purpose of interim analyses compared to final (end of last visit) analyses
– Risks and considerations for early stopping after interim-analyses: Liu (2022) https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-022-06689-9.
– Evidence-based stopping rules to prevent futility and premature early stopping (refs for that?)
– Guyatt – The dangers of stopping trials early (https://www.bmj.com/bmj/section-pdf/187582?path=/bmj/345/7865/Analysis.full.pdf).

Challenges with recruitment and retention

– Recruitment and retention are main reasons for research waste
– Refer to Trial Forge database

How to communicate during a trial

– Regular updates on randomisations
– Regular updates on central data monitoring

We lack a theme here?

Last patient, last visit: closure of a trial

– Checklist of what needs to be done
– How to best prepare for the analysis and reporting stage?
– Roadblocks or legal issues?
– See landscape analysis (several resources had plenty of material on this topic)

Introducing Blended Learning Open Course5

– How to analyse the data correctly
– The importance of prespecifying analytical methods

Data preparation:from raw data to analytical dataset

– Pathway from extracting raw data to manageable dataset
– Data cleaning equalsharmonisation, standardisation, and many (undocumented) analytical choices. May hamper the integrity, reproducibility and transparency.
– Currently no guidelines on how to do this, no prespecification in the protocols, and no GCP requirements

Statistical software bias ▬ a well-known and underreported bias(MHO)

– The reproducibility crisis is multifaceted whereas statistical software bias is among the culprits. Simple analyses show varying results when analyzing identical datasets in different statistical software. The aim of this paper is to show how reanalysis of previously carried out trials with the most predominantly used statistical software can result in different results – i.e. exemplify statistical software bias.

Analytical recommendations for randomized clinical trials ▬ a brief overview (MHO)

– Randomised clinical trials are being analysed using a multitude of different statistical analyses. However, all analyses differ including assumptions, power, and most of all interpretation vary. Here, we aim to provide a brief overview using a decision flowchart of suggested statistical analysis and a practical guide on best practice and pitfalls. All these based on the type of outcome and design of the randomized clinical trial.

. Protocol for investigations of statistical analysis between different statistical software ▬ a standardised methodology (MHO)

– A standardised methodology to investigate the optimal analysis can minimize the potential influence from software bias. This protocol aims to provide a standardized methodology on how to investigate differences, and, further, to recommend specific analysis how the optimal analysis. The methodology can be used in generic investigations of software bias and as part of statistical analysis plans before analyses are carried out. The specific analysis will be graded as ‘recommended’, ‘somewhat bias’, and ‘not recommended’

Investigation of statistical analyses of dichotomous outcomes in randomised clinical trials (MHO)

– Using the standardised methodology from PAPER 3, this paper will investigate different methods for analysing dichotomous outcomes in randomised clinical trials, including logistic regression with strata as fixed effect and logistic regression with strata as fixed effect and site as random effects.

Investigation of statistical analyses of continuous outcomes in randomised clinical trials (MHO)

– Using the standardised methodology from PAPER 3, this paper will investigate different methods for analysing continuous outcomes in randomised clinical trials, including linear regression with strata as fixed effect and linear regression with strata as fixed effect and site as random effects.

Investigation of statistical analysis of skewed continuous outcomes in randomised clinical trials (MHO)

– – Using the standardised methodology from PAPER 3, this paper will investigate different methods for analysing skewed continuous outcomes in randomised clinical trials, including Van Elteren’s test.

Investigation of statistical analysis of ordinal outcomes in randomised clinical trials (MHO)

– – Using the standardised methodology from PAPER 3, this paperwill investigate different methods for analysing ordinal outcomes in randomised clinical trials, including proportional odds logistic regression

Investigation of interaction for subgroup analysis in randomised clinical trials (MHO)

– Using the standardised methodology from PAPER 3, this paper will investigate different methods for analysing interaction in randomised clinical trials, including quantile, linear and logistic regression.

Investigation of statistical analyses of time-to-event outcomes with unequal variance in randomised clinical trials (MHO)

Sample size and power calculations using simulations for randomised clinical trials ▬ analysis specific calculations (MHO)

– Based on the standardised methodology, this paper will investigate the limitations of the standard sample size analysis using t-test or chi square compared to simulation based methods, and provide recommendations on when each method should be chosen

Missing data

– An overview of various types of missing data: trial level (publication bias), outcome level (selective reporting), site level, patient level, patient-outcome level
– Methods for dealing with missing data: Last observation carried forward, best/worst case scenarios, multiple imputations methods
– The impact of missing data on the reliability of trial results?
– Evidence in support of any of these methods?
– R-package on missing data: https://cran.r-project.org/web/packages/micemd/index.html.

Minimal important difference vs statistical significance

– The conceptual difference between hypothesis testing and statistical significance and the concept of minimal important difference.
– A statistically significant result may not be clinically relevant.
– Example of antidepressants for adults, Munkholm (2019) https://pubmed.ncbi.nlm.nih.gov/31248914/.
– Discussion whether to abolish p-values (https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1447512#abstract).

How to update the pre-trial SR with any new data

What is data sharing

– Key concept of sharing data, types of data, and FAIR principles.
– Based on overview article by Tai et al. (2025) https://www.jclinepi.com/article/S0895-4356(25)00253-7/fulltext

Status quo of data sharing requirements in journals, registries, and funders

– Funder requirement, e.g. DeVito (2018) (https://pubmed.ncbi.nlm.nih.gov/29710154/).
– Legal obligations in EU and FDAA 2007
– Journals also require data sharing statement – not to confuse with a requirement, and BMJ now require IPD for all published trials (https://www.bmj.com/content/384/bmj.q324).
– Empirical assessments of data sharing; Danchev (2021) survey https://pubmed.ncbi.nlm.nih.gov/33507256/and Ohman (2021). Sharing of IPD – scoping review. https://pubmed.ncbi.nlm.nih.gov/34408052/

How and where to publish your oral and written results

– Trial registries, journals, other sources
– International and national meetings
– Choice of journal
– More
– John Ioannidis

How to prepare and share your data – aggregate data

– Group level data can be shared on trial registries (note the WHO Outcome Set) and in traditional journal publications
– A few references to highlight how frequent selective reporting and outcome switching occur and that it has not really improved, e.g. COMPARE project (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3173-2) and Dwan (2014) https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001666.
– Other platforms: Vivli, YODA, Clinical Study Data Request, Project Data Sphere (https://data.projectdatasphere.org/projectdatasphere/html/home)
– Practical recommendations: FAIR use, specification of meta-data, and adherence to protocol – selective reporting/outcome switching
– ECRIN’s data sharing report

How to prepare and share your data – IPD

– Challenges to sharing IPD: risk of deanonymisation and identification
– Empirical assessments of the risks and barriers?
– Models for sharing IPD: sharing upon request (gatekeerp model), third-party platform, unrestricted access. Ventresca et al. (2020) managing IPD for meta-analyses (https://pubmed.ncbi.nlm.nih.gov/32398016/).
– Existing guidance documents on how to share data
o Rodriguez (2022). Scoping review of recommendations (https://pubmed.ncbi.nlm.nih.gov/35730910/).
o Keerie (2019). Practice guidance on how to share RCT data (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-017-2382-9).
o IOM report (2015). https://pubmed.ncbi.nlm.nih.gov/25590113/
o Smith (2015). Sharing IPD (https://pubmed.ncbi.nlm.nih.gov/26675031/)
o Smith (2019). Preparing IPD (https://pubmed.ncbi.nlm.nih.gov/28712359/).
– Concrete examples on how to do it or examples of people who have successfully shared their data (see the Smith articles above)
– Overview of existing clinical trial sharing platforms
o Wilson (2021). Assessment of 7 platforms. https://jmla.pitt.edu/ojs/jmla/article/view/992

Types of research data

– An overview of the granularity of clinical trial data (aggregate summary versus IPD)
– Aggregate group level data. Brief overview.
– Individual patient data. Brief overview.

Introducing Blended Learning Open Course 6

– Ethical obligation and research waste
– Practical limitations, ethical concerns, lack of standards

Proposals for improving data sharing

– No common meta-data framework for clinical trial data
– Journals do not advocate for easy data sharing
– Data sharing is not prioritised
– Solution: WHO / Zenodo should organise a simple meta-data framework which can be uploaded to trial registries and on journal websites.