Research tools and issues

How to write statistical analysis section in

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
medical research
Alok Kumar Dwivedi ‍ ‍

► Additional supplemental ABSTRACT reporting of statistical analyses in studies.2 3

material is published online Reporting of statistical analysis is essential in any Three things are required for statistical anal-
only. To view, please visit
the journal online ([Link] clinical and translational research study. However, yses, namely knowledge of the conceptual
doi.​org/​10.​1136/​jim-​2022-​ medical research studies sometimes report statistical framework of variables, research design, and
002479). analysis that is either inappropriate or insufficient evidence-­based applications of statistical anal-
Department of Molecular to attest to the accuracy and validity of findings and ysis with statistical software.4 5 The concep-
and Translational Medicine, conclusions. Published works involving inaccurate tual framework provides possible biological
Division of Biostatistics and statistical analyses and insufficient reporting and clinical pathways between independent
Epidemiology, Texas Tech influence the conduct of future scientific studies, variables and outcomes with role specification
University Health Sciences of variables. The research design provides a
including meta-­analyses and medical decisions.
Center El Paso, El Paso,
Texas, USA Although the biostatistical practice has been protocol of study design and data generation
improved over the years due to the involvement of process (DGP), whereas the evidence-­ based
Correspondence to statistical reviewers and collaborators in research statistical analysis approach provides guidance
Dr Alok Kumar Dwivedi, studies, there remain areas of improvement for for selecting and implementing approaches
Department of Molecular transparent reporting of the statistical analysis after evaluating data with the research design.2 5
and Translational Medicine. Ocaña-­Riola6 reported a substantial percentage
section in a study. Evidence-­based biostatistics
Division of Biostatistics &
Epidemiology., Texas Tech practice throughout the research is useful for of articles from high-­impact medical journals
University Health Sciences generating reliable data and translating meaningful contained errors in statistical analysis or data
Center El Paso, El Paso, data to meaningful interpretation and decisions in interpretation. These errors in statistical anal-
Texas, USA; medical research. Most existing research reporting yses and interpretation of results do not only
a​ lok.​dwivedi@t​ tuhsc.​edu
guidelines do not provide guidance for reporting impact the reliability of research findings but
Accepted 1 June 2022 methods in the statistical analysis section that also influence the medical decision-­ making
Published Online First helps in evaluating the quality of findings and data and planning and execution of other related
16 June 2022 interpretation. In this report, we highlight the global studies. A survey of consulting biostatisticians
and critical steps to be reported in the statistical in the USA reported that researchers frequently
analysis of grants and research articles. We provide request biostatisticians for performing inap-
clarity and the importance of understanding study propriate statistical analyses and inappropriate
objective types, data generation process, effect size reporting of data.7 This implies that there is a
use, evidence-­based biostatistical methods use, and need to enforce standardized reporting of the
development of statistical models through several statistical analysis section in medical research
thematic frameworks. We also provide published which can also help rreviewers and investiga-
examples of adherence or non-­adherence to tors to improve the methodological standards
methodological standards related to each step in of the study.
the statistical analysis and their implications. We Biostatistical practice in medicine has been
believe the suggestions provided in this report can improving over the years due to continuous
have far-­reaching implications for education and efforts in promoting awareness and involving
strengthening the quality of statistical reporting and expert services on biostatistics, epidemiology,
biostatistical practice in medical research. and research design in clinical and translational
research.8–11 Despite these efforts, the quality
of reporting of statistical analysis in research
studies has often been suboptimal.12 13 We
INTRODUCTION noticed that none of the methods reporting
Biostatistics is the overall approach to how documents were developed using evidence-­
we realistically and feasibly execute a research based biostatistics (EBB) theory and practice.
idea to produce meaningful data and translate The EBB practice implies that the selection
© American Federation for data to meaningful interpretation and deci- of statistical analysis methods for statistical
Medical Research 2022. sions. In this era of evidence-­based medicine analyses and the steps of results reporting and
Re-­use permitted under and practice, basic biostatistical knowledge interpretation should be grounded based on the
CC BY-­NC. No commercial becomes essential for critically appraising evidence generated in the scientific literature
re-­use. Published by BMJ.
research articles and implementing findings for and according to the study objective type and
To cite: Dwivedi AK. better patient management, improving health- design.5 Previous works have not properly eluci-
J Investig Med care, and research planning.1 However, it may dated the importance of understanding EBB
2022;70:1759–1770. not be sufficient for the proper execution and concepts and related reporting in the write-­up

Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479 1759

 Research tools and issues
of statistical analyses. As a result, reviewers sometimes ask the reporting of the following components in the statistical

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
to present data or execute analyses that do not match the analysis section of the study.
study objective type.14 We summarize the statistical analysis
steps to be reported in the statistical analysis section based Step 1: specify study objective type and outcomes
on review and thematic frameworks. (overall approach)
The study objective type provides the role of important
METHODS variables for a specified outcome in statistical analyses
We identified articles describing statistical reporting prob- and the overall approach of the model building and model
lems in medicine using different search terms (online reporting steps in a study. In the statistical framework, the
supplemental table 1). Based on these studies, we prioritized problems are classified into descriptive and inferential/
commonly reported statistical errors in analytical strategies analytical/confirmatory objectives. In the epidemiological
and developed essential components to be reported in the framework, the analytical and prognostic problems are
statistical analysis section of research grants and studies. broadly classified into association, explanatory, and predic-
We also clarified the purpose and the overall implication tive objectives.19 These study objectives (figure 1) may be
of reporting each step in statistical analyses through various classified into six categories: (1) exploratory, (2) associa-
examples. tion, (3) causal, (4) intervention, (5) prediction and (6) clin-
ical decision models in medical research.20
RESULTS The exploratory objective type is a specific type of deter-
Although biostatistical inputs are critical for the entire minant study and is commonly known as risk factors or
research study (online supplemental table 2), biostatis- correlates study in medical research. In an exploratory
tical consultations were mostly used for statistical anal- study, all covariates are considered equally important for the
yses only15. Even though the conduct of statistical analysis outcome of interest in the study. The goal of the exploratory
mismatched with the study objective and DGP was identified study is to present the results of a model which gives higher
as the major problem in articles submitted to high-­impact accuracy after satisfying all model-­related assumptions. In
medical journals.16 In addition, multivariable analyses were the association study, the investigator identifies predefined
often inappropriately conducted and reported in published exposures of interest for the outcome, and variables other
studies.17 18 In light of these statistical errors, we describe than exposures are also important for the interpretation

Figure 1 Comparative assessments of developing and reporting of study objective types and models. Association measures include odds
ratio, risk ratio, or hazard ratio. AUC, area under the curve; C, confounder; CI, confidence interval; E, exposure; HbA1C: hemoglobin A1c; M,
mediator; MFT, model fit test; MST, model specification test; PI, predictive interval; R2, coefficient of determinant; X, independent variable; Y,
outcome.
1760 Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479
 Research tools and issues
and considered as covariates. The goal of an association Examples

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
study is to present the adjusted association of exposure A study26 showed a favorable effect of the beta-­ blocker
with outcome.20 In the causal objective study, the investi- intervention on survival outcome in patients with advanced
gator is interested in determining the impact of exposure(s) human epidermal growth factor receptor (HER2)-­negative
on outcome using the conceptual framework. In this study breast cancer without adjusting for all the potential
objective, all variables should have a predefined role (expo- confounding effects (age or menopausal status and Eastern
sures, confounders, mediators, covariates, and predictors) in Cooperative Oncology Performance Status) in primary
a conceptual framework. A study with a causal objective is analyses or validation analyses or using a propensity score-­
known as an explanatory or a confirmatory study in medical adjusted analysis, which is an EBB preferred method for
research. The goal is to present the direct or indirect effects analyzing non-­ randomized studies.27 Similarly, another
of exposure(s) on an outcome after assessing the model’s study had the goal of developing a predictive model for
fitness in the conceptual framework.19 21 The objective of an prediction of Alzheimer’s disease progression.28 However,
interventional study is to determine the effect of an inter- this study did not internally or externally validate the
vention on outcomes and is often known as randomized or performance of the model as per the requirement of a
non-­randomized clinical trials in medical research. In the predictive objective study. In another study,29 investiga-
intervention objective model, all variables other than the tors were interested in determining an association between
intervention are treated as nuisance variables for primary metabolic syndrome and hepatitis C virus. However, the
analyses. The goal is to present the direct effect of the inter- authors did not clearly specify the outcome in the analysis
vention on the outcomes by eliminating biases.22–24 In the and produced conflicting associations with different anal-
predictive study, the goal is to determine an optimum set yses.30 Thus, the outcome should be clearly specified as per
of variables that can predict the outcome, particularly in the study objective type.
external settings. The clinical decision models are a special
case of prognostic models in which high dimensional data
at various levels are used for risk stratification, classifica- Step 2: specify effect size measure according to study
tion, and prediction. In this model, all variables are consid- design (interpretation and practical value)
ered input features. The goal is to present a decision tool The study design provides information on the selection
that has high accuracy in training, testing, and validation of study participants and the process of data collection
data sets.20 25 Biostatisticians or applied researchers should conditioned on either exposure or outcome (figure 2).
properly discuss the intention of the study objective type The appropriate use of effect size measure, tabular presen-
before proceeding with statistical analyses. In addition, it tation of results, and the level of evidence are mostly
would be a good idea to prepare a conceptual model frame- determined by the study design.31 32 In cohort or clinical
work regardless of study objective type to understand study trial study designs, the participants are selected based on
concepts. exposure status and are followed up for the development

Figure 2 Effect size according to study design.

Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479 1761
 Research tools and issues
of the outcome. These study designs can provide multiple exact p values are recommended to be reported in research

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
outcomes, produce incidence or incidence density, and are studies, p values do not provide any information about
preferred to be analyzed with risk ratio (RR) or hazards the effect size. Compared with p values, the CI provides
models. In a case–control study, the selection of partici- a confidence range of the effect size that contains the true
pants is conditioned on outcome status. This type of study effect size if the study were repeated and can be used to
can have only one outcome and is preferred to be analyzed determine whether the results are statistically significant
with an odds ratio (OR) model. In a cross-­sectional study or not.43 Both p value and 95% CI provide complementary
design, there is no selection restriction on outcomes or information and thus need to be specified in the statistical
exposures. All data are collected simultaneously and can be analysis section.24 44
analyzed with a prevalence ratio model, which is mathemat- Researchers often test one or more comparisons or
ically equivalent to the RR model.33 The reporting of effect hypotheses. Accordingly, the side and the level of signifi-
size measure also depends on the study objective type. For cance for considering results to be statistically significant
example, predictive models typically require reporting of may change. Furthermore, studies may include more than
regression coefficients or weight of variables in the model one primary outcome that requires an adjustment in the level
instead of association measures, which are required in other of significance for multiplicity. All studies should provide
objective types. There are agreements and disagreements the interval estimate of the effect size/regression coefficient
between OR and RR measures. Due to the constancy and in the primary analyses. Since the interpretation of data
symmetricity properties of OR, some researchers prefer analysis depends on the study hypothesis, researchers are
to use OR in studies with common events. Similarly, the required to specify the level of significance along with the
collapsibility and interpretability properties of RR make it side (one-­sided or two-­sided) of the p value in the test for
more appealing to use in studies with common events.34 To considering statistically significant results, adjustment of the
avoid variable practice and interpretation issues with OR, level of significance due to multiple comparisons or multi-
it is recommended to use RR models in all studies except plicity, and reporting of interval estimates of the effect size
for case–control and nested case–control studies, where in the statistical analysis section.45
OR approximates RR and thus OR models should be used.
Otherwise, investigators may report sufficient data to
Examples
compute any ratio measure. Biostatisticians should educate
A study46 showed a significant effect of fluoxetine on relapse
investigators on the proper interpretation of ratio measures
rates in obsessive-­compulsive disorder based on a one-­sided
in the light of study design and their reporting.34 35
p value of 0.04. Clearly, there was no reason for using a one-­
sided p value as opposed to a two-­sided p value. A review of
Examples the appropriate use of multiple test correction methods in
Investigators sometimes either inappropriately label their multiarm clinical trials published in major medical journals
study design36 37 or report effect size measures not aligned in 2012 identified over 50% of the articles did not perform
with the study design,38 39 leading to difficulty in results multiple-­testing correction.47 Similar to controlling a fami-
interpretation and evaluation of the level of evidence. The lywise error rate due to multiple comparisons, adjustment
proper labeling of study design and the appropriate use of of the false discovery rate is also critical in studies involving
effect size measure have substantial implications for results multiple related outcomes. A review of RCTs for depression
interpretation, including the conduct of systematic review between 2007 and 2008 from six journals reported that
and meta-­analysis.40 A study31 reviewed the frequency of only limited studies (5.8%) accounted for multiplicity in the
reporting OR instead of RR in cohort studies and random- analyses due to multiple outcomes.48
ized clinical trials (RCTs) and found that one-­third of the
cohort studies used an OR model, whereas 5% of RCTs
Step 4: account for DGP in the statistical analysis
used an OR model. The majority of estimated ORs from
(accuracy)
these studies had a 20% or higher deviation from the corre-
The study design also requires the specification of the selec-
sponding RR.
tion of participants and outcome measurement processes
in different design settings. We referred to this specific
Step 3: specify study hypothesis, reporting of p values, design feature as DGP. Understanding DGP helps in deter-
and interval estimates (interpretation and decision) mining appropriate modeling of outcome distribution in
The clinical hypothesis provides information for evaluating statistical analyses and setting up model premises and units
formal claims specified in the study objectives, while the of analysis.4 DGP (figure 3) involves information on data
statistical hypothesis provides information about the popu- generation and data measures, including the number of
lation parameters/statistics being used to test the formal measurements after random selection, complex selection,
claims. The inference about the study hypothesis is typically consecutive selection, pragmatic selection, or systematic
measured by p value and confidence interval (CI). A smaller selection. Specifically, DGP depends on a sampling setting
p value indicates that the data support against the null (participants are selected using survey sampling methods
hypothesis. Since the p value is a conditional probability, it and one subject may represent multiple participants in
can never tell about the acceptance or rejection of the null the population), clustered setting (participants are clus-
hypothesis. Therefore, multiple alternative strategies of p tered through a recruitment setting or hierarchical setting
values have been proposed to strengthen the credibility of or multiple hospitals), pragmatic setting (participants are
conclusions.41 42 Adaption of these alternative strategies is selected through mixed approaches), or systematic review
only needed in the explanatory objective studies. Although setting (participants are selected from published studies).
1762 Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479
 Research tools and issues

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
Figure 3 Common features of the data generation process.

DGP also depends on the measurements of outcomes in an published studies using the Korean National Health and
unpaired setting (measured on one occasion only in inde- Nutrition Examination Survey did not incorporate survey
pendent groups), paired setting (measured on more than sampling structure in statistical analyses, producing biased
one occasion or participants are matched on certain subject estimates and inappropriate findings. Likewise, another
characteristics), or mixed setting (measured on more than study52 highlighted the need for maintaining methodolog-
one occasion but interested in comparing independent ical standards while analyzing data from the National Inpa-
groups). It also involves information regarding outcomes or tient Sample. A systematic review53 identified that over
exposure generation processes using quantitative or cate- 50% of studies did not specify whether a paired t-­test or
gorical variables, quantitative values using labs or validated an unpaired t-­test was performed in statistical analysis in
instruments, and self-­reported or administered tests yielding the top 25% of physiology journals, indicating poor trans-
a variety of data distributions, including individual distri- parency in reporting of statistical analysis as per the data
bution, mixed-­type distribution, mixed distributions, and type. Another study54 also highlighted the data displaying
latent distributions. Due to different DGPs, study data may errors not aligned with DGP. As per DGP, delay in treat-
include messy or missing data, incomplete/partial measure- ment initiation of patients with cancer defined from
ments, time-­ varying measurements, surrogate measures, the onset of symptom to treatment initiation should be
latent measures, imbalances, unknown confounders, instru- analyzed into three components: patient/primary delay,
ment variables, correlated responses, various levels of clus- secondary delay, and tertiary delay.55 Similarly, the number
tering, qualitative data, or mixed data outcomes, competing of cancerous nodes should be analyzed with count data
events, individual and higher-­level variables, etc. The perfor- models.56 However, several studies did not analyze such
mance of statistical analysis, appropriate estimation of stan- data according to DGP.57 58
dard errors of estimates and subsequently computation of
p values, the generalizability of findings, and the graphical
display of data rely on DGP. Accounting for DGP in the Step 5: apply EBB methods specific to study design
analyses requires proper communication between investi- features and DGP (efficiency and robustness)
gators and biostatisticians about each aspect of participant The continuous growth in the development of robust
selection and data collection, including measurements, statistical methods for dealing with a specific problem
occasions of measurements, and instruments used in the produced various methods to analyze specific data types.
research study. Since multiple methods are available for handling a specific
problem yet with varying performances, heterogeneous
Examples practices among applied researchers have been noticed.
A study49 compared the intake of fresh fruit and komatsuna Variable practices could also be due to a lack of consensus
juice with the intake of commercial vegetable juice on on statistical methods in literature, unawareness, and the
metabolic parameters in middle-­aged men using an RCT. unavailability of standardized statistical guidelines.2 5 59
The study was criticized for many reasons, but primarily However, it becomes sometimes difficult to differentiate
for incorrect statistical methods not aligned with the study whether a specific method was used due to its robustness,
DGP.50 Similarly, another study51 highlighted that 80% of lack of awareness, lack of accessibility of statistical software
Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479 1763
 Research tools and issues
to apply an alternative appropriate method, intention to event per variable (EVR) is important to avoid overfitting

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
produce expected results, or ignorance of model diagnos- in any type of modeling; therefore, screening of variables
tics. To avoid heterogeneous practices, the selection of may be required in some association and explanatory
statistical methodology and their reporting at each stage of studies, which may be accomplished using a backward
data analysis should be conducted using methods according stepwise method that needs to be clarified in the statistical
to EBB practice.5 Since it is hard for applied researchers analyses.10 In a predictive study, a model with an optimum
to optimally select statistical methodology at each step, we set of variables producing the highest accuracy should be
encourage investigators to involve biostatisticians at the used. The optimum set of variables may be screened with
very early stage in basic, clinical, population, translational, the random forest method or bootstrap or machine learning
and database research. We also appeal to biostatisticians methods.64 65 Different methods of variable selection and
to develop guidelines, checklists, and educational tools to adjustments may lead to different results. The screening
promote the concept of EBB. As an effort, we developed process of variables and their adjustments in the final multi-
the statistical analysis and methods in biomedical research variable model should be clearly mentioned in the statistical
(SAMBR) guidelines for applied researchers to use EBB analysis section.
methods for data analysis.5 The EBB practice is essential
for applying recent cutting-­edge robust methodologies to
Examples
yield accurate and unbiased results. The efficiency of statis-
A study66 evaluating the effect of hydroxychloroquine
tical methodologies depends on the assumptions and DGP.
(HDQ) showed unfavorable events (intubation or death) in
Therefore, investigators may attempt to specify the choice
patients who received HDQ compared with those who did
of specific models in the primary analysis as per the EBB.
not (hazard ratio (HR): 2.37, 95% CI 1.84 to 3.02) in an
unadjusted analysis. However, the propensity score-­adjusted
Examples analyses as appropriate with the interventional objective
Although details of evidence-­based preferred methods are model showed no significant association between HDQ use
provided in the SAMBR checklists for each study design/ and unfavorable events (HR: 1.04, 95% CI 0.82 to 1.32),
objective,5 we have presented a simplified version of which was also confirmed in multivariable and other propen-
evidence-­based preferred methods for common statistical sity score-­adjusted analyses. This study clearly suggests that
analysis (online supplemental table 3). Several examples results interpretation should be based on a multivariable
are available in the literature where inefficient methods not analysis only in observational studies if feasible. A recent
according to EBB practice have been used.31 57 60 study10 noted that approximately 6% of multivariable anal-
yses based on either logistic or Cox regression used an inap-
propriate selection method of variables in medical research.
Step 6: report variable selection method in the
This practice was more commonly noted in studies that did
multivariable analysis according to study objective type
not involve an expert biostatistician. Another review61 of
(unbiased)
316 articles from high-­ impact Chinese medical journals
Multivariable analysis can be used for association, predic-
revealed that 30.7% of articles did not report the selection
tion or classification or risk stratification, adjustment,
of variables in multivariable models. Indeed, this inappro-
propensity score development, and effect size estimation.61
priate practice could have been identified more commonly
Some biological, clinical, behavioral, and environmental
if classified according to the study objective type.18 In RCTs,
factors may directly associate or influence the relationship
it is uncommon to report an adjusted analysis based on
between exposure and outcome. Therefore, almost all health
prognostic variables, even though an adjusted analysis may
studies require multivariable analyses for accurate and unbi-
produce an efficient estimate compared with an unadjusted
ased interpretations of findings (figure 1). Analysts should
analysis. A study assessing the effect of preemptive interven-
develop an adjusted model if the sample size permits. It is
tion on development outcomes showed a significant effect
a misconception that the analysis of RCT does not require
of an intervention on reducing autism spectrum disorder
adjusted analysis. Analysis of RCT may require adjustment
symptoms.67 However, this study was criticized by Ware68
for prognostic variables.23 The foremost step in model
for not reporting non-­ significant results in unadjusted
building is the entry of variables after finalizing the appro-
analyses. If possible, unadjusted estimates should also be
priate parametric or non-­parametric regression model. In
reported in any study, particularly in RCTs.23 68
the exploratory model building process due to no prefer-
ence of exposures, a backward automated approach after
including any variables that are significant at 25% in the Step 7: provide evidence for exploring effect modifiers
unadjusted analysis can be used for variable selection.62 63 (applicability)
In the association model, a manual selection of covariates Any variable that modifies the effect of exposure on the
based on the relevance of the variables should be included outcome is called an effect modifier or modifier or an
in a fully adjusted model.63 In a causal model, clinically interacting variable. Exploring the effect modifiers in
guided methods should be used for variable selection and multivariable analyses helps in (1) determining the appli-
their adjustments.20 In a non-­ randomized interventional cability/generalizability of findings in the overall or specific
model, efforts should be made to eliminate confounding subpopulation, (2) generating ideas for new hypotheses,
effects through propensity score methods and the final (3) explaining uninterpretable findings between unadjusted
propensity score-­adjusted multivariable model may adjust and adjusted analyses, (4) guiding to present combined
any prognostic variables, while a randomized study simply or separate models for each specific subpopulation, and
should adjust any prognostic variables.27 Maintaining the (5) explaining heterogeneity in treatment effect. Often,
1764 Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479
 Research tools and issues
investigators present adjusted stratified results according to Assessment of EVR in multivariable analysis can be used to

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
the presence or absence of an effect modifier. If the expo- avoid the overfitting issue of a multivariable model.18
sure interacts with multiple variables statistically or concep-
tually in the model, then the stratified findings (subgroup) Examples
according to each effect modifier may be presented. Other- Some review studies highlighted that 73.8%–92% of the
wise, stratified analysis substantially reduces the power of articles published in MEDLINE had not assessed the model
the study due to the lower sample size in each stratum and diagnostics of the multivariable regression models.17 61 72
may produce significant results by inflating type I error.69 Contrary to the monotonically, linearly increasing relation-
Therefore, a multivariable analysis involving an interaction ship between systolic blood pressure (SBP) and mortality
term as opposed to a stratified analysis may be presented in established using the Framingham’s study,74 Port et al75
the presence of an effect modifier.70 Sometimes, a quantita- reported a non-­ linear relationship between SBP and all-­
tive variable may emerge as a potential effect modifier for cause mortality or cardiovascular deaths by reanalysis of the
exposure and an outcome relationship. In such a situation, Framingham’s study data set. This study identified a different
the quantitative variable should not be categorized unless a threshold for treating hypertension, indicating the role
clinically meaningful threshold is not available in the study. of linearity assessment in multivariable models. Although
In fact, the practice of categorizing quantitative variables a non-­ Gaussian distribution model may be required for
should be avoided in the analysis unless a clinically mean- modeling patient delay outcome data in cancer,55 a study
ingful cut-­off is available or a hypothesis requires for it.71 In analyzed patient delay data using an ordinary linear regres-
an exploratory objective type, any possible interaction may sion model.57 An investigation of the development of
be obtained in a study; however, the interpretation should predictive models and their reporting in medical journals
be guided based on clinical implications. Similarly, some identified that 53% of the articles had fewer EVR than the
objective models may have more than one exposure or recommended EVR, indicating over half of the published
intervention and the association of each exposure according articles may have an overfitting model.18 Another study76
to the level of other exposure should be presented through attempted to identify the anthropometric variables associ-
adjusted analyses as suggested in the presence of interaction ated with non-­insulin-­dependent diabetes and found that
effects.70 none of the anthropometric variables were significant after
adjusting for waist circumference, age, and sex, indicating
the presence of collinearity. A study reported detailed sparse
Examples data problems in published studies and potential solutions.73
A review of 428 articles from MEDLINE on the quality of
reporting from statistical analyses of three (linear, logistic,
and Cox) commonly used regression models reported that Step 9: report type of primary and sensitivity analyses
only 18.5% of the published articles provided interaction (consistency)
analyses,17 even though interaction analyses can provide a Numerous considerations and assumptions are made
lot of useful information. throughout the research processes that require assessment,
evaluation, and validation. Some assumptions, executions,
and errors made at the beginning of the study data collec-
Step 8: assessment of assumptions, specifically the tion may not be fixable13; however, additional information
distribution of outcome, linearity, multicollinearity, collected during the study and data processing, including
sparsity, and overfitting (reliability) data distribution obtained at the end of the study, may
The assessment and reporting of model diagnostics are facilitate additional considerations that need to be veri-
important in assessing the efficiency, validity, and useful- fied in the statistical analyses. Consistencies in the research
ness of the model. Model diagnostics include satisfying findings via modifications in the outcome or exposure
model-­specific assumptions and the assessment of sparsity, definition, study population, accounting for missing data,
linearity, distribution of outcome, multicollinearity, and model-­related assumptions, variables and their forms, and
overfitting.61 72 Model-­specific assumptions such as normal accounting for adherence to protocol in the models can be
residuals, heteroscedasticity and independence of errors evaluated and reported in research studies using sensitivity
in linear regression, proportionality in Cox regression, analyses.77 The purpose and type of supporting analyses
proportionality odds assumption in ordinal logistic regres- need to be specified clearly in the statistical analyses to
sion, and distribution fit in other types of continuous and differentiate the main findings from the supporting find-
count models are required. In addition, sparsity should also ings. Sensitivity analyses are different from secondary or
be examined prior to selecting an appropriate model. Spar- interim or subgroup analyses.78 Data analyses for secondary
sity indicates many zero observations in the data set.73 In the outcomes are often referred to as secondary analyses, while
presence of sparsity, the effect size is difficult to interpret. data analyses of an ongoing study are called interim anal-
Except for machine learning models, most of the parametric yses and data analyses according to groups based on patient
and semiparametric models require a linear relationship characteristics are known as subgroup analyses.
between independent variables and a functional form of an
outcome. Linearity should be assessed using a multivariable Examples
polynomial in all model objectives.62 Similarly, the appro- Almost all studies require some form of sensitivity anal-
priate choice of the distribution of outcome is required ysis to validate the findings under different conditions.
for model building in all study objective models. Multicol- However, it is often underutilized in medical journals. Only
linearity assessment is also useful in all objective models. 18%–20.3% of studies reported some forms of sensitivity
Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479 1765
 Research tools and issues
analyses.77 78 A review of nutritional trials from high-­quality and effect size measure should be specified in the statistical

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
journals reflected that 17% of the conclusions were reported analysis section of research studies.
inappropriately using findings from sensitivity analyses not
based on the primary/main analyses.77
Examples
Published articles from reputed journals inappropriately
Step 10: provide methods for summarizing, displaying, summarized a categorized variable with mean and range,81
and interpreting data (transparency and usability) summarized a highly skewed variable with mean and stan-
Data presentation includes data summary, data display, dard deviation,57 and treated a categorized variable as a
and data from statistical model analyses. The primary continuous variable in regression analyses.82 Similarly,
purpose of the data summary is to understand the distri- numerous examples from published studies reporting inap-
bution of outcome status and other characteristics in the propriate graphical display or inappropriate interpretation
total sample and by primary exposure status or outcome of data not aligned with DGP or variable types are illustrated
status. Column-­wise data presentation should be preferred in a book published by Bland and Peacock.83 84 A study used
according to exposure status in all study designs, while qualitative data on MRI but inappropriately presented with
row-­ wise data presentation for the outcome should be a Box-­ Whisker plot.81 Another study reported unusually
preferred in all study designs except for a case–control high OR for an association between high breast parenchymal
study.24 32 Summary statistics should be used to provide enhancement and breast cancer in both premenopausal and
maximum information on data distribution aligned with postmenopausal women.85 This reporting makes suspicious
DGP and variable type. The purpose of results presentation findings and may include sparse data bias.86 A poor tabular
primarily from regression analyses or statistical models is presentation without proper scaling or standardization of
to convey results interpretation and implications of find- a variable, missing CI for some variables, missing unit and
sample size, and inconsistent reporting of decimal places
ings. The results should be presented according to the study
could be easily noticed in table 4 of a published study.29
objective type. Accordingly, the reporting of unadjusted
Some published predictive models87 do not report intercept
and adjusted associations of each factor with the outcome
or baseline survival estimates to use their predictive models
may be preferred in the determinant objective model, while
in clinical use. Although a direct comparison of effect sizes
unadjusted and adjusted effects of primary exposure on
obtained from the same model may be avoided if the units
the outcome may be preferred in the explanatory objective
are different among variables,35 a study had an objective
model. In prognostic models, the final predictive models
to compare effect sizes across variables but the authors
may be presented in such a way that users can use models
performed comparisons without standardization of vari-
to predict an outcome. In the exploratory objective model,
ables or using statistical tests.88
a final multivariable model should be reported with R2 or
area under the curve (AUC). In the association and interven-
tional models, the assessment of internal validation is crit- A SAMPLE FOR WRITING STATISTICAL ANALYSIS
ically important through various sensitivity and validation SECTION IN MEDICAL JOURNALS/RESEARCH STUDIES
analyses. A model with better fit indices (in terms of R2 or Our primary study objective type was to develop a (select
AUC, Akaike information criterion, Bayesian information from figure 1) model to assess the relationship of risk factors
criterion, fit index, root mean square error) should be final- (list critical variables or exposures) with outcomes (specify
ized and reported in the causal model objective study. In the type from continuous/discrete/count/binary/polytomous/
predictive objective type, the model performance in terms time-­to-­event). To address this objective, we conducted a
of R2 or AUC in training and validation data sets needs to (select from figure 2 or any other) study design to test the
be reported (figure 1).20 21 There are multiple purposes of hypotheses of (equality or superiority or non-­inferiority or
data display, including data distribution using bar diagram equivalence or futility) or develop prediction. Accordingly,
or histogram or frequency polygons or box plots, compari- the other variables were adjusted or considered as (specify
sons using cluster bar diagram or scatter dot plot or stacked role of variables from confounders, covariates, or predic-
bar diagram or Kaplan-­Meier plot, correlation or model tors or independent variables) as reflected in the conceptual
assessment using scatter plot or scatter matrix, clustering or framework. In the unadjusted or preliminary analyses as per
pattern using heatmap or line plots, the effect of predictors the (select from figure 3 or any other design features) DGP,
with fitted models using marginsplot, and comparative eval- (specify EBB preferred tests from online supplemental table
uation of effect sizes from regression models using forest 3 or any other appropriate tests) were used for (specify vari-
plot. Although the key purpose of data display is to highlight ables and types) in unadjusted analyses. According to the EBB
critical issues or findings in the study, data display should practice for the outcome (specify type) and DGP of (select
essentially follow DGP and variable types and should be from figure 3 or any other), we used (select from online
user-­friendly.54 79 Data interpretation heavily relies on the supplemental table 1 or specify a multivariable approach)
effect size measure along with study design and specified as the primary model in the multivariable analysis. We
hypotheses. Sometimes, variables require standardization used (select from figure 1) variable selection method in the
for descriptive comparison of effect sizes among exposures multivariable analysis and explored the interaction effects
or interpreting small effect size, or centralization for inter- between (specify variables). The model diagnostics including
preting intercept or avoiding collinearity due to interac- (list all applicable, including model-­ related assumptions,
tion terms, or transformation for achieving model-­related linearity, or multicollinearity or overfitting or distribution
assumptions.80 Appropriate methods of data reporting and of outcome or sparsity) were also assessed using (specify
interpretation aligned with study design, study hypothesis, appropriate methods) respectively. In such exploration, we
1766 Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479
 Research tools and issues
identified (specify diagnostic issues if any) and therefore the in (specify the form of the variables) and therefore the effect

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
multivariable models were developed using (specify poten- or association of (list exposures or variables) on outcome
tial methods used to handle diagnostic issues). The other should be interpreted in terms of changes in (specify inter-
outcomes were analyzed with (list names of multivariable pretation unit) exposures/variables. List all other additional
approaches with respective outcomes). All the models used analyses if performed (with full details of all models in a
the same procedure (or specify from figure 1) for variable supplementary file along with statistical codes if possible).
selection, exploration of interaction effects, and model
diagnostics using (specify statistical approaches) depending
on the statistical models. As per the study design, hypoth- CONCLUDING REMARKS
esis, and multivariable analysis, the results were summa- We highlighted 10 essential steps to be reported in the
rized with effect size (select as appropriate or from figure 2) statistical analysis section of any analytical study (figure 4).
along with (specify 95% CI or other interval estimates) and Adherence to minimum reporting of the steps specified
considered statistically significant using (specify the side in this report may enforce investigators to understand
of p value or alternatives) at (specify the level of signifi- concepts and approach biostatisticians timely to apply these
cance) due to (provide reasons for choosing a significance concepts in their study to improve the overall quality of
level). We presented unadjusted and/or adjusted estimates methodological standards in grant proposals and research
of primary outcome according to (list primary exposures or studies. The order of reporting information in statistical
variables). Additional analyses were conducted for (specific analyses specified in this report is not mandatory; however,
reasons from step 9) using (specify methods) to validate clear reporting of analytical steps applicable to the specific
findings obtained in the primary analyses. The data were study type should be mentioned somewhere in the manu-
summarized with (list summary measures and appropriate script. Since the entire approach of statistical analyses is
graphs from step 10), whereas the final multivariable model dependent on the study objective type and EBB practice,
performance was summarized with (fit indices if applicable proper execution and reporting of statistical models can be
from step 10). We also used (list graphs) as appropriate taught to the next generation of statisticians by the study
with DGP (specify from figure 3) to present the critical objective type in statistical education courses. In fact, some
findings or highlight (specify data issues) using (list graphs/ disciplines (figure 5) are strictly aligned with specific study
methods) in the study. The exposures or variables were used objective types. Bioinformaticians are oriented in studying

Figure 4 Summary of reporting steps, purpose, and evaluation measures in the statistical analysis section.
Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479 1767
 Research tools and issues

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
Figure 5 Role of interrelated disciplines according to study objective type.

determinant and prognostic models toward precision medi- Acknowledgements The author would like to thank the reviewers for their
cine, while epidemiologists are oriented in studying associ- careful review and insightful suggestions.
ation and causal models, particularly in population-­based Contributors AKD developed the concept and design and wrote the
observational and pragmatic settings. Data scientists are manuscript.
heavily involved in prediction and classification models in Funding The authors have not declared a specific grant for this research from
personalized medicine. A common thing across disciplines any funding agency in the public, commercial or not-­for-­profit sectors.
is using biostatistical principles and computation tools to Competing interests AKD is a Journal of Investigative Medicine Editorial
address any research question. Sometimes, one discipline Board member. No other competing interests declared.
expert does the part of others.89 We strongly recommend Patient consent for publication Not required.
using a team science approach that includes an epidemi- Provenance and peer review Commissioned; externally peer reviewed.
ologist, biostatistician, data scientist, and bioinformati- Data availability statement Data sharing not applicable as no datasets
cian depending on the study objectives and needs. Clear generated and/or analyzed for this study.
reporting of data analyses as per the study objective type
Supplemental material This content has been supplied by the author(s).
should be encouraged among all researchers to minimize It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not
heterogeneous practices and improve scientific quality and have been peer-­reviewed. Any opinions or recommendations discussed are
outcomes. In addition, we also encourage investigators to solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all
strictly follow transparent reporting and quality assess- liability and responsibility arising from any reliance placed on the content.
Where the content includes any translated material, BMJ does not warrant the
ment guidelines according to the study design (https:// accuracy and reliability of the translations (including but not limited to local
[Link]/) to improve the overall quality regulations, clinical guidelines, terminology, drug names and drug dosages),
of the study, accordingly STROBE (Strengthening the and is not responsible for any error and/or omissions arising from translation
Reporting of Observational Studies in Epidemiology) for and adaptation or otherwise.
observational studies, CONSORT (Consolidated Standards Open access This is an open access article distributed in accordance with
of Reporting Trials) for clinical trials, STARD (Standards the Creative Commons Attribution Non Commercial (CC BY-­NC 4.0) license,
for Reporting Diagnostic Accuracy Studies) for diagnostic which permits others to distribute, remix, adapt, build upon this work non-­
commercially, and license their derivative works on different terms, provided
studies, TRIPOD (Transparent Reporting of a multivariable the original work is properly cited, an indication of whether changes were
prediction model for Individual Prognosis OR Diagnosis) made, and the use is non-­commercial. See: [Link]
for prediction modeling, and ARRIVE (Animal Research: licenses/by-nc/4.0/.
Reporting of In Vivo Experiments) for preclinical studies.
ORCID iD
The steps provided in this document for writing the statis-
Alok Kumar Dwivedi [Link]
tical analysis section is essentially different from other guid-
ance documents, including SAMBR.5 SAMBR provides a
guidance document for selecting evidence-­based preferred REFERENCES
methods of statistical analysis according to different study 1 Oster RA, Devick KL, Thurston SW, et al. Learning gaps among statistical
designs, while this report suggests the global reporting competencies for clinical and translational science learners. J Clin Transl Sci
of essential information in the statistical analysis section 2020;5:e12.
2 Sauerbrei W, Abrahamowicz M, Altman DG, et al. Strengthening analytical
according to study objective type. In this guidance report, thinking for observational studies: the STRATOS initiative. Stat Med
our suggestion strictly pertains to the reporting of methods 2014;33:5413–32.
in the statistical analysis section and their implications on the 3 Thiese MS, Arnold ZC, Walker SD. The misuse and abuse of statistics in
interpretation of results. Our document does not provide biomedical research. Biochem Med 2015;25:5–11.
guidance on the reporting of sample size or results or statis- 4 Steven S, RMGTMK H. Statistical modeling methods: challenges and strategies.
Biostatistics & Epidemiology 2020;4:105–39.
tical analysis section for meta-­analysis. The examples and 5 Dwivedi AK, Shukla R. Evidence‐based statistical analysis and methods in
reviews reported in this study may be used to emphasize the biomedical research (SAMBR) checklists according to design features. Cancer
concepts and related implications in medical research. Rep 2020;3:e1211.

1768 Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479

 Research tools and issues
6 Ocaña-­Riola R. The use of statistics in health sciences: situation analysis and 35 Davies HT, Crombie IK, Tavakoli M. When can odds ratios mislead? BMJ

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
perspective. Stat Biosci 2016;8:204–19. 1998;316:989–91.
7 Wang MQ, Yan AF, Katz RV. Researcher requests for inappropriate analysis 36 Oh S, Chung J, Baek S, et al. Postoperative expressive aphasia associated with
and reporting: a U.S. survey of consulting Biostatisticians. Ann Intern Med intravenous midazolam administration: a 5-­year retrospective case-­control
2018;169:554–8. study. J Int Med Res 2020;48:030006052094875.
8 Zhang G, Chen JJ. Biostatistics faculty and NIH awards at U.S. medical schools. 37 Chen ML, Gupta A, Chatterjee A, et al. Association between unruptured
Am Stat 2015;69:34–40. intracranial aneurysms and downstream stroke. Stroke 2018;49:2029–33.
9 Harrington D, D’Agostino RB, Gatsonis C, et al. New Guidelines for Statistical 38 Sturdik I, Krajcovicova A, Jalali Y, et al. Pathophysiology and risk factors for
Reporting in the Journal. N Engl J Med 2019;381:285–6. cholelithiasis in patients with Crohn’s disease. Physiol Res 2019;68:S173–82.
10 Nojima M, Tokunaga M, Nagamura F. Quantitative investigation of 39 Liao Y-­T, Yang S-­Y, Liu H-­C, et al. Cardiac complications associated with
inappropriate regression model construction and the importance of medical short-­term mortality in schizophrenia patients hospitalized for pneumonia: a
statistics experts in observational medical research: a cross-­sectional study. nationwide case-­control study. PLoS One 2013;8:e70142.
BMJ Open 2018;8:e021129. 40 Doi SA, Furuya-­Kanamori L, Xu C, et al. Controversy and debate: questionable
11 Ciolino JD, Spino C, Ambrosius WT, et al. Guidance for biostatisticians on their utility of the relative risk in clinical research: paper 1: a call for change to
essential contributions to clinical and translational research protocol review. J practice. J Clin Epidemiol 2022;142:271–9.
Clin Transl Sci 2021;5:e161. 41 Halsey LG. The reign of the p-­value is over: what alternative analyses could we
12 Gosselin R-­D. Insufficient transparency of statistical reporting in preclinical employ to fill the power vacuum? Biol Lett 2019;15:20190174.
research: a scoping review. Sci Rep 2021;11:3335. 42 RLWNA L. The ASA statement on p-­Values: context, process, and purpose. The
13 Brown AW, Kaiser KA, Allison DB. Issues with data and analyses: errors, American Statistician;70:129–33.
underlying themes, and potential solutions. Proc Natl Acad Sci U S A 43 Page P. Beyond statistical significance: clinical interpretation of rehabilitation
2018;115:2563–70. research literature. Int J Sports Phys Ther 2014;9:726–36.
14 Bacchetti P. Peer review of statistics in medical research: the other problem. 44 Weinberg CR. It’s time to rehabilitate the P-­value. Epidemiology
BMJ 2002;324:1271–3. 2001;12:288–90.
15 Sima AP, Rodriguez VA, Bradbrook KE, et al. Incorporating professional 45 Ou F-­S, Le-­Rademacher JG, Ballman KV, et al. Guidelines for statistical reporting
recommendations into a graduate-­level statistical consulting laboratory: a case in medical journals. J Thorac Oncol 2020;15:1722–6.
study. J Clin Transl Sci 2020;5:e62. 46 Romano S, Goodman W, Tamura R, et al. Long-­Term treatment of obsessive-­
16 Fernandes-­Taylor S, Hyun JK, Reeder RN, et al. Common statistical and research compulsive disorder after an acute response: a comparison of fluoxetine versus
design problems in manuscripts submitted to high-­impact medical journals. placebo. J Clin Psychopharmacol 2001;21:46–52.
BMC Res Notes 2011;4:304. 47 Wason JMS, Stecher L, Mander AP. Correcting for multiple-­testing in multi-­arm
17 Real J, Forné C, Roso-­Llorach A, et al. Quality reporting of multivariable trials: is it necessary and is it done? Trials 2014;15:364.
regression models in observational studies: review of a representative 48 Tyler KM, Normand S-­LT, Horton NJ. The use and abuse of multiple outcomes in
sample of articles published in biomedical journals. Medicine randomized controlled depression trials. Contemp Clin Trials 2011;32:299–304.
2016;95:e3653. 49 Aiso I, Inoue H, Seiyama Y, et al. Compared with the intake of commercial
18 Bouwmeester W, Zuithoff NPA, Mallett S, et al. Reporting and vegetable juice, the intake of fresh fruit and komatsuna (Brassica rapa L. var.
methods in clinical prediction research: a systematic review. PLoS Med perviridis) juice mixture reduces serum cholesterol in middle-­aged men: a
2012;9:e1001221–12. randomized controlled pilot study. Lipids Health Dis 2014;13:102.
19 Shmueli G. To explain or to predict? Statistical Science 2010;25:289–310. 50 Allison DB, Antoine LH, George BJ. Incorrect statistical method in parallel-­
20 Kent P, Cancelliere C, Boyle E, et al. A conceptual framework for prognostic groups RCT led to unsubstantiated conclusions. Lipids Health Dis 2016;15:77.
research. BMC Med Res Methodol 2020;20:172. 51 Kim Y, Park S, Kim N-­S, et al. Inappropriate survey design analysis of the Korean
21 Sainani KL. Explanatory versus predictive modeling. Pm R 2014;6:841–4. National health and nutrition examination survey may produce biased results. J
22 Baser O. Choosing propensity score matching over regression adjustment Prev Med Public Health 2013;46:96–104.
for causal inference: when, why and how it makes sense. J Med Econ 52 Khera R, Angraal S, Couch T, et al. Adherence to methodological standards in
2007;10:379–91. research using the National inpatient sample. JAMA 2017;318:2011–8.
23 Kent DM, Trikalinos TA, Hill MD. Are unadjusted analyses of clinical trials 53 Weissgerber TL, Garcia-­Valencia O, Garovic VD, et al. Why we need to
inappropriately biased toward the null? Stroke 2009;40:672–3. report more than ’Data were Analyzed by t-­tests or ANOVA’. Elife 2018;7.
24 Cummings P, Rivara FP. Reporting statistical information in medical Journal doi:10.7554/eLife.36163. [Epub ahead of print: 21 12 2018].
articles. Arch Pediatr Adolesc Med 2003;157:321–4. 54 Weissgerber TL, Milic NM, Winham SJ, et al. Beyond bar and line graphs: time
25 Luo W, Phung D, Tran T, et al. Guidelines for developing and reporting machine for a new data presentation paradigm. PLoS Biol 2015;13:e1002128.
learning predictive models in biomedical research: a multidisciplinary view. J 55 Alok Kumar D, Nand DS, Suryanarayana D, et al. An epidemiological study on
Med Internet Res 2016;18:e323. delay in treatment initiation of cancer patients. Health 2012;4. doi:10.4236/
26 Spera G, Fresco R, Fung H, et al. Beta blockers and improved progression-­free health.2012.42012
survival in patients with advanced HER2 negative breast cancer: a retrospective 56 Dwivedi AK, Dwivedi SN, Deo S, et al. Statistical models for predicting number
analysis of the ROSE/TRIO-­012 study. Ann Oncol 2017;28:1836–41. of involved nodes in breast cancer patients. Health 2010;2:641–51.
27 Austin PC. An introduction to propensity score methods for reducing the 57 Poum A, Promthet S, Duffy SW, et al. Factors associated with delayed diagnosis
effects of confounding in observational studies. Multivariate Behav Res of breast cancer in northeast Thailand. J Epidemiol 2014;24:102–8.
2011;46:399–424. 58 Ravdin PM, De Laurentiis M, Vendely T, et al. Prediction of axillary lymph node
28 Chen X, Zhou Y, Wang R, et al. Potential clinical value of multiparametric status in breast cancer patients by use of prognostic indicators. J Natl Cancer
PET in the prediction of Alzheimer’s disease progression. PLoS One Inst 1994;86:1771–5.
2016;11:e0154406. 59 Evans RG, Su D-­F. Data presentation and the use of statistical tests in
29 Shaheen M, Echeverry D, Oblad MG, et al. Hepatitis C, metabolic syndrome, biomedical journals: can we reach a consensus? Clin Exp Pharmacol Physiol
and inflammatory markers: results from the Third National Health and Nutrition 2011;38:285–6.
Examination Survey [NHANES III]. Diabetes Res Clin Pract 2007;75:320–6. 60 Baker D, Lidster K, Sottomayor A, et al. Two years later: journals are not yet
30 Rajkumar P, Dwivedi AK, Dodoo CA, et al. The association between metabolic enforcing the ARRIVE guidelines on reporting standards for pre-­clinical animal
syndrome and hepatitis C virus infection in the United States. Cancer Causes studies. PLoS Biol 2014;12:e1001756.
Control 2020;31:569–81. 61 Zhang Y-­Y, Zhou X-­B, Wang Q-­Z, et al. Quality of reporting of multivariable
31 Knol MJ, Le Cessie S, Algra A, et al. Overestimation of risk ratios by odds logistic regression models in Chinese clinical medical journals. Medicine
ratios in trials and cohort studies: alternatives to logistic regression. CMAJ 2017;96:e6972.
2012;184:895–9. 62 Sauerbrei W, Perperoglou A, Schmid M, et al. State of the art in selection of
32 Althouse AD, Raffa GM, Kormos RL. Your results, explained: clarity variables and functional forms in multivariable analysis-­outstanding issues.
provided by row percentages versus column percentages. Ann Thorac Surg Diagn Progn Res 2020;4:3.
2016;101:15–17. 63 Bursac Z, Gauss CH, Williams DK, et al. Purposeful selection of variables in
33 Dwivedi AK, Mallawaarachchi I, Lee S, et al. Methods for estimating relative logistic regression. Source Code Biol Med 2008;3:17.
risk in studies of common binary outcomes. J Appl Stat 2014;41:484–500. 64 Austin PC. Bootstrap model selection had similar performance for selecting
34 Cummings P. The relative merits of risk ratios and odds ratios. Arch Pediatr authentic and noise variables compared to backward variable elimination: a
Adolesc Med 2009;163:438–45. simulation study. J Clin Epidemiol 2008;61:1009–17.

Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479 1769

 Research tools and issues
65 Chen R-­C, Dewi C, Huang S-­W, et al. Selecting critical features for data 78 Thabane L, Mbuagbaw L, Zhang S, et al. A tutorial on sensitivity analyses

J Investig Med: first published as 10.1136/jim-2022-002479 on 16 June 2022. Downloaded from [Link] on January 29, 2023 by guest. Protected by copyright.
classification based on machine learning methods. J Big Data 2020;7:52. in clinical trials: the what, why, when and how. BMC Med Res Methodol
66 Geleris J, Sun Y, Platt J, et al. Observational study of hydroxychloroquine in 2013;13:92.
hospitalized patients with Covid-­19. N Engl J Med 2020;382:2411–8. 79 Kelleher C, Wagener T. Ten guidelines for effective data visualization in scientific
67 Whitehouse AJO, Varcin KJ, Pillar S, et al. Effect of preemptive intervention publications. Environmental Modelling & Software 2011;26:822–7.
on developmental outcomes among infants showing early signs of 80 Althouse AD, Below JE, Claggett BL, et al. Recommendations for statistical
autism: a randomized clinical trial of outcomes to diagnosis. JAMA Pediatr reporting in cardiovascular medicine: a special report from the American heart
2021;175:e213298. association. Circulation 2021;144:e70–91.
68 Ware RS. Reporting both Unadjusted and adjusted estimates is essential 81 DeLeo MJ, Domchek SM, Kontos D, et al. Breast MRI fibroglandular volume and
to the interpretation of randomized clinical trial results. JAMA Pediatr parenchymal enhancement in BRCA1 and BRCA2 mutation carriers before and
2022;176:325–6. immediately after risk-­reducing salpingo-­oophorectomy. AJR Am J Roentgenol
69 Wang R, Ware JH. Detecting moderator effects using subgroup analyses. Prev 2015;204:669–73.
Sci 2013;14:111–20. 82 Dontchos BN, Rahbar H, Partridge SC, et al. Are qualitative assessments of
70 Knol MJ, VanderWeele TJ. Recommendations for presenting analyses of effect background parenchymal enhancement, amount of Fibroglandular tissue on
modification and interaction. Int J Epidemiol 2012;41:514–20. Mr images, and mammographic density associated with breast cancer risk?
71 Figueiras A, Domenech-­Massons JM, Cadarso C. Regression models: calculating Radiology 2015;276:371–80.
the confidence interval of effects in the presence of interactions. Stat Med 83 Bland M, Peacock J. Statistical questions in evidence-­based medicine, 2000.
1998;17:2099–105. 84 Chen JC, Cooper RJ, McMullen ME, et al. Graph quality in top medical journals.
72 Ernst AF, Albers CJ. Regression assumptions in clinical psychology research Ann Emerg Med 2017;69:453–61.
practice-­a systematic review of common misconceptions. PeerJ 2017;5:e3323. 85 Telegrafo M, Rella L, Stabile Ianora AA, et al. Breast MRI background
73 Greenland S, Mansournia MA, Altman DG. Sparse data bias: a problem hiding parenchymal enhancement (BPE) correlates with the risk of breast cancer.
in plain sight. BMJ 2016;352:i1981. Magn Reson Imaging 2016;34:173–6.
74 Li C, Chen Y, Zheng Q, et al. Relationship between systolic blood pressure and 86 Thompson CM, Mallawaarachchi I, Dwivedi DK, et al. The association of
all-­cause mortality: a prospective study in a cohort of Chinese adults. BMC background parenchymal enhancement at breast MRI with breast cancer: a
Public Health 2018;18:107. systematic review and meta-­analysis. Radiology 2019;292:552–61.
75 Port S, Demer L, Jennrich R, et al. Systolic blood pressure and mortality. Lancet 87 Ramspek CL, Jager KJ, Dekker FW, et al. External validation of prognostic
2000;355:175–80. models: what, why, how, when and where? Clin Kidney J 2021;14:49–58.
76 Wei M, Gaskill SP, Haffner SM, et al. Waist circumference as the best predictor 88 Hansson O, Zetterberg H, Buchhave P, et al. Association between CSF
of noninsulin dependent diabetes mellitus (NIDDM) compared to body mass biomarkers and incipient Alzheimer’s disease in patients with mild cognitive
index, waist/hip ratio and other anthropometric measurements in Mexican impairment: a follow-­up study. Lancet Neurol 2006;5:228–34.
Americans--a 7-­year prospective study. Obes Res 1997;5:16–23. 89 Goldstein ND, LeVasseur MT, McClure LA. On the convergence of epidemiology,
77 de Souza RJ, Eisen RB, Perera S, et al. Best (but oft-­forgotten) practices: sensitivity biostatistics, and data science. Harv Data Sci Rev 2020;2. doi:10.1162/99608f
analyses in randomized controlled trials. Am J Clin Nutr 2016;103:5–17. 92.9f0215e6. [Epub ahead of print: 30 04 2020].

1770 Dwivedi AK. J Investig Med 2022;70:1759–1770. doi:10.1136/jim-2022-002479