I Did My Own Research: How to Read and Analyze a Scientific Paper

Reading scientific literature is incredibly important for both clinicians and researchers. Up-to-date knowledge and understanding of scientific proceedings will not only keep you abreast of new developments but also aid awareness of significant trends relevant to effective and safe standards of care.^1,2

Reasons to read scientific papers include:

updating yourself on progress in a particular field of study
researching solutions for specific diagnostic or therapeutic problems
understanding causation versus association
revealing pathophysiology and clinical features of specific diseases
guiding independent research^1,2

This article will review the types of research papers you are likely to encounter, how to analyze a scientific paper, explore basic statistics you may come across, and highlight the importance of critical analysis of the data presented.

What types of research papers are there?

In spite of the internet becoming an efficient source of obtaining information, reading scientific journal articles, whether from print or electronic media, still remains the most reliable way of acquiring new, high-level information.^1-3 In general, articles published in peer-reviewed journals will possess the highest degree of credibility.

Types of published literature can involve any of the following:

Review papers

Review articles may be narrative or systematic.^1,2

Narrative Review

These articles provide a broad overview of the available literature without any specific questions being answered. This type of review is typically an update or summary of research studies pertaining to a specific topic.^1,2

Systematic Reviews

Systematic reviews address a specific question about a topic and have a rigorous analysis of methodology.^1,2 These reviews use particular criteria for paper selection and evaluates the research in a qualitative manner. A specific type of systematic review, called a meta-analysis, combines data from several studies and statistically examines the outcome of the question at hand.^1,2

For a meta-analysis, data from multiple studies may be combined and reanalyzed using established statistical methods.^1,2 In general, due to their analysis strategy, meta-analyses have more power than single studies.

In concert with this, as a general rule, it is important that clinical decisions are not based solely on one or two studies without account taking a comprehensive account of all the research available on that respective topic.

Original (“Primary”) research

Original research is a cornerstone of scientific publications and is written to present findings on new scientific discoveries or elaborate upon earlier work.^1,2 Original research usually consists of the following facets:

Abstract

The abstract provides a brief overview of the article and can be read in a systematic way by answering central questions regarding what the study was hoping to answer, how the study was conducted, the results, and their implications.^1,2 The reader should make note of any questions raised while reading the abstract and be sure that answers have been elucidated after reading the article.^1,2

Introduction

This section provides background information regarding the study and usually a statement of the research hypothesis, if applicable.^1,2 Essentially, this section should give an understandable rationale for why the study was conducted and should mention existing knowledge as well as gaps in previous research regarding the topic under consideration.^1,2 The reader can later return to this section after finishing the paper to see if the hypothesis/main question of the paper was indeed addressed.^1,2

Methods

This section provides details as to how the study was performed, specific procedures that were followed, instruments used, and variables measured.^1,2,4,5 There should be enough information in this section to understand how the study was carried out, including the number of study subjects and criteria for inclusion and exclusion.^1,2,4,5

Here, it is important to look into how investigators controlled for possible confounding factors. For example, subject demographics such as age, sex, diet and medications are often included in statistical analysis to learn if findings still hold in light of these external variables.^1,2,4,5

Please note, if you read this section and do not understand the experimental methodology, it will be challenging to judge the accuracy of results and conclusions. A critical component is whether the type of research is prospective or retrospective, randomized or simply observational.^1,2,4,5

Results

This section includes the results and readers should go through this segment meticulously to determine if the results are reliable and valid — did the study measure what it was supposed to and are the results applicable to the clinic patients you see?^1,2,4,5 If statistical analysis was done (as it usually is for scientific papers), it should be noted if the correct statistical tests were performed and if the levels of significance are appropriate for the study (more on this later).

When analyzing statistical results, it is important to not only know whether a difference or association is significant, but also whether it is large enough to be useful in clinical practice.^1,2,4,5

A brief overview of statistics

Statistics can be daunting to even some of the most experienced readers of scientific literature. Statistics mathematically describe differences among groups or relationships among variables.^4,5 Some basic statistical topics and terms to understand include the following.

Hypothesis

This is a prediction of an outcome based on a treatment, exposure, or intervention.^4,5

H0 (null hypothesis): The null hypothesis states, be definition, that here is no association between variable of interest and the outcome.^4,5
H1 (alternative hypothesis): If the study can reject the null hypothesis, then one can accept the alternative hypothesis; there is an association between the variable of interest and outcome.^4,5

Descriptive statistics

Descriptive statistics generally involves gathering, classifying, summarizing, analyzing, hypothesis testing, and determining relationships between what is being studied and outcomes.^4,5

Qualitative data

This is usually categorical (e.g., sex, specific diagnosis) and not numerical.^4,5

Quantitative data

These data can be measured (e.g., age, weight, intraocular pressure, viral load, etc.).^4,5

Mean

This is an average; the sum of all scores and values, divided by the number of subjects in the study or a group.^4,5

Median

This is a score that falls closest to the middle; half the scores are higher and half are lower than this value (tells you what the “middle ground” is).^4,5

Standard deviation

This tells you the spread and of the data.^4,5

Statistical significance

This attempts to answer the question of the likelihood of a difference being real.^4,5

One of the most important statistical items is the probability (p-value) which describes how likely a result is due to the be real or by chance.^4,5

If the p-value is low, (<5% or p<0.05), one can deduce that the result obtained has a “less than 5% chance” of being random.^4,5 If p<0.01, there is a less than 1% chance that the results are due to chance alone and are therefore statistically significant.^4,5 It is important to note that a result can be statistically, but not clinically significant.

Risk Ratio (RR)

Risk ratio calculates the number of patients in the group who achieve a specific endpoint divided by the total number of patients in the group.^4,5 If RR = 1, there is no relationship between exposure and outcome (“null value”); thus, the outcome is just as likely to occur in an intervention as well as in the control group.^4,5 If RR>1, there is a risk factor present.^4,5 If RR<1, there is a protective factor present.^4,5

Odds Ratio (OR)

Odds are defined as the number of patients in the group who achieve the stated endpoint divided by the number of patients who do not.^4,5 Odds ratio is the ratio of odds in the intervention versus the control group.4,5 OR will be close to RR if the endpoint occurs infrequently.

Confidence Intervals (CI)

Typically seen in papers as “95% CI”, the range listed dictates that it is 95% certain that the actual population effect will occur within the range.^4,5 The width of the CI indicates the precision of the estimate (i.e., wider interval has less precision; a very long interval has a higher doubt of accuracy in predicting effect size).^4,5 If the CI includes 1, this indicates an inability to demonstrate a statistically significant difference between the groups compared.^4,5

Parametric Statistics

Assumes normal distribution (“Bell curve”) of the means of the group and equal variance.^4,5 Commonly used tests here are Analysis of variance (ANOVA), which tests if there is a significant difference between the means of 2 or more groups, as well as the Students t-test, which is used to test the null hypothesis.^4,5 The t-test is performed when measurements are made on the same subjects before and after treatment.^4,5 The Pearson correlation coefficient (r) can also be used here.^4,5

Non-parametric statistics

This is used to analyze ordinal and categorical data when the means are not normally distributed (i.e., not a Bell curve, “skewed”)—these tests usually have less statistical power.^4,5 Commonly used tests here include the Sign test, Wilcoxon’s signed rank test, Mann-Whitney U test, Kolmogorov-Smirnov test, Kruskal-Wallis test, Jonckheere test, Friedman test, and Spearman rank order.^4,5

Additional tests to analyze categorical data included the chi-square test, which compares frequencies and tests whether observed data differ from that of expected data, if there were no differences between groups.^4,5 Fischer’s exact test determines if there are non-random associations between two variables and can calculate exact probability.^4,5 McNemar’s test may also be used here.^4,5

Analyzing results sections

When analyzing the results section, it is important to look for validity – did the study measure what it said it would measure?^1-4 It is also important to identify any bias that may be present.^1-5

Bias is defined as any inclination which may prevent impartial consideration or interpretation of a question or results.

In research, this can occur when an error is introduced into methodology by selecting or encouraging one outcome over others.⁵ Bias can occur in the planning, data collection, analysis, and publication phases of research.⁵ Therefore, a thorough understanding and recognition of bias, as well as how it affects study results is essential for the practice of evidence-based medicine.⁵

Discussion

This section involves interpretation of the results and study implications.^1,2 Here, the research questions answered are discussed concerning previous literature and clinical implications.^.1,2 This section typically discusses study strengths and limitations, providing recommendations about areas that need further investigation.^1,2

How to approach a scientific paper

So, where do you start? Often it is best to begin by reading the title and abstract of an article so that the key elements of the article can be read in an effective and efficient manner.^1,2

Scientific literature continues to grow at an exponential rate: between 1978-2001, the number of articles published annually and listed in MEDLINE alone jumped from 272,344 to 398,778 articles per year.^1,2 Based on these numbers, to stay up to date with current knowledge, a physician practicing general medicine would have to read well over 10 articles per day.^1,2

Though we are aware this is unrealistic for most, the above framework will aid critical analysis of the published literature.

Becoming ‘fluent’ in the language of research analysis takes time, practice, and patience. The more scientific literature you read, the more you will become comfortable with the formats presented and how to accurately and efficiently extract meaningful data that can impact clinical practice.

References

Subramanyam RV. Art of reading a journal article: Methodically and effectively. J Oral Maxillofac Pathol. 2013; 17(1): 65-70
Akobeng AK. Understanding systematic reviews and meta-analysis. Arch Dis Child. 2005;90:845-848
Pannucci C. Wilkins E. Identifying and Avoiding Bias in Research. Plastic Reconstr Surg. 2010; 126(2): 619-625
Ali Z, Bhaskar SB. Basic statistical tools in research and data analysis. Indian J Anaesth. 2016;60(9): 662-669
Enriquez R. Statistics for Dummies? Understanding Statistics for Research Staff! https://www.nacc.org/docs/conference/2015/SU8_Williams_Statistics_for_Dummies_Rachel_Enriquez.pdf