Chapter 8 Experimental Study Designs Learning Objectives (abridged) • State how study designs compare with respect to validity of causal inference • Distinguish between a controlled experiment and a quasi-experiment • Describe the scope of intervention studies • Define the term controlled clinical trials and give examples • Explain the phases in testing a new drug or vaccine Learning Objectives (abridged) • Discuss blinding and crossover in clinical trials. • Define what is meant by community trials. • Discuss ethical aspects of experimentation with human subjects. True Experimental Studies • Most convincing for conferring evidence of associations between risk factors and outcomes • Manipulation of study factor and randomization of subjects • An example is a randomized clinical trial. Women’s Health Initiative • Hormone Replacement Therapy (HRT) – Epidemiologic studies had shown that HRT use had significant benefits against coronary heart disease. – Clinical trials had failed to demonstrate any benefit. – Large body of epidemiologic research had observed that women who took HRT had elevated risks of breast cancer. Women’s Health Initiative • Hormone Replacement Therapy (HRT) – To resolve the question of risks versus benefits of HRT, a clinical trial was conducted. – Demonstrated that: • the epidemiologic findings on cancer were generally accurate • the benefits on cardiovascular disease had been overestimated – Results • Use of HRT decreased 40%-80% after the trial was stopped QuasiExperiment/Community Trial • Ranked immediately below controlled experiments in rigor • Investigator is unable to randomly allocate subjects to the conditions. • There may be contamination across the conditions of the study. Intervention Studies • An investigation involving intentional change in some aspect of the status of subjects • Used to test efficacy of preventive or therapeutic measures • Manipulation of the study factor and randomization of study subjects Intervention Studies • Two categories: – Clinical trials (focus on the individual) – Community trial or community intervention (focus on the group or community. • NOTE: Controlled clinical trials may be conducted both at the individual and community levels. Clinical Trials: Definition • A research activity that involves the administration of a test regimen to humans to evaluate its efficacy and safety • Wide variation in usage: – The first use of the term was for studies in humans without any control treatment – Now denotes a rigorously designed and executed experiment involving RANDOM ALLOCATION of test and control treatments Characteristics of Clinical Trials • Carefully designed and rigidly enforced protocol • Tightly controlled in terms of eligibility, delivery of the intervention, and monitoring out outcomes • Duration ranges from days to years • Participation is generally restricted to a highly selected group of individuals. Characteristics of Clinical Trials • Once subjects agree to participate, they are randomly assigned to one of the study groups, e.g., intervention or control (placebo) History of Clinical Trials • In 1537, Ambroise Paré applied experimental treatment for battlefield wounds. • East India Shipping Company (1600) found that lemon juice protected against scurvy. • James Lind (1747) used the concurrently treated control group method. History of Clinical Trials • Edward Jenner’s efforts to develop a smallpox vaccine in the late 18th century • Most recent historical developments include the use of multicenter trials. – Instrumental in the development of treatments for infectious diseases and recently in chronic diseases that are of noninfectious origin Prophylactic and Therapeutic Trials • A prophylactic trial evaluates the effectiveness of a substance that is used to prevent disease; it can also involve a prevention program. • A therapeutic trial involves the study of curative drugs or a new surgical procedure to improve the patient’s health. Outcomes of Clinical Trials • Referred to as clinical end points • May include rates of disease, death, or recovery • The outcome of interest is measured in the intervention and control arms of the trial to evaluate efficacy--these must be measured in a comparable manner. Examples of Clinical Trials • Medical Research Council Vitamin Study—studied role of folic acid in preventing neural tube defects. • South Bronx, NY, STD Program— evaluated effectiveness of education efforts to prevent spread of sexually transmitted diseases (STDs). Blinding (Masking) • To maintain the integrity of a study and reduce the potential for bias, the investigator may utilize one of two popular approaches: –Single-blind design: subject unaware of group assignment –Double-blind design: Neither subject nor experimenter is aware of group assignment Phases of Clinical Trials • Before a vaccine, drug, or treatment can be licensed for general use, it must go through several stages of development. • This lengthy process requires balance to: – protect the public from a potentially deleterious vaccine – satisfy the urgent needs for new vaccines Stages in the Development of A Vaccination Program • Pre-licensing evaluation of vaccine – Phase I trials: Safety of adult volunteers – Phase II trials: Immunogenicity and reactogenicity in the target population. – Phase III trials: protective efficacy • Post-licensing evaluation – Safety and efficacy of vaccine – Disease surveillance – Serologic surveillance – Measurement of vaccine coverage Phase IV Trials • There can be more than three phases in a clinical trial. • Phase IV trials involve post-marketing research to gather more information about risks and benefits of a drug. Randomization • Method of choice for assigning subjects to the treatment or control conditions of a clinical trial. • Non-random assignment may cause mixing of the effects of the intervention with differences (e.g., demographic) among the participants of the trial. Crossover Designs • Any change of treatment for a patient in a clinical trial involving a switch of study treatments • In planned crossovers a protocol is developed in advance, and the patient may serve as his or her own control. • Unplanned crossovers exist for various reasons, such as patient’s request to change treatment. Ethical Aspects of Human Experimentation • Benefits must outweigh risks. • Ethical issues: – Informed consent – Withholding treatment known to be effective – Protective the interests of the individual patient – Monitoring for side effects – Deciding when to withdraw a patient Reporting the Results of Clinical Trials • The CONSORT Statement is a protocol that guides the reporting of randomized trials by providing a 22item checklist and a flowchart. Summary of Clinical Trials • Strengths: – Provide the greatest control over: • the amount of exposure • the timing and frequency of exposure • the period of observation – Ability to randomize reduces the likelihood that groups will differ significantly. Summary of Clinical Trials (cont’d) • Limitations: – Artificial setting – Limited scope of potential impact – Adherence to protocol is difficult to enforce – Ethical dilemmas Community Trials • Community intervention trials determine the potential benefit of new policies and programs • Intervention: Any program or other planned effort designed to produce changes in a target population • Community refers to a defined unit, e.g., a county, state, or school district Community Trials (cont’d) • Start by determining eligible communities and their willingness to participate • Collect baseline measures of the problem to be addressed in the intervention and control communities • Use a variety of measures, e.g., disease rates, knowledge, attitudes, and practices Community Trials (cont’d) • Communities are randomized and followed over time • Outcomes of interest are measured Examples of Community Trials • • • • • North Karelia Project Minnesota Heart Health Program Stanford Five-City Project Pawtucket Heart Health Program Community Intervention Trial for Smoking Cessation (COMMIT) • Project Respect Summary of Community Trials: Advantages • They represent the only way to estimate directly the impact of change in behavior or modifiable exposure on the incidence of disease. Summary of Community Trials: Disadvantages • They are inferior to clinical trials with respect to ability to control entrance into study, delivery of the intervention, and monitoring of outcomes. • Fewer study units are capable of being randomized, which affects comparability. • They are affected by population dynamics, secular trends, and nonintervention influences. Four Stages of Evaluation • Formative: Will all plans and procedures work as conceived? • Process: Is the program serving the target group as planned? • Impact: Has the program produced any changes among the target group? • Outcome: Did the program accomplish its ultimate goal? Overview of Quasi-Experimental Study Designs Type of Study Design Group(s) Pretest Intervention Posttest Posttest only Intervention O X X Intervention X X X Intervention X X X Control X O X Intervention 1 X X X Intervention 2 O X X Control 1 X O X Control 2 O O X (has only one group) Pretest/Posttest (has only one group) Pretest/Posttest/Control (has two groups) Solomon Four-Group (has four groups) Note. O = not used; X = used. Quasi-Experimental Designs • Posttest only--observations are made only after the program has been delivered. • Pretest/Posttest--baseline and follow-up observations are made. • Pretest/Postest/Control--observations are made in both intervention and control groups before and after the program. Quasi-Experimental Designs (cont’d) • Solomon Four-Group assignment: – Used to overcome the Hawthorne Effect. – Uses four equivalent groups, two intervention and two control: • Two are observed before and after intervention. • Two are observed only after intervention. Chapter 9 Measures of Effect Learning Objectives • Explain the meeting of absolute and relative effects • Calculate and interpret the following measures: risk difference, population risk difference, etiologic fraction, and population etiologic fraction • Discuss the role of statistical tests in epidemiologic research • Apply Hill’s criteria for evaluation of epidemiologic associations Effect Measure • A quantity that measures the effect of a factor on the frequency or risk of a health outcome Three Effect Measures • Attributable Fractions – Measure the fraction of cases due to a factor. • Risk and Rate Differences – Measure the amount a factor adds to the risk or rate of a disease. • Risk and Rate Ratio – Measure the amount by which a factor multiplies the risk or rate of disease. Absolute vs. Relative Effects • Absolute – Attributable risk is also known as a rate difference or risk difference. – Population risk difference • Relative – Relative risk – Etiologic fraction – Population etiologic fraction Risk Difference (Attributable Risk) • Risk difference--the difference between the incidence rate of disease in the exposed group (Ie) and the incidence rate of disease in the nonexposed group (Ine). • Risk difference = Ie - Ine Calculation of Risk Difference • For women younger than age 75, the incidence (Ie) of hip fractures per 100,000 person-days was highest in the winter (0.41), and the incidence (Ine) was lowest in the summer (0.29). The risk difference between the two seasons (Ie - Ine) was 0.41 - 0.29, or 0.12 per 100,000 person-days. Population Risk Difference • Measures the benefit to the population derived by modifying a risk factor. Etiologic Fraction • Defined as the proportion of the rate in the exposed group that is due to the exposure. • Also termed attributable proportion or attributable fraction. Population Etiologic Fraction • Provides an indication of the effect of removing a particular exposure on the burden of disease in the population. • Also termed attributable fraction in the population. Statistical Measures of Effect • Significance tests • The P value • Confidence interval Null Hypothesis • Underlying all statistical tests is a null hypothesis, which states that there is no difference among the groups being compared. • The parameters may consist of the prevalence or incidence of disease in the population. Significance Tests • Used to decide whether to reject or fail to reject a null hypothesis. • Involves computation of a test statistic, which is compared with a critical value obtained from statistical tables. • The critical value is set by the significance level of the test. • The significance level is the chance of rejecting the null hypothesis when, in fact, it is true. The P Value • Indicates the probability that the findings observed could have occurred by chance alone. • However, a nonsignificant difference is not necessarily attributable to chance alone. The P Value (cont’d) • Possible meaning of nonsignificant differences: For studies with a small sample size the sampling error may be large, which can lead to a nonsignificant test even if the observed difference is caused by a real effect. Confidence Interval (CI) • A computed interval of values that, with a given probability, contains the true value of the population parameter. • The degree of confidence is usually stated as a percentage; commonly the 95% CI is used. • Influenced by variability of the data and sample size. Clinical vs. Statistical Significance • While small differences in disease frequency or low magnitudes of relative risk (RR) may be significant, they may have no clinical significance. • Conversely, with small sample sizes, large differences or measures of effect may be clinically important and worthy of additional study. Statistical Power • The ability of a study to demonstrate an association if one exists. • Determined by: Frequency of the condition under study. – Magnitude of the effect. – Study design. – Sample size. – Evaluating Epidemiologic Associations • Five key questions to be asked: – Could the association have been observed by chance? • – Determined through the use of statistical tests. Could the association be due to bias? • Bias refers to systematic errors, i.e., how samples were selected or how data was analyzed. Evaluating Epidemiologic Associations (cont’d) • • Could other confounding variables have accounted for the observed relationship? To whom does this association apply? – – • Representativeness of sample Participation rates Does the association represent a causeand-effect relationship? – Considers criteria of causality. Types of Associations between Factors and Outcomes • Not statistically associated (independent) • Statistically associated Statistical Association • When a factor and outcome are statistically associated, the relationship can be: – Non-causal – Causal • Indirect • Direct Multiple Causality • Also referred to as multifactorial etiology. • “…requirement that more than one factor be present for disease to develop…” Models of Multiple Causality • Epidemiologic triangle • Web of causation, e.g., in avian influenza • Wheel model, e.g., childhood lead poisoning • Pie model, e.g., lung cancer Chapter 10 Data Interpretation Issues Learning Objectives • Distinguish between random and systematic errors • State and describe sources of bias • Identify techniques to reduce bias at the design and analysis phases of a study • Define what is meant by the term confounding and provide three examples • Describe methods to control confounding Validity of Study Designs • The degree to which the inference drawn from a study, is warranted when account it taken of the study, methods, the representativeness of the study sample, and the nature of the population from which it is drawn. Validity of Study Designs • Two components of validity: Internal validity – External validity – Internal Validity • A study is said to have internal validity when there have been proper selection of study groups and a lack of error in measurement. • Concerned with the appropriate measurement of exposure, outcome, and association between exposure and disease. External Validity • External validity implies the ability to generalize beyond a set of observations to some universal statement. • A study is externally valid, or generalizable, if it allows unbiased inferences regarding some other target population beyond the subjects in the study. Sources of Error in Epidemiologic Research • Random errors • Systematic errors (bias) Random Errors • Reflect fluctuations around a true value of a parameter because of sampling variability. Factors That Contribute to Random Error • Poor precision • Sampling error • Variability in measurement Poor Precision • Occurs when the factor being measured is not measured sharply. • Analogous to aiming a rifle at a target that is not in focus. • Precision can be increased by increasing sample size or the number of measurements. • Example: Bogalusa Heart Study Sampling Error • Arises when obtained sample values (statistics) differ from the values (parameters) of the parent population. • Although there is no way to prevent a non-representative sample from occurring, increasing the sample size can reduce the likelihood of its happening. Variability in Measurement • The lack of agreement in results from time to time reflects random error inherent in the type of measurement procedure employed. Bias (Systematic Errors) • “Deviation of results or inferences from the truth, or processes leading to such deviation. Any trend in the collection, analysis, interpretation, publication, or review of data that can lead to conclusions that are systematically different from the truth.” Factors That Contribute to Systematic Errors • Selection bias • Information bias • Confounding Selection Bias • Refers to distortions that result from procedures used to select subjects and from factors that influence participation in the study. • Arises when the relation between exposure and disease is different for those who participate and those who theoretically would be eligible for study but do not participate. • Example: Respondents to the Iowa Women’s Health Study were younger, weighed less, and were more likely to live in rural, less affluent counties than nonrespondents. Information Bias • Can be introduced as a result of measurement error in assessment of both exposure and disease. • Types of information bias: – Recall bias: better recall among cases than among controls. • Example: Family recall bias Information Bias (cont’d) Interviewer/abstractor bias--occurs when interviewers probe more thoroughly for an exposure in a case than in a control. – Prevarication (lying) bias--occurs when participants have ulterior motives for answering a question and thus may underestimate or exaggerate an exposure. – Confounding • The distortion of the estimate of the effect of an exposure of interest because it is mixed with the effect of an extraneous factor. • Occurs when the crude and adjusted measures of effect are not equal (difference of at least 10%). • Can be controlled for in the data analysis. Criteria of Confounders • To be a confounder, an extraneous factor must satisfy the following criteria: Be a risk factor for the disease. – Be associated with the exposure. – Not be an intermediate step in the causal path between exposure and disease. – Simpson’s Paradox as an Example of Confounding • Simpson’s paradox means that an association in observed subgroups of a population may be reversed in the entire population. • Illustrated by examining the data (% of black and gray hats) first according to two individual tables and then by combining all the hats on a single table. Simpson’s Paradox (cont’d) • When the hats are on separate tables, a greater proportion of black hats than gray hats on each table fit. – On table 1: • 90% of black hats fit • 85% of gray hats fit – On table 2: • 15% of black hats fit • 10% of gray hats fit Simpson’s Paradox (cont’d) Simpson’s Paradox (cont’d) • When the man returns the next day and all of the hats are on one table: – 60% of gray hats fit (18 of 30) – 40% of black hats fit (12 of 30) Note that combining all of the hats on one table is analogous to confounding. Examples of Confounding • Air pollution and bronchitis are positively associated. Both are influenced by crowding, a confounding variable. • The association between high altitude and lower heart disease mortality also may be linked to the ethnic composition of the people in these regions. Techniques to Reduce Selection Bias • Develop an explicit (objective) case definition. • Enroll all cases in a defined time and region. • Strive for high participation rates. • Take precautions to ensure representativeness. Reducing Selection Bias Among Cases • Ensure that all medical facilities are thoroughly canvassed. • Develop an effective system for case ascertainment. • Consider whether all cases require medical attention; consider possible strategies to identify where else the cases might be ascertained. Reducing Selection Bias Among Controls • Compare the prevalence of the exposure with other sources to evaluate credibility. • Attempt to draw controls from a variety of sources. Techniques to Reduce Information Bias • Use memory aids; validate exposures. • Blind interviewers as to subjects’ study status. • Provide standardized training sessions and protocols. • Use standardized data collection forms. • Blind participants as to study goals and classification status. • Try to ensure that questions are clearly understood through careful wording and pretesting. Methods to Control Confounding • Prevention strategies--attempt to control confounding through the study design itself. • Three types of prevention strategies: – – – • Randomization Restriction Matching Two types of analysis strategies: – – Stratification Multivariate techniques Randomization • Attempts to ensure equal distributions of the confounding variable in each exposure category. • Advantages: – Convenient, inexpensive; permits straightforward data analysis. • Disadvantages: – – Need control over the exposure and the ability to assign subjects to study groups. Need large sample sizes. Restriction • May prohibit variation of the confounder in the study groups. – For example, restricting participants to a narrow age category can eliminate age as a confounder. • Provides complete control of known confounders. • Unlike randomization, cannot control for unknown confounders. Matching • Matches subjects in the study groups according to the value of the suspected or known confounding variable to ensure equal distributions. • Frequency matching--the number of cases with particular match characteristics is tabulated. • Individual matching--the pairing of one or more controls to each case based on similarity in sex, race, or other variables. Matching (cont’d) • Advantages: Fewer subjects are required than in unmatched studies of the same hypothesis. – May enhance the validity of a follow-up study. – • Disadvantages: – Costly because extensive searching and recordkeeping are required to find matches. Two Analysis Strategies to Control Confounding • Stratification--analyses performed to evaluate the effect of an exposure within strata (levels) of the confounder. • Multivariate techniques--use computers to construct mathematical models that describe simultaneously the influence of exposure and other factors that may be confounding the effect. Advantages of Stratification • Performing analyses within strata is a direct and logical strategy. • Minimum assumptions must be satisfied for the analysis to be appropriate. • The computational procedure is straightforward. Disadvantages of Stratification • Small numbers of observations in some strata. • A variety of ways to form strata with continuous variables. • Difficulty in interpretation when several confounding factors must be evaluated. • Categorization results in loss of information. Multivariate Techniques • Advantages: Continuous variables do not need to be converted to categorical variables. – Allow for simultaneous control of several exposure variables in a single analysis. – • Disadvantages: – Potential for misuse. Publication Bias • Occurs because of the influence of study results on the chance of publication. – Studies with positive results are more likely to be published than studies with negative results. Publication Bias (cont’d) • May result in a preponderance of false-positive results in the literature. • Bias is compounded when published studies are subjected to meta-analysis.
Purchase answer to see full attachment