Questionnaire
Due on July 20, 2018, 11:59PM EST
Residents in Norfolk, VA, have expressed concern about coal dust in their neighborhoods
(Ghent, Larchmont, West Ghent, and Lambert’s Point). The coal dust comes from the Lambert’s
Point coal terminal operations and loaded coal cars arriving from Appalachia.1 The Virginia
Department of Environmental Quality doesn’t monitor actual coal dust pollution levels in air.
The residents have submitted petitions to the Virginia Department of Health (VDH) about their
concerns and have asked the department to assess how coal dust affects their health. Coal dust
contains several heavy metals (arsenic, lead, mercury, nickel). Possible health effects include
asthma attacks, bronchitis, heart and lung disease in both adults and children, and impaired
neurological development in children. To respond to the request, the VDH has planned to
conduct a health assessment by collecting questionnaire s from residents in Norfolk to gain a
better understanding of environmental pollution and its associated health impacts. You work for
VDH as a health assessor and have been assigned to this case. Your main job responsibility is
to design a questionnaire, which can depict the health status of residents from coal dust
exposure and residents’ perspectives on coal dust pollution. Please use the principles you have
learned from this class to develop a questionnaire with the following categories:
1. Demographic information.
2. Environmental Exposure condition and history. You can gain perspectives from
residents about possible exposure to pollutants. Also, questions should be designed to
assess exposure duration and exposure doses/concentrations of coal dust. Data from
the questionnaire should be both quantitative and qualitative.
3. Health Status. Questionnaire questions should be developed to assess incidences of
health effects described in the background information. Health impacts include
respiratory and cardiovascular health/disease.
4. Possible Confounding factors, e,g, smoking, drinking, and age
The basic requirements and format of the questionnaire include that 1) its length should be no
more than 10 pages, and 2) questions should include both open-ended and fill-in-the-blank.
You can simple develop a questionnaire in a word document or use the template at
https://www.questionnaire monkey.com/ Once you complete the development of the
questionnaire , please list the principles at the end of your questionnaire , as described in the
PowerPoint slides, that you have applied on the last page of your questionnaire .
You have an option to develop your own synopsis for a questionnaire. This synopsis should
relate to your study. Your questionnaire shall still cover the basic requirements and format as
described in the paragraph above.
Questionnaires
Slide 2 Learning Objectives
By the end of this class, students will be able to
• Describe the steps to develop a questionnaire
• List the five categories of information typically included in a questionnaire and give
examples of each
• Describe the three general types of questions and identify the best situation in which to
use each type
• List five considerations for wording questions
• List the primary methods for administering a questionnaire and the advantages and
disadvantages of each method
• List five tips for effective interviewing
• Discuss the ethical and privacy concerns related to conduct interview
• Identify ways to protect the confidentiality of a respondent
This lecture focuses on effective information gathering to understanding human exposure to
environmental toxins. It explores how to create a questionnaire which information to collect,
how to write and organize the questions, and how to administer the questionnaire.
Slide 3 Survey vs. questionnaire
• Terms “survey” and “questionnaire” are often used interchangeably
– “Survey” refers to a descriptive research technique or methodology (also
includes interviews, and focus groups)
– “Questionnaire” refers to a paper and pencil instrument through which
information is obtained
• For the purposes of outcomes assessment, questionnaires are frequently developed to
survey a population
Slide 4 Developing a questionnaire
Investigators should follow these steps when they develop a questionnaire:
1 identify the leading objective or hypothesis about the source of the problem
2. identify the information needed to address the objective or test the hypothesis
3. identify the information needed for the logistics of the study
4. write the questions to collect this information
5. organize the questions into questionnaire format
6. determine how the information gathered will be managed
7. test the questionnaire
8. revise the questionnaire
9. administer the questionnaire (after training interviewers)
Let’s walk through the steps of writing, organizing, and administering a questionnaire.
Slide 5 How questionnaire design fits into the overall process of research design
The figure shows how questionnaire design fits into the overall process of research design. The
first step is to define problem that leads to specific research objectives, following hypothesis to
be tested.
Slide 6 Step I: Identify objective(s) and leading hypothesis
Before writing the first question of a questionnaire, it is very important for investigators to
understand what they are looking for. To do so, the investigators needs to identify the
objective of the investigation and leading hypothesis, which include what you want to ask and
of whom and consider how the data will be analyzed, what information is needed, and account
for any factors that might confound the results. For example, in the E coli outbreak,
investigators conducted hypothesis-generating interviews that helped identify consumption of
contaminated alfalfa sprouts or salad as the most likely cause of illness. Keeping the hypothesis
in mind allowed them to design a well-structured questionnaire that focused on when people
were likely infected, what they ate, where they ate it, and where the food came from, as well as
other activities and risk factors that would help prove or disprove the hypothesis.
Slide 7 Step 2 - identify the information needed to address the objective or test the hypothesis
After identifying the leading hypothesis, but before writing questions and organizing them into
a questionnaire, investigators must identify the information they need. Because a hypothesis is
a carefully constructed supposition about the relationship between an exposure and a disease
or health event, there are specific areas of information that should be defined for collection of
information to effectively test the hypothesis:
• One or more proposed exposures
• One or more health outcomes of interest
• Boundaries on the person/population at risk (i.e., who was exposed or developed the
outcome of interest)
• Boundaries on place (i.e., where the exposure or outcome of interest occurred)
• Boundaries on time (i.e., when the exposure or outcome of interest occurred)
Slide 8. Step 2 - identify the information needed to address the objective or test the hypothesis
(cont)
To collect information to address these components of testing a hypothesis, investigators often
collect information in several categories: demographic information to define the population
and the place; exposure or risk factor information to define the exposures being investigated,
including place and time; and clinical information to define the health outcome of interest,
including place and time.
Demographic information characterizes the at-risk population by age, sex, highest education
level attained, location and other identifying information. Such information is useful in
describing the problem under investigation. It is also important in the search for possible
confounders, or factors that can distort the apparent exposure-disease relationship.
Demographic categories such as age, sex, education level, and location can be confounders;
investigators need to evaluate them to determine whether demographic and other factors
affect the relationship they see –or think they see- between an exposure and a disease in a
study.
Slide 9 Example-demographic information
A researcher conducted a project to test whether polycyclic aromatic hydrocarbons (PAHs)
from coke-oven emissions affect male reproductive health. She hypothesized PAHs can
decrease sperm motility and morphology. Human subjects were coke-oven workers working at
a steel plant. She developed a questionnaire for this study. The first section of the questions
aims to collect demographic information including age, education, marital condition, and BMI.
Slide 10 Clinical Information
Clinical information includes signs and symptoms of the disease, date of onset of illness, receipt
of medical care or treatment, and results of laboratory testing. This information allows
investigators to characterize the illness, decide who has the outcome of interest, and chart the
time course of the problem. Determining who has developed the outcome of interest is critical
in a cohort or cross-sectional study. It will also be important in a case-control study, as
investigators need to verify the outcome status of participants. It is not uncommon to find out
that some cases do not meet the case definition or that some individuals considered as controls
do not fit the control criteria.
Slide 11 Example-Clinical Information
Let use the formetioned project as an example, the researcher investigated male reproductive
health in relation to PAH exposure. She developed some questions in a questionnaire to collect
clinical information. She used this information to verify the outcome status of participants.
Since her study was related to male reproductive health, it makes sense that the questions
should generate information about reproductive status and other diseases related to
reproductive status, including sperm quality. Below are some questions included in this
questionnaire, e.g. cancer treatment, any clinical procedures on reproductive organs.
If the researcher collaborated with physicians, she may get clinical information from patients’
records. However, due to patient privacy issue, Internal of Board of an institute generally
requires closely evaluation whether the procedures gaining the information deals with privacy
matter. We will discuss IRB review in details later.
Slide 12 Exposure or risk factor information
Exposure or risk factor information is used to explore the hypothesis under investigation. This
information will likely be the focus of the questionnaire for an analytic study, such as a casecontrol or cohort study. The data collected will be very specific for the problem under
investigation and will reflect the hypothesis about the source of the problem. Exposure or risk
factor information often includes the following items:
• Exposure (or not) of the respondent to the factor of interest
• Route of exposure
• Amount of exposure
• Timing of exposure
• Other details of exposure, such as the brand and source of a contaminated food
Slide 13-Example- Exposure or risk factor information
As an example, let’s return to the E.coli 0157:H7 outbreak that was detected in Michigan in
June 1997 and the leading hypothesis that people became ill from contaminated alfalfa sprouts
or salads. What information do investigators need to explore and test this hypothesis?
Obviously, it is important to know if the respondent ate either of these items. In addition,
investigators want to know when people ate the alfalfa sprouts or salads, as eating these items
before or after the period of interest are unlikely to be of concern. Because a dosage effect
might affect the outcome, it is also important to know how much was eaten. Even a small
number of E.coli 0157:H7 organisms can lead to infection, however, so exploring the amount
eaten is unlikely to be of great interest in this outbreak, but it might be of concern with other
pathogens. Finally, if alfalfa sprouts or salads are the cause of the outbreak, produce from a
particular source might be the cause of the problem. Therefore, investigators need to learn
where the respondent bought or ate these items and if they can trace them to a particular
producer or distributer. Table 8-1 summarizes these pieces of information.
Slide 14 Example- Exposure or risk factor information (cont’)
The Table summaries these pieces of information. As shown on this Table, questions in a
questionnaire to test hypotheses may include “ Did you eat either of these items?”; When did
you eat them? How much did you eat?
Slide 15 Step 3-Identify the information needed for the logistics of the study
Identifying information includes the respondent’s name or other personal identifiers, address,
and telephone number. This information is important for the logistics of the study. It is used to
identify the subject and allows investigators to update the questionnaire as more information
becomes available. Identifying information can be used to link the questionnaire to other
records, such as laboratory results or other questionnaires. It also can prevent duplicate entry
of records if an individual is interviewed twice during a study or if questionnaires get mixed up
during data entry.
Slide 16 Step 3-Identify the information needed for the logistics of the study (cont’)
The source of the information is the individual supplying the information as well as the person
receiving it (i.e., the interviewer). Identifying the person supplying the information provides
some insight into the information’s validity. Is the person providing the information the study
subject or a surrogate such as the subject’s spouse or parent? In most situations, the actual
subject of the study will provide better information than others who speak on behalf of the
subject, although if the affected individuals are infants, children, or deceased persons someone
else will have to speak for them. Identifying the interviewer is also important. If any questions
are left unanswered, the handwriting is illegible, or some responses are nonsensical, the
interviewer might be able to help correct these problems. Tracking the interviewer can also
help identify a more serious problem: An inadequately trained interviewer may consistently
misread a question, interpret a question inappropriately, or misquote a respondent’s answers.
Analyzing the data and stratifying the information by interviewer may reveal patterns
suggesting this very problem. If interviewers are properly trained, this problem will be greatly
minimized, but investigators must take the possibility into account.
Slide 17 Write the questions
After identifying the leading hypothesis and determining the information they need to test the
hypothesis or for the logistics of the study, investigators can begin writing questions. The
questions on an epidemiologic questionnaire are typically arranged into five categories of
information:
• Identifying information
• Demographic information
• Clinical information
• Exposure or risk factor information
• Source of information
Whoever writes the questionnaire will determine the exact format and wording of the
questions included based on his or her knowledge of the problem- that is, the hypothesis being
tested, the kind of information to be collected, and the affected population. Each epidemiologic
investigation will have differentiated requirements, but we can make some generalizations
about what types of questions to ask and how to construct them.
Slide 18 Types of questions
Investigators use three types of questions in questionnaires for epidemiologic studies:
• Open-ended questions
• Fill-in-the blank questions
• Closed-ended questions:
o With categorical response options
o With ordinal response options
Slide 19 Open-ended questions
Open-ended questions allow the respondent to say whatever he or she desires, rather than
requiring a response based on a specific list of choices; the possible responses are limitless.
Open-ended questions give the respondent the opportunity to express his or her own
perspectives about a subject. They are useful in characterizing attitudes, beliefs, and behaviors.
(They are also useful during hypothesis generation, when investigators know little about a
problem and the range of potential responses to a question is largely unknown.) From the
investigator’s perspective, however, analyzing open-ended questions can be difficult and timeconsuming. Unless the investigator intends to supply only a narrative report of the responses,
he or she must read, categorize, and code every response. As a result, investigators tend to
limit their use of this question type, particularly in studies involving large numbers of subjects.
Slide 20 Open-ended questions (cont’)
As an example of how open-ended questions work, consider the questionaire that the Centers
for Medicare & Medicaid Services (CMS) undertakes each year. Among other things, the survey
seeks to determine influenza vaccination rates among Medicare beneficiaries and most
frequently cited reasons for not receiving influenza vaccine. Respondents are asked, “Did you
have a flu shot for last winter? Those who respond “No” are then asked, “Why didn’t you get a
flu shot for last winter?” Respondents are free to give any reason or reasons. The following
responses are fictitious but might be similar to the answers given by some respondents:
• I did not get the vaccine because I had read a report in the newspaper that is could
make you sick. Some of these illnesses are minor, like pain or redness where the vaccine
is injected or a slight fever, but others are serious. In fact, the report said that some
people developed a disease called Guillain-Barre syndrome due to the flu vaccine, and
one elderly man died.
• I have macular degeneration and don’t drive. I have a difficult time getting to the
doctor’s and I have to ask my friend to drive me. My doctor said I should get vaccinated
•
in October and I wrote it on the calendar, but I didn’t want to impose on my friend. She
is so busy and her husband is quite ill. Anyway, because I don’t get out much – just to
church and the grocery store – I don’t think I need the vaccination.
I got flue the last time I got the flu shot. I started feeling bad as I was driving home from
the clinic. I felt kind of achy and then my throat got scratchy. I was laid up in bed for
days. My doctor tells me I should get the flu shot every year but I won’t make the same
mistake again.
Slide 21 Example - open-ended questions (cont’)
To determine the primary reasons that a respondent did not get vaccinated against influenza,
the interviewers then coded each response into one of the following categories:
A. Did not know the shot was needed
B. Shot could cause flu
C. Shot could have side effects or cause disease
D. Didn’t think it would prevent the flu/could cause flu anyway
E. Flu not serious/would not get flu anyway/ not at risk
F. Doctor did not recommend the shot
G. Doctor recommended against getting shot/ allergic to shot/ medical reasons
H. Don’t like shots or needles/ concerns about soreness or rash/ local reactions
I. Inconvenient to get shot/ unable to get to location
J. Did not think about it/ forgot/ missed it
K. Cost of shot/ not worth the money
L. Had shot before/didn’t need it again
M. Other (specify)
N. Refused
O. Don’t know
As you can imagine, it would take a fair amount of time to code these data. Thus, in general,
investigators should use this type of question only when it is absolutely necessary.
Slide 22 Fill-in-the-blank questions
Fill-in-the-blanks, like open-ended questions, do not provide any response choices. With fill-inthe-blanks, however, it is expected that the response will be relatively short, such as one or two
words. These questions should be used when the possible response categories are numerous
and the question measures a simple respondent attribute, such as age, educational level, or
place of residence; when collecting a date, such as birth date, date of onset of illness, or date of
exposure; or when quantifying something specific, such as a number of visits to a healthcare
provider. Fill-in-the-blanks do not work well when longer, more involved responses are
expected.
Slide 23 Example – open-ended questions
In which county do you live? _________________ (DK = don’t know, R = refused)
What is your date of birth? _______________ (mm/dd/yyyy) (DK = don’t know, R = refused)
When did you receive your pneumonia vaccination? ______________ (mm/dd/yyyy)
(DK = don’t know; R = refused)
How many times did you see your healthcare provider in the last year? _____________
(DK = don’t know; R =refused)
Like open-ended questions, fill-in-the-blank questions often require that the investigator
categorize and code the responses. However, because the responses are generally short, the
burden is not as great as with open-ended questions.
Slide 24 Close-ended questions
Close-ended questions have response choices from which the respondent must choose. They
are used when possible responses are known and the range is relatively narrow (that is,
responses can be provided to respondents in a short list). Unlike open-ended and fill-in-theblank questions, close-ended questions do not require the investigator to categorize and code
the responses. They do require him or her to satisfy the following conditions:
• Anticipate the most likely responses to the question
• Present the responses as a list of mutually exclusive choices
• State the responses in a clear and concise manner that respondents can easily
understand
Slide 25 Close-ended Questions
Close-ended questions can be either categorical or ordinal (although the difference between
the two may not be evident until the data are analyzed). In categorical close-ended questions,
the available responses fit into categories that have no particular order or inherent numerical
value with respect to each other. For example, typical categorical question related to
demographics are yes/no, male/female. In general, answers fall into certain categories and
respondents pick the one that applies to them. Investigators usually analyze categorical data
using frequency, proportion, and percentages. There are two considerations. One is that
questions on a continuum should stay on a continuum if they will be directly analyzed. For
example, age – allow respondent to write in their age rather than check off a box for their age
range. Second, Categories must be collectively exhaustive & mutually exclusive (next slide)
Slide 26 Categories – Mutually Exclusive
Below information is an example that categories must be mutually exclusive
Slide 27 Example – close-ended questions
Here is an example of open-ended question used on the Centers for Disease Control and
Prevention’s (CDC’s) Behavioral Risk Factor Surveillance System (BRFSS) questionnaire .
Which one or more of the following would you say is your race?
1. White
2. Black or African American
3. Asian
4. Native Hawaiian or other Pacific Islander
5. American Indian or Alaska Native
6. Other [specify]
7. Refused
In this example, the response does not have any numerical relationship to each other. “White”
bears no quantitative relationship to “Black or African American,” which bears no quantitative
relationship to “Asian,” and so on.
Slide 28 Ordinal close-ended Questions
In ordinal closed-ended questions, the available responses tend to describe a range of choices
along a continuum and to have a numerical value with respect to one another. For example, in
seeking information about a respondent’s education level, the BRFSS uses the following ordinal
close-ended question:
What is the highest grade of year of school you competed?
1. Never attended school or only attended kindergarten
2. Grades 1 through 8 (Elementary)
3. Grades 9 through 11 (Some high school)
4. Grade 12 or GED (High school graduate)
5. College 1 year to 3 years (Some college or technical school)
6. College 4 years or more (College graduate)
7. Refused
Clearly, the numbered responses are related to one another with respect to the level of
education a respondent has had: The lower-numbered responses suggest less formal education
and the high-numbered responses suggest more formal education.
All three question types – open-ended, fill-in-the-blank, and close-ended – have their place.
Investigators select which type to use based on the kind of information they wish to collect and
their expectations and prior knowledge of that information. Early in an investigation, when
investigators may know little about the problem, they are more likely to use open-ended
questions. As they gather more information, they should be able to anticipate possible
responses and, therefore, become more likely to use close-ended questions.
Slide 29 Wording the questions
Choosing the type of question is just the first step in creating the question. Investigators need
to take great care with the specific wording and the response categories (if applicable).
Following are a few guidelines to consider when writing questions for an epidemiologic study:
• Use language that the respondents can understand. Avoid technical jargon, slang, and
abbreviations. Depending on the level of education of respondents, investigators might
need to further simplify the language. For example:
Use: Has a physician or other healthcare provider measured your blood pressure to
determine if you have high blood pressure?
Not: Have you been evaluated for hypertension?
• If the questionnaire is translated from one language to another, investigators should
test it with native speakers. A good test is to have the translated questionnaire “backtranslated” --- that is, translated from the language intended for use with the
respondents into the language used by the investigators. Backtranslation allows the
investigators to see or hear what the respondents will be reading or hearing and may
reveal potential problems.
• Limit each question to a single idea. Combining questions can result in unclear or
conflicting responses. For example:
Use: Have you had vomiting? Have you had diarrhea as defined by three or more loose
stools in a 24-hour period?
• Word each question as precisely as possible. Avoid words with vaguely defined
meanings, leaving as little to the respondent’s interpretation as possible. Exercise
caution with general adjectives and adverbs that may have different meanings to
different people, such as “big,” “bad,” and “nicely.”
Use: Have you had three or more loose stools in any 24-hour period between April 25
and May 1?
Not: Did you severe diarrhea?
• Do not phrase questions in a way that suggests a response -- that is, as a leading
question – or implies a value judgment. How respondents perceive the interviewer’s
attitude can influence the responses they provide. For example:
Use: Has your child been immunized against measles?
Not: Childhood immunizations are important ways parents can protect their children
against serious diseases. Has your child been immunized against measles?
Slide 30 Wording the Questions
• Avoid double negatives. For example:
Use: Do you always buckle your seat belt while riding in a car?
Not: Do you never go in a car without buckling your seat belt?
• Always include a “Don’t know” or “Refused” category with potential responses. This
option will help distinguish between respondents who do not answer a question
because they do not remember the answer, choose not to answer, or just skipped the
question.
Slide 31 Wording the Questions
When creating the responses for a close-ended question, be sure that the categories cover all
potential responses and are mutually exclusive (that is, they do not overlap). Each response
should fit into one and only one category. For example:
Which of the following age categories do you fit into?
A. 1 – 20 years
B. 20 – 40 years
C. 40 – 60 years
D. Greater than or equal to 60 years
E. Don’t know
F. Refused
These categories do not have a place for an infant younger than one year of age. Also, they are
not mutually exclusive because they fail to indicate whether someone who is 40 years old
would fit into category B or C; there are similar problems for people aged 20 or 60 years. The
following categories are clearer:
A. Less than 20 years
B. 20 – 39 years
C. 40 – 59 years
D. 60 years or older
E. Don’t know
F. Refused
Slide 32 Organize the Questions into Questionnaire Format
Once investigators know the categories of information collected in an epidemiologic
questionnaire and the general types of questions, they can turn their attention to assembling
the questions into a questionnaire. This section includes a number of thoughts about what
makes a good questionnaire and helps it flows smoothly. The following several slides cover the
following thoughts:
Introduction the questionnaire
Management the length of the questionnaire
Ask only essential questions
Organization the questions
Slide 33 Introduction the Questionnaire
Each questionnaire should begin with an introduction. The introduction serves several
purposes. It allows investigators to identify the organization sponsoring the study and explain
the purpose of the study. Both pieces of information will lend credence to the undertaking and
increase the likelihood that the respondent will participate and answer honestly. To avoid
biasing the respondent, the introduction should not provide too much information about the
investigation, such as the suspected source of the problem. Investigators can and should
explain the purpose of the study in general terms to help the respondent understand the
importance of the interview and his or her part in the investigation. The introduction should
also include information about how long the interview is likely to take and reassurance that the
respondent’s answers will be held in the strictest of confidence.
Here is an example of the introduction to a study questionnaire, taken from a sample
questionnaire on the CDC’s OutbreakNet website
(http://www.cdc.gov/outbreaknet/toolkit/standard_ques.pdf):
Hello. My name is _______________________ and I’m calling from the ____________State
Health Department. I’m calling because there have been several cases of _______________ in
our community and we are working to identify the source of infection, so we can prevent
additional illness in the community. We understand that you are one of the people who had
this illness. I would like to ask you some questions about your illness and foods that you are
before becoming ill, that will help us in this work. This will take about _____ minutes. Can we
go ahead?
Slide 34 Manage the length of the questionnaire
In general, a questionnaire should be as short as possible and should focus on the hypothesis
being tested in the study. Two important reasons to limit the length of a questionnaire are the
resources needed to administer it and the willingness of respondents to complete it. Asking a
great number of questions may also lead to another problem, called “multiple comparisons”:
Comparing the study group and the control group on too many factors can increase the
likelihood that investigators will find a statistical association between an exposure and the
disease under investigation due to chance alone. Try to keep the length of questionnaire for
interview within 25-30 minutes.
Slide 35 Ask Only Essential Questions
Investigators need to prioritize the questions to include in the questionnaire, focusing on the
leading hypothesis and the discrete pieces of information needed to evaluate it. Only then
should they try to strike a balance between evaluating the hypothesis and taking advantage of
the opportunities that access to the study population permits. The questionnaire should include
only questions that are important to resolving the health problem, rather than also including
questions whose answers are just interesting to know.
Slide 36 Organize the Questionnaire
When investigators lay out a questionnaire, they should organize the questions to help the
interviewer develop rapport with the respondent. The best means to do so depends largely on
the nature of the questionnaire, the problem being explored, and the persons being
interviewed. When designing a questionnaire, investigators may want to use these formatting
tips:
• Group together similar types of information or topics.
• Ask more general questions first, followed by more specific questions.
• Lead with the least sensitive questions, leaving the most personal or potentially
threatening questions – those regarding sexual habits, religious beliefs, political
orientation, or income – for last, when the respondent may be more comfortable with
the interviewer.
• Ask the most important questions – those relating to the leading hypothesis - first.
Regardless of which approach investigators take, the questionnaire should be logical and well
organized. It should also be easy to read and complete. Instructions for the questionnaire or
specific questions should be clearly stated. Questions and pages should be numbered. Each
page should include an identifying code for the respondent. Responses to questions, or the
space for recording responses, should be clearly separated from the questions.
Slide 37 Include Skip Patterns
The questionnaire should include “skip patterns” to avoid asking irrelevant questions. A skip
pastern usually begins with a screening question that tells the interviewer whether subsequent
questions pertain to a particular respondent. If they do, the interviewer reads the questions. If
not, the interviewer is instructed to skip over the questions and continue with the next set.
Here is an example of a skip pattern. Below is an example of questionnaire including questions
related to work history.
As you can see, use of skip patterns can prevent a respondent from listening to and having to
answer many questions that don’t pertain to him or her. They can save time and are less
wearing on the patience of the respondent. Skip patterns should always be clearly marked and
easy to follow.
Slide 38 Include an Ending Statement
The questionnaire should conclude with an ending statement. The interviewer should thank
the respondent for his or her input and time, and provide the respondent with a means to
contact the interviewer or the study investigator if her or she has questions or remembers
additional information pertinent to the investigation. Providing a contact number will also
reassure the respondent that the interview was part of a legitimate undertaking. Here are two
examples of language to include an ending statement:
• That was last question. I realize that I have asked you a lot of questions. Do you have
any questions or comments for me?
• Thank you very much for your time and cooperation. Your assistance with this
investigation may help us prevent further case of ____ in this community. If you think
of something else and want to reach us, please leave a message for [NAME OF PRIMARY
INVESTIGATOR] at xxx-xxx-xxxx and we will get back to you.
Slide 39 Determine How the Information Will Be Managed
If investigators plan to enter the questions and answers form their investigation questionnaire
into an electronic database, they need to think about how they will arrange the raw data (i.e.,
the answer from a questionnaire) into ready-to-analyze data. The answers to each of these
questions are variables that must be assigned a value, coded, and possibly grouped or
categorized so that investigators can analyze the information. Thinking about how they will
code the responses while designing their questionnaire, before any data are gathered, will help
investigators collect the data in a format they can use.
Slide 40 Coding
Coding is the process of translating the information gathered from questionnaires or other
investigations into something that can be analyzed, typically using a computer program. Coding
involves assigning a value to the information obtained from a questionnaire; often that value is
given a label. In addition to helping organize data, coding can make the data more resistance.
A common coding system (code and label) for dichotomous variables is this:
0 = No
1 = Yes
Similarly, we might code the dichotomous variable for sex:
0 = Female
1 = Male
Many analysis software packages allow users to attach a label to variable values. The computer
automatically labels the zeros as male and the ones as female in the previous example. This
type of labeling can make it easier to understand the output as shown here:
Without Label: Variable SEX Frequency Percent:
0 21 60%
1 14 40%
The coding process is similar for other categorical variables. For the variable “education,” we
might code the responses as follows:
0 = Did not graduate from high school
1 = High school graduate
2 = Some college or post-high school education
3 = College graduate
Slide 41 Test Questionnaire
No matter how careful investigators are in developing questionnaires and phrasing each
question, sometimes what seems like a clear question to a public health specialist may baffle
the respondent. It is good practice – and often a critical step – to allow others to review the
questionnaire and provide comments, and to pre-test a questionnaire before administering it to
the intended group of respondents. Pre-testing is also referred to as “pilot testing” or “ field
testing.”
The pre-test helps investigators assess all aspects of the questionnaire, including clarity,
wording, sequence, format, and instructions. It can also help investigators determine the time
required and resources needed to complete the interviews. Here is a list of questions a pre-test
of a questionnaire can help answer:
Are all of the words easy to understand?
Do all respondents interpret the questions the same way?
Are skip patterns followed correctly?
Does any part of the questionnaire suggest bias on the interviewer’s part?
Does the questionnaire create a positive impression that motivates people to respond?
Do the responses seem appropriate and consistent with the intent for the question?
Is each question yielding the intended information?
How long did it take to complete the questionnaire?
The final step before administrating the questionnaire is to revise it based on comments from
colleagues and the pre-test.
Slide 42 Administering the Questionnaire - Format
A study questionnaire is merely an instrument. The questionnaire must be properly
administered to ensure the validity and reliability of the information collected. To make sure
the questionnaire is administered properly, investigators must select the best format to use,
train interviewers if necessary, and take steps to protect the rights and welfare of the
respondents.
Format
Two general formats of questionnaire administration are distinguished: A questionnaire can be
administered by an interviewer or self-administered (or some combination of the two). In an
interviewer-administered questionnaire, the interviewer reads the questionnaire to the
respondent, including instructions and potential response categories, if appropriate, and
records the responses. Such an interview can take place in person or by telephone. In a selfadministered questionnaire, the respondent reads the questionnaire and records his or her
own responses. A self-administered questionnaire can be either paper or computer based.
Slide 43 Strengths and Weaknesses of Interview-Administered Versus Self-Administered
Questionnaires
Each form of questionnaire administration has its advantages and disadvantages. In the
interview-administered questionnaires, the interviewer can explain or clarify items in the
questionnaire. He or she can ensure that respondents complete all items on the questionnaire
and, if necessary, prompt the respondent. Interviewer-administered questionnaires can also
use more complex designs, such as skip patterns, than self-administered questionnaires,
although computer-assisted self-administered questionnaires can often be programmed to
follow predetermined skip patterns and provide some of the prompts than an interviewer
might provide. Interviewer-administered questionnaires tend to have higher response rates
than self-administered questions, with more accurate recording of responses. Moreover, faceto-face interviews tend to have higher response rates than telephone interviews.
On the downside, interviewer-administered questionnaires are more costly and more time
consuming to administer than self-administered questionnaires. They require interviewers to
locate subjects and arrange a time to meet and/or talk. Interviewer-administered
questionnaires may seem less anonymous to respondents than self-administered
questionnaires. As a result, respondents may be less honest in their responses- some may be
more likely to give answers they think the interviewer wants to hear. Finally, the intervieweradministered questionnaire carries a risk that the interviewer will interject his or her own
feelings or interpretations as he or she reads the questions to respondents or records their
answers.
Slide 44 Selection of Which Type of Questionnaire to Use
When they decide which type of questionnaire to use, investigators should weight the relative
advantages and disadvantages of each format. They should also consider the available
resources – financial and staff – and determine how quickly they need to complete the
interviews. They should also keep in mind that the form of administration they select will affect
the instructions provided to respondents and the physical layout of the questionnaire.
Training interviews
If investigators decide to use interviewers to collect the study information, they have yet
another step to undertake: training the interviewers. To be effective, interviewers need to
know the questionnaire thoroughly (especially the intent of the questions), the way in which it
is laid out, and its skip patterns. They need to be able to pronounce the words in the
questionnaire and move from question to question in a smooth and effortless manner.
Interviewers also need to be trained to record responses in the format the study designers have
selected.
Training interviewers is critical to good questionnaire administration. An excellent way to begin
training is to read through the questionnaire as a group, allow interviewers to ask questions
about the questionnaire, and then have them practice administering the questionnaire to
others. Interviewers can also benefit from watching a more experienced interviewer complete
an interview. Some investigators begin the training of interviewers while the questionnaire is
still in development, so they can use feedback from interviewers to improve the questionnaire
Slide 45 Human Subjects and Ethical Considerations
Investigators are responsible for protecting the rights and welfare of the people whom they
recruit to participate in their studies. This responsibility affects many aspects of an
epidemiologic investigation, not just data collection. This section touches on the main points
briefly as they affect questionnaire development and administration.
Privacy
• Investigators should do everything in their power to ensure the privacy of each
respondent and the confidentiality of his or her responses. To facilitate this outcome,
investigators should take the following steps:
• Administer the questionnaire in a private setting in which conversations cannot be
overheard
• Interview the respondent alone, unless the presence of another person is required, and
then only with the explicit permission of the respondent.
• Do not share information gained from an interview, except with others who need to
know for the purposes of the investigation and as allowed by law.
• Keep completed or partially completed questionnaires out of the sight of others, ideally
in a locked filing cabinet or desk drawer (or in password-protected files).
• Consider removing identifying information from the completed forms once the
information is no longer needed for the logistics of the study.
• Follow all federal, state, or institutional requirements to protect respondent’s
confidentiality.
Slide 46 National Regulations and Institute enforcement
The Health Insurance Portability and Accountability Act (HIPAA) established national standards
for electronic exchange, privacy, and security of certain health information. In particular, the
HIPAA Privacy Rule limits the use and disclosure of individually identifiable health information
without authorization by the individual. HIPAA recognizes, however, the need for public health
authorities and others responsible for ensuring public health and safety to have access to
protected health information to carry out their public health mission.
Each institute has review procedures in place to protect the privacy of human subjects. All
investigators are required to submit documents related to proposed research involving human
subjects for review by an institutional Review Board (IRB). Old Dominion University has
established an Institutional Review Board (IRB) and five college committees to review all
proposed research involving human subjects to ensure that the subjects' rights and welfare are
adequately protected. The IRB is composed primarily of faculty members from disciplines in
which research involving human subjects is integral to that discipline's work, as well members
from the community whose primary interests are non-scientific. The human subjects review
process is administered through the ODU Office of Research. Researchers are REQUIRED to
comply with the Old Dominion University Procedures for Review of Human Subjects Research.
This information and related regulations can be founded at the link:
http://www.odu.edu/ao/research/compliance/humans.shtml
All human subject researchers, research administrators, and review committee members must
obtain adequate training. Online training is available via the Collaborative Institutional Training
Initiative (CITI) website. In this CITI training, investigators will be aware of HIPPA regulations
related to human subjects’ privacy and all necessary requirements and procedures. IRB will
determine whether informed Consent Document is needed before collecting questionnaire. If
needed, human subjects should be informed of the objective of this study, and an risks involved
with their participation. Human subjects need to voluntarily sign the consent form before
collecting questionnaire.
Slide 47 Summary
The most important steps in constructing a questionnaire for an exposure assessment
investigation are to carefully frame and refine the hypothesis under investigation and to decide
which information is needed to test. Develop and administering a questionnaire to collect
information to exposure assessment comes with many challenges and great responsibilitiesincluding the investigators’ legal and ethical responsibilities to the people they want to study.
Purchase answer to see full
attachment