The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1176-6093.htm
Mixed methods research
in accounting
Mixed methods
research in
accounting
Jennifer Grafton and Anne M. Lillis
Department of Accounting and BIS, The University of Melbourne,
Melbourne, Australia, and
5
Habib Mahama
School of Accounting and BIS, The Australian National University,
Canberra, Australia
Abstract
Purpose – The purpose of this paper is to set the scene for this special issue by synthesising the vast
array of literature to examine what constitutes mixed methods research, and the associated strengths
and risks attributed to this approach.
Design/methodology/approach – This paper takes the form of a literature review. The authors
draw on extensive methods research from a diverse range of social science disciplines to identify and
explore key definitions, opportunities and risks in mixed methods studies. They review a number of
accounting studies that adopt mixed methods research approaches. This allows the authors to analyse
variance in how mixed methods research is conceptualised across these studies and evaluate the
perceived strengths and limitations of specific mixed methods design choices.
Findings – The authors identify a range of opportunities and challenges in the conduct of mixed
methods research and illustrate these by reference to both published studies and the other contributions
to this special issue.
Originality/value – With the exception of Modell’s work, there is sparse discussion of the application
and potential of mixed methods research in the extant accounting literature.
Keywords Research methods, Accounting
Paper type Literature review
1. Introduction
Mixed methods research has a long history in the social sciences (Creswell, 2009; Jick,
1979; Johnson et al., 2007). The management literature abounds with studies that adopt a
mixed methods approach and methodological papers that examine the properties of this
research strategy (Greene, 2008; Tashakkori and Creswell, 2007b). Despite the
development of mixed methods research designs in the social sciences over several
decades and the recent growth in the popularity of mixed methods research as a “third
methodology” (Hall and Howard, 2008) or “third paradigm” (Denscombe, 2008), there is
still little evidence or sustained discussion of mixed methods research in the accounting
literature (see Modell, 2005, 2009, 2010, for an exception). This is particularly notable in
the management accounting context, given the wide acceptance that qualitative
methods already enjoy in this arena. Several calls in the literature acknowledge this
potential to complement positivist/functionalist paradigms with aspects of case-based
research (Ferreira and Merchant, 1992; Ittner and Larcker, 2001; Modell, 2005; Shields,
1997). Thus, in this paper, we review literature on mixed methods research originating
The authors are grateful to Deryl Northcott for helpful comments on this paper.
Qualitative Research in Accounting &
Management
Vol. 8 No. 1, 2011
pp. 5-21
q Emerald Group Publishing Limited
1176-6093
DOI 10.1108/11766091111124676
QRAM
8,1
6
from diverse social sciences and consider a range of definitions, opportunities and risks
associated with mixed methods research in accounting in general and management
accounting in particular.
“Mixed methods research” recently has gained popular acceptance as the term to define
research designs that combine qualitative and quantitative methods in a single study
(Johnson et al., 2007); but this method is variously referred to throughout the literature as
convergent methodology, multiple/multi-method/multitrait research, convergent
validation, between or across method triangulation, multiple operationalism, blended
research, integrative research, and mixed research (Denzin, 1978; Jick, 1979; Johnson et al.,
2007). An extensive literature considers the nature of mixed methods research, how the use
of mixed methods within a single study can both extend and strengthen potential findings,
and the potential pitfalls of integrating methods (Bryman, 2007; Johnson et al., 2007;
Modell, 2005; Teddlie and Tashakkori, 2009; Yin, 2006).
In this paper, we set the scene for this special issue by synthesising the vast array of
literature to examine what constitutes mixed methods research, and the associated
strengths and risks attributed to this approach. We illustrate the application of mixed
methods with three published management accounting studies (Davila and Foster, 2007;
Modell and Lee, 2001; Wouters and Wilderom, 2008) and one financial accounting study
(Graham et al., 2005). We examine these published studies for what they tell us about the
strengths of mixed method designs as well as the tensions and trade-offs in execution.
Similar themes of relative strengths, tensions and trade-offs are evident in the other
papers in this special issue that reflect on applications of mixed methods design (see the
papers by Malina, Nørreklit and Selto, Murphy and Maguire, and De Silva).
At the outset, we draw a “soft” distinction between mixed methods and mixed
methodologies. To the extent that mixed methods rely on the joint exploitation of
qualitative and quantitative methods, this can occur within either a positivist/functionalist
or interpretive paradigm. However, we refer to this as a soft distinction because the
definitions we draw on frequently refer to elements of mixing methods and methodologies.
Furthermore, many scholars argue that moving between quantitative and qualitative
methods by definition implies a methodological shift, whether acknowledged or not. While
recognising that the boundary is “fuzzy”, we initially eschew questions of mixing
methodologies to focus on issues associated with the more straightforward application of
mixed methods, largely from a positivist/functionalist perspective. We then return to the
question of mixing methodologies. In particular, we pay attention to ongoing debates
regarding the compatibility of quantitative and qualitative methodologies within a single
study and the perceived possibilities for successfully combining methods with such
distinct epistemological and ontological positions (see also De Loo and Lowe in this special
issue for elaborated discussion on the contention of mixing methodologies).
It is not our intention in the paper to prescribe how to perform the mixing of methods
(see instead Creswell and Plano-Clark, 2007, or Teddlie and Tashakkori, 2009). Nor do
we introduce or discuss strategies to assess the quality (reliability and validity) of mixed
methods studies (see instead Dellinger and Leech, 2007, and Kihn and Ihantola in this
issue).
Section 2 defines and introduces mixed methods research. We then turn to the
contentious question of mixing methodologies (Section 3). Sections 4 and 5 explore the
question of why researchers mix methods and the risks in doing so, respectively.
In Section 6, we draw on four published examples of mixed methods research
in the accounting literature. We then conclude the paper by drawing together the
literature on mixed methods and the “reality” observed in the examples discussed.
Ultimately, we consider the future potential for mixed methods research in accounting.
2. What is mixed methods research?
Mixed methods research is now widely accepted across diverse social science disciplines
as a separate research strategy with its own distinct worldview, vocabulary and
techniques (Denscombe, 2008; Hall and Howard, 2008; Johnson et al., 2007; Tashakkori
and Creswell, 2007b; Teddlie and Tashakkori, 2003). Despite this, as evidenced by the
responses of 19 leaders in the field solicited by Johnson et al. (2007), there is significant
variation in the definition of mixed methods research. However, the majority of the
definitions provided, and popular opinion in the discipline at large, seem to suggest that
mixed methods designs include both a quantitative and qualitative component. Where
inconsistencies and disagreements seem to originate is in the consideration of how these
quantitative and qualitative components are related, and whether these components
reflect quantitative and qualitative data collection and analysis techniques (i.e. methods)
and/or quantitative and qualitative approaches to research (i.e. methodologies)
(Denscombe, 2008; Tashakkori and Creswell, 2007b). Further points of contention relate
to the focus placed on the quantitative and qualitative components of the study
(the weighting decision), at what stages of the study quantitative and qualitative
components are mixed (the mixing decision) and in which order quantitative and
qualitative methods are used (the timing decision) (Creswell and Plano-Clark, 2007; Hall
and Howard, 2008; Jick, 1979). As Johnson et al. (2007) note, it is perhaps not surprising
that a fixed definition of mixed methods research remains elusive as definitions can and
usually will continue to evolve over time as a method grows. However, this looseness in
definition is not necessarily fatal (Creswell and Tashakkori, 2007) as “having the term
not cast in stone is intellectually useful and allows for reshaping understandings” (Guba,
1990, p. 17).
In the social sciences, the concept of methods “triangulation” dates to the work of
Campbell and Fiske (1959) who propose the use of more than one research method as part
of a validation strategy to ensure the explained variance is the result of the underlying
phenomenon and not an artefact of the research method adopted. Subsequent
researchers elaborate on the nature of methods triangulation, distinguishing
within-methods triangulation (the use of multiple quantitative or multiple qualitative
elements) from between-methods (the use of both quantitative and qualitative
elements)[1] and delineating method triangulation from data, investigator and theory
triangulation[2] (Webb et al., 1966). Studies have been considered mixed on the basis of:
addressing two types of research questions; the manner in which research questions are
developed; adopting two types of sampling procedures, data-collection techniques, types
of data or data analysis; and presenting two types of conclusions (Tashakkori and
Creswell, 2007b).
In this section, we analyse the concept of what is seen to constitute mixed methods
research from the platform of a broad definition provided by Tashakkori and Creswell
(2007b, p. 4) wherein mixed methods research is considered to be:
[. . .] research in which the investigator collects and analyses data, integrates the findings, and
draws inferences using both qualitative and quantitative approaches or methods in a single
study or program of inquiry.
Mixed methods
research in
accounting
7
QRAM
8,1
We select this definition as it makes prominent what we consider to be two essential
aspects of a study that combines quantitative and qualitative elements:
(1) the notion of “integration” between quantitative and qualitative elements[3];
and
(2) the importance of developing a “single study or program of inquiry”.
8
Further, rather than considering mixed methods in a narrow or “pure” sense, this
definition is inclusive in allowing for the possibility of mixing encompassing both
method and methodology, and permitting variation in the weighting, mixing and timing
decisions of researchers (Johnson et al., 2007).
Integration of methods
The attribute of integration between qualitative and quantitative elements of a mixed
method study is evident primarily in the way interdependencies between the multiple
strands of the study are embedded in the research design, and managed in the analysis.
Complementary or sequenced quantitative and qualitative components of a study that
do not involve reciprocal interdependencies between these research strands may not be
considered to be mixed methods (Bazeley, 2009; Bryman, 2007). Integration “might occur
through iteration, blending, nesting or embedding” (Bazeley, 2009, p. 204). Examples of
these forms of integration across the research process provided by Bazeley (2009, p. 205)
include: using the results of one analysis employed to approach the analysis of another
form of data; the synthesis of data from a variety of sources for joint interpretation; the
comparison of coded qualitative data across groups defined by categorical or scaled
variables; the conversion of qualitative to quantitative coding to allow for descriptive,
inferential or exploratory statistical analyses; the creation of “blended” variables to
facilitate further analysis; flexible, iterative analyses involving multiple, sequenced
phases where the conduct of each phase arises out of or draws on the analysis of the
preceding phase.
Yin (2006) notes that the integration of quantitative and qualitative strands of mixed
methods studies can occur in many ways. The articulation of research questions, the
identification of samples and units of analysis, the data collection methods used and the
analytic strategies employed are all implicated in the integrative quality of mixed
method design. Yin (2006) proposes that the more the quantitative and qualitative
elements are integrated into research procedures, the stronger the “mix” of methods that
results. Creswell and Tashakkori (2007), further note that “strong mixed methods”
studies integrate the quantitative and qualitative results of the study into coherent
conclusions or inferences. Bazeley (2009) considers the integration of quantitative and
qualitative results the minimum requirement for a study to qualify as mixed methods in
design, and observes that blending data and meshing analyses is far less frequent in
practice. A critical foundation for integration may be the development of an overarching
mixed methods research question (Mertens, 2007; Tashakkori and Creswell, 2007a).
Mixed methods studies that struggle to integrate findings are usually those that develop
either qualitative or quantitative research questions and then use mixed methods solely
for data collection. The inference drawn from the study by Mertens (2007) is that
developing an overarching mixed methods question is a design necessity in mixed
methods studies if mixed methods researchers are to present integrated and coherent
research results.
Single program of inquiry
Importantly, mixed methods designs must also ensure that the integrity of the single
study focus is maintained and the study does not inadvertently devolve into two or more
parallel studies (Yin, 2006). While there is potential to integrate qualitative and
quantitative findings either within or across studies, we restrict our definition of mixed
method studies to those that integrate qualitative and quantitative findings within a
single study. We do this for several reasons. First, the importance of a mixed methods
research question and research design as an antecedent to effective integration of
findings is premised on the notion that integration is occurring in a single study
(Mertens, 2007; Tashakkori and Creswell, 2007a). Second, authors are really only able to
assess the extent to which two sets of data converge, contradict or extend one another
with a clear understanding of the construct definitions and the domain of observables
that govern the collection of both datasets. This is generally only evident in a single
study. It is possible that multiple sequential studies by the same author may go some
way to achieving the required consistency in definitions to integrate data across studies.
Malina et al. (in this issue) present such a possibility. It is notable, however, that
while their reflections refer to sequential studies, they rely on repeat analysis of a single
dataset to produce the primary mixed methods contribution that they describe. Multiple
studies by different authors are much less likely to satisfy the need for consistency in
definitions and domain of observables. In such cases, the integration of findings relies on
meta-analyses that address the added complications of variability caused by time, study
design, sampling and definitional differences (Johnson and Onwuegbuzie, 2004; Yin,
2006).
In summary, mixed methods research designs are characterized by the use of both
qualitative and quantitative methods within a single study, with a focus on the
integration of these multiple strands in both study design and data analysis. While such
a definition emphasises the mixing of methods, it implicitly embraces the potential to
mix methodologies, as does the literature from which we draw our definition. However,
the question of mixing methodologies emerges as somewhat more contentious than the
mixing of methods. In the next section, we consider competing views on the viability or
authenticity of mixed methodologies.
3. Mixing methodologies
A comprehensive review of the mixed methods literature shows that there are two
dominant views about the mixing of methodologies: the incompatibility thesis and the
pragmatists’ view. Researchers who articulate an incompatibility thesis about mixing
methodologies argue that qualitative and quantitative methodologies draw from
different epistemological assumptions and have different research cultures that work
against the convergence of research methodologies (Brannen, 2005; Sale et al., 2002; Scott
and Briggs, 2009). This argument is premised on the idea that qualitative research
methodology is based on inductive logic of enquiry, which is considered to be
diametrically opposed to the hypothetic-deductive logic that underpins quantitative
methodology. Accordingly, these two methodologies speak to different ways of knowing
reality and thus have different implications for deriving research questions and require
different research processes, suggesting that any attempt to mix them will create tension
and lead to difficulties in interpreting results. For instance, the hypothetic-deductive
logic requires more structured protocols, representativeness of a sample
Mixed methods
research in
accounting
9
QRAM
8,1
10
and, by implication, the generalisability of results requires a much larger sample. In
contrast, inductive logic does not require large, representative samples. For Sale et al.
(2002), these two different methodological approaches are incommensurate and should
therefore not be mixed in any mixed methods studies.
Some researchers extend the incompatibility thesis to argue that mixing
methodologies can create argumentative incoherence by attempting to “blend
paradigms with incommensurable epistemic and ontological foundations” (Scott and
Briggs, 2009, p. 230; Johnson and Onwuegbuzie, 2004; Sale et al., 2002). They contend
that qualitative and quantitative methodologies draw on incompatible paradigmatic
assumptions that work against any attempt at mixing the two methodologies.
Qualitative methodologies are said to derive from interpretivism and constructivism
where it is generally assumed that reality has no existence prior to the activity of
investigation and where the focus is on shared meanings rather than causal relations
(Sale et al., 2002). In contrast, quantitative methodology is based on positivism and its
assumption of an objective reality that can be studied without researcher influence
(Sale et al., 2002). Given these different paradigmatic assumptions, Sale et al. (2002) argue
that qualitative and quantitative methodologies do not study the same phenomenon and
should not therefore be mixed in any way. They argue further that even when the same
phenomenon is explored through these two methodologies, the definitions of these
phenomena will differ and the approach to knowing the phenomena will also differ, thus
making it impractical to mix the two methodologies.
The conclusion drawn from the incompatibility thesis is that the mixing of
methodologies in mixed methods research is not a reasonable proposition and should
therefore be discouraged. Incompatibility theorists state, however, that the fact that the
two methodologies are incommensurate does not mean that researchers should abandon
mixed methods research. Mixed methods can be used in a single study if it is done for
complementary purposes (Sale et al., 2002).
In sharp contrast to the incompatibility thesis, pragmatists share the view that
mixing methodologies is a sensible thing to do in mixed methods research (Brannen,
2005). In building an argument for mixing methodologies, pragmatists criticise the
incompatibility thesis for emphasising differences between qualitative and quantitative
methodologies and ignoring opportunities for convergence (Brannen, 2005; Bryman,
2007; Hammersley, 1992). The pragmatists contend that it will be difficult to reduce
mixed methods research to just methods since one cannot separate methods from the
larger research process (Creswell and Tashakkori, 2007). Research methods and design
are shaped largely by the research questions under investigation and the research
questions derive from the research purpose (Tashakkori and Creswell, 2007a). Given
that research purposes are based on epistemological and methodological assumptions, it
stands to reason that a meaningful mixed methods study should draw from both
qualitative and quantitative research methodologies (Brannen, 2005; Tashakkori and
Creswell, 2007a).
In stressing the relevance of mixing methodologies in mixed methods research,
Brannen (2005) states that in analysing mixed methods data, researchers reflect on the
different kinds of truth claims underpinning the data and also take into account the fact
that the different types of data being analysed are constituted by the different
methodological assumptions and methods that elicit them. This implies that it will be a
senseless exercise to attempt to produce a unitary account from data collected from
different methods without any attempt to combine the methodological assumptions that
underlie the data collected. Given that one cannot separate methods from the broader
research process, the pragmatists argue that mixed methods research should focus on
the entire research process by tying methods to an integrated set of philosophical and
methodological assumptions (Creswell and Tashakkori, 2007; Tashakkori and Teddlie,
1998). This, according to Scott and Briggs (2009), is a sensible approach to unifying data
and presenting an integrated and coherent set of results.
In an empirical study of mixed methods researchers, Bryman (2007) found that
“pragmatic” researchers seem not to dwell on epistemological and ontological positions
but rather focus on ways of combining qualitative and quantitative methodologies in the
overall research process. Scott and Briggs (2009, p. 231) also argue that in practice,
mixed method researchers are guided by the rationale for their research, rather than by
epistemology and conclude “methodology is in practice commonly agnostic to
epistemology”. The implication here is the rejection of the incompatibility thesis in
practice as researchers seek convergence in methodology and allow such convergence to
guide the research process.
In the prior section, integration of research findings was described as fundamental to
the execution of mixed methods. Bryman (2007) contends that the lack of integration in
mixed methodology studies is not so much a consequence of a clash in epistemological and
ontological positions but rather is due to the practical difficulty of tying the two
methodologies together. This practical difficulty is partly because of concerns about
establishing the validity of mixed methods research. Together, these validation
frameworks should facilitate the integration of qualitative and quantitative methodologies
and theories in ways that bypass epistemological and ontological positions.
For the remainder of this paper, we take a pragmatic view in which we treat mixed
methods as potentially but not necessarily including mixed methodology studies. In the
next section, we examine a range of rationales for mixing methods and/or
methodologies.
4. Why mix methods?
The rationale for pursuing mixed methods research designs rests largely on the premise
that the weaknesses in each individual method will be compensated by the
counter-balancing strengths of the other (Jick, 1979). “Methods should be mixed in a
way that has complementary strengths and nonoverlapping weaknesses” (Johnson and
Turner, 2003, p. 299). The advantages to mixed methods research rest on the
development of a research strategy that is effective in exploiting the advantages of
quantitative and qualitative methods, while neutralising the “costs” or “risks”
associated with each method ( Jick, 1979; Modell, 2005). While the literature identifies a
wealth of advantages potentially applicable to mixed methods designs, the benefits of
complementary research strategies within a single study can generally be categorised as
allowing researchers to:
.
extend findings beyond those observable using a single method;
.
identify empirical contradictions that might otherwise be missed (Denzin, 1978);
and
.
observe convergence in findings from different strands of the research, thereby
building confidence in the research (Denzin, 1978).
Mixed methods
research in
accounting
11
QRAM
8,1
12
Central amongst the rationales advanced in support of the use of mixed methods
research is the enhanced credibility and validity of research findings and the reduced
potential that research results reflect a unique method artefact (Denzin, 1978; Jick, 1979;
Modell, 2005). Research enhancement through mixed methods is constructed in a range
of ways. The notion of extension implies that mixed methods contribute “more” than
can be achieved by the application of individual methods, potentially by addressing
different aspects of a research question. Just as methods are tailored to research
questions, different parts of research questions within a study may require the
application of different methods (Yin, 2006). Examples would be studies that address
both “what” and “why” questions or “process” and “outcome” questions. In this issue,
the Murphy and Maguire study illustrates the application of complementary methods
to assess “outcome” and “stakeholder perceptions” as separate dimensions of an
overarching research question. In these cases, appropriate methods are combined to
enable exploration of salient complementary questions. The aim is not triangulation or
the use of mixed methods to cross-validate results. In such cases, the research findings
are the result of the combined application of the complementary methods required to
answer the different elements of the question.
The notions of convergence and contradiction relate more to the application of
complementary research strategies to the same research question. Yin (2006) refers to
counterpart analysis in which a combination of methods can be used to assess convergence
in both the measurement of constructs and the relationships between constructs. Mixed
methods research provides better and stronger inferences through corroboration (Modell,
2005). Two of the functions of mixed methods research described by Greene (2008) are
concerned with the strengthening of inferences: triangulation and complementarity. The
data are richer and thicker (Jick, 1979). Assessing convergence and strengthening
inferences can of course, also expose contradictions. The quantitative and qualitative
elements of mixed methods studies provide an opportunity for researchers to incorporate
divergent views that can result in a deeper understanding of the research problem.
Divergent findings are valuable in that they can promote the reexamination of conceptual
frameworks and the assumptions underlying the strands of the research (Denzin, 1978;
Modell, 2005) and lead to novel future lines of enquiry (Jick, 1979). In the extant
management accounting literature mixed method designs that focus on establishing
convergence and/or contradiction between findings commonly incorporate surveys and
interviews or focus groups addressing a similar set of questions to determine whether
consistent themes emerge in both quantitative and qualitative data (Modell, 2005).
While most writers in this area deliberately avoid granting any privilege to specific
methods in a mixed method setting, arguing that it is the multiplicity of methods in itself
that generates value, Jick (1979) argues persuasively that the real contribution of mixed
method designs to triangulation comes from the contribution of qualitative data.
He suggests that it is the closeness of the qualitative researcher to the research context
which illuminates the research problems. “Qualitative data are used as the critical
counterpoint to quantitative methods” ( Jick, 1979, p. 609). This view attaches superior
insight to qualitative data.
5. Risks in mixing methods
One of the primary risks in mixing methods is the failure to adequately integrate the
design, execution, analysis and interpretation of the quantitative and qualitative strands
of research. Integration is a distinguishing feature of mixed method studies and a critical
quality in defining the contribution from such studies (Yin, 2006). The need for
integration applies regardless of the purpose of mixing methods – convergence,
contradiction or extension. The full contribution of studies in any of these domains really
depends on the effort researchers devote to the challenging task of integration of
methods. Lack of integration pushes the problem back onto readers, leaving them in a
similar position to that gained from sequential studies by different authors using
different methods.
Teddlie and Tashakkori (2009, p. 286) contend that most importantly, researchers
must devote attention to ensuring the integration of quantitative and qualitative
research findings in drawing inferences in mixed methods studies. While numerous
textbooks provide copious advice as to “how” to integrate the design, execution and
analysis phases of mixed methods studies (Creswell and Plano-Clark, 2007; Teddlie and
Tashakkori, 2009), much less guidance is available regarding the integration of research
findings. Indeed, Bryman (2007) questions whether we have yet actually determined
what it means to integrate findings in mixed method research. This may be particularly
problematic in accounting where there are few examples in the literature that model the
integration of results in mixed methods studies. Effective integration of results has been
identified as less common than it should be (Greene, 2008) and even rare (Niglas, 2004).
Many authors highlight use of a mixed methods design but give much greater attention
to one method rather than the other, or present their findings in parallel, such that there is
little if any integration (Bryman, 2007).
Why is integration of results so problematic? The potential to integrate findings may
be partially dependent on starting with an integrated conceptual study design rather
than a parallel design. Where a study adopts mixed methods to explore complementary
questions using different methods, are the complementary questions (e.g. “what” and
“why”) linked at a conceptual level? Where a study adopts mixed methods to explore
convergence and contradictions, is the potential contribution from convergent and
contradictory insights clearly identified and motivated at the study design level?
Conceptual integration seems important in establishing a coherent foundation from
which to exploit multiple perspectives.
Qualitative and quantitative data do not blend easily. There are always caveats
relating to differences in the domain of observables, interpretations of constructs and
whether indeed the data speak to a common point. There are potential difficulties in
establishing the validity of mixed methods research (Bryman, 2007). Some researchers,
however, have provided validity criteria and validation frameworks that should
overcome this problem. Dellinger and Leech (2007), for example, have integrated validity
criteria from qualitative and quantitative methodologies to develop a validation
framework that seeks to unify thinking about validity in mixed methods research.
Modell (2005) also offers validity criteria for assessing external, internal and construct
validity in mixed methods studies. Later in this special issue, Kihn and Ihantola examine
the contributions of two other validation frameworks for mixed methods research
proposed by Teddlie and Tashakkori (2003) and Onwuegbuzie and Johnson (2006).
Nonetheless, paradigm debates as to whether, methodologically, it is possible to
integrate quantitative and qualitative research may continue to make researchers
nervous about integrating the various strands of their research (Bazeley, 2009).
Mixed methods
research in
accounting
13
QRAM
8,1
14
It is also easier to integrate convergent or complementary findings than it is to integrate
conflicting findings. Contradictions leave researchers with a dilemma. The orientations
and preferences of researchers may systematically privilege either qualitative or
quantitative data, thus leading researchers to focus on competition between methods to
resolve contradictions rather than exploiting the contradictions themselves as a product of
integration.
Beyond the key issue of integration of quantitative and qualitative research findings,
there are a number of other potential risks associated with mixed methods. Given
scarcity of research time and funding, all research carries an opportunity cost. Splitting
research effort across two methods in the one study may result in serious compromises
in the application of either one or both methods, potentially leading to perceptions of two
things done badly rather than one thing done well. A particular risk in thinly spread
effort is that of superficial fieldwork, or small-scale surveys, compromising the quality
of qualitative and quantitative data, respectively.
6. Published examples of mixed method research in the accounting
literature
In this section, we provide examples of the way mixed methods are used in accounting
research. We did not conduct an exhaustive search of mixed methods research in
accounting. First, the variety of potential terms used within studies to describe methods
precludes a reliable and exhaustive search. Such a search could involve each
combination of individual methods (i.e. each quantitative method – experimental,
archival and survey – in combination with terms such as “qualitative”, “field study” and
“case study”) in conjunction with “accounting” or its sub-disciplines. Such searches
produce tens of thousands of entries. At a minimum, they include all studies that use
both quantitative and qualitative data; such as extended responses within surveys, and
manipulation check questionnaires in experiments. In contrast, searches of “accounting”
along with “mixed methods” or “multiple methods” produces virtually no result. We
deemed an exhaustive search impossible, and decided instead to provide examples from
the literature of the way mixed methods tend to be used in accounting. The vast majority
of examples of mixed methods appear to be the combination of surveys and interviews
(Modell, 2005).
We include here three examples of the application of mixed methods, and one example
of the application of mixed methodologies. These specific examples are not selected as
the most notable or highest quality examples of mixed methods/methodologies in the
literature. We do not set out to make such a judgment. Rather, the examples selected
exemplify both the value of mixing methods and also some of the challenges in doing so.
Davila and Foster (2007)
This paper examines the evolution of management control systems in early stage
startup companies. The authors draw on publicly available data as well as a survey
and semi-structured interviews. Publicly available data are used to triangulate or
validate survey responses where possible (financial and funding information).
Survey data are used to capture the dynamic evolution of management control
systems. Semi-structured interviews are used to clarify and triangulate survey
responses as well as to provide a richer description of context in which to understand
why systems were adopted. Thus, the mixed data sources can be described as used
for both triangulation of common elements (convergence) as well as discovery of
complementary elements (extension).
In reality, in this study, there is little evidence of the interview data in the paper. There
are a few illustrative quotations relegated to a half-page in an appendix. These quotes
are used to support the choice of variables used in the regression equation examining the
rate of adoption of management control systems. The interview data thus take a
secondary role to the survey data and there is little evidence of effective integration of
quantitative and qualitative findings. There is no evidence of the systematic use of
interview data to complement, clarify or triangulate the survey data. The publicly
available data also play a minor role in validating specific elements of the survey data.
Thus, the study interprets survey data with minimal input from other data sources.
While the study is introduced as a “multi-method, multi-case field research design” there
is little evidence of this design in the way the study is reported. Little would be lost to the
paper without the qualitative data. Nonetheless, we acknowledge that the paper as
published reflects the resolution of a range of tensions relating to content and focus, and
our conclusions may well understate the value of the qualitative data to the researchers
in framing ideas, lines of inquiry, potential mental models of relationships in the data
and perhaps fueling a broader research program.
This study also potentially exemplifies a risk in mixed methods research, which we
have not highlighted above – the risk that certain data and findings will become
privileged through journal editorial and review preferences or due to restrictions on the
length of published papers. This is, however, conjecture as we have no particular
insights into the editorial or review process for this paper.
Wouters and Wilderom (2008)
This study examines perceptions of the enabling quality of performance measurement
systems, and tries to isolate the causes of performance measurement systems being
perceived as enabling. In contrast to the Davila and Foster (2007) study described above,
the Wouters and Wilderom (2008) study represents a fully integrated mixed methods
design with constant cross-referencing from qualitative interview to quantitative survey
data. This study represents well the opportunities to use mixed methods to approach
different elements of a research question. Thus, the multiple data sources are used more
to complement one another than to triangulate or validate.
Wouters and Wilderom (2008) operationalise their dependent variable – the
perceived enabling quality of performance measurement systems – through several
items on a panel survey executed within their case study organisation. Of their three
independent variables, they seek to operationalise one through the panel survey
(professionalism or orientation to learning) and the others (experimentation and
experience-based design) through their interviews. While not framed by the authors this
way, they in effect use the survey to capture personal perspectives and the interviews
(along with document analysis and other qualitative data sources) to capture the
performance measurement system development process. While it might seem that this
focus could be reversed, such that the process data are collected by survey and the
perception data collected through interviews, the design is in fact robust and well suited
to the research questions. In this case, it is important to obtain a cross-section of “quality”
perceptions (rendering the survey approach useful) and to obtain a reliable description of
a singular process (rendering interviews and document analysis useful).
Mixed methods
research in
accounting
15
QRAM
8,1
16
Interestingly, this approach also offered the side benefit of separating the qualitative
evaluation of the system (the dependent variable) from the identification of the
determinants of quality (two of the three independent variables) thus minimising
“common response bias”. The authors do not highlight this advantage themselves, but
their study does highlight the opportunity to use mixed methods to reduce such bias
through the use of complementary data sources.
Graham et al. (2005)
This is a seminal paper in the financial accounting literature that uses a survey along
with interviews with chief financial officers to identify factors that drive reported
earnings and disclosure decisions. The study is designed to articulate clearly with the
wealth of archival and analytical studies in the financial accounting literature
addressing these questions. While the study does not extend to actually drawing on
archival data as well as the survey and interview data, it does integrate the findings so
clearly with extant research using archival data and analytical methods the study can
almost be viewed as a triangulation of survey, interview and archival data sources.
The rationale for the survey and interview design in this study is to gain both breadth
and depth of insights into the decisions that lie behind the archival evidence of earnings
management and disclosure choices. The authors seek to gain better insight into
the complex interplay among a range of incentives chief executive officers face in both the
discretionary application of generally accepted accounting principles to determine the
levels of earnings to report, and also in more general disclosure choices. The authors also
seek to build more robust theory relating to the causal relationships between context and
the earnings and disclosure-related decisions studied. The survey and interview data are
clearly used to triangulate or validate findings (convergence) with extensive integration
of the two data sources throughout the paper. The question of integration with archival
data remains somewhat more contentious as these data are not collected within the single
study and are thus not part of the mixed methods design. However, financial accountants
have the advantage of common archival data sources adopting constant definitions,
which are to a large extent independent of the researchers themselves. Thus, it is easier
for the survey and interview data to speak to these common “stable” data sources.
Most notable about this paper as a mixed methods contribution is that it is in the
financial accounting sub-discipline. There is little qualitative work in financial
accounting, so it is potentially lagging management accounting in both the development
and acceptance of mixed methods research strategies. Graham et al. (2005) do, however,
demonstrate very well how much is to be gained by supplementing traditional archival
data sources with forays into both surveys and qualitative data collection in financial
accounting.
Modell and Lee (2001)
Modell and Lee (2001) is referred to by Modell (2010, p. 124) as a mixed methodology
paper:
Whilst our initial hypothesis and the design of the survey instrument were primarily informed
by functionalist approaches (e.g. agency theory), we retained some openness to alternative
theoretical perspectives and combined the survey with semi-structured interviews.
Modell and Lee (2001) examine the link between decentralization and the application of
the controllability principle in a public sector setting. In developing the theoretical
framework for their study, Modell and Lee (2001) draw on both functionalist literature
relating to the application of the controllability principle (Bushman et al., 1995;
Merchant, 1989), as well as neo-institutional sociology to capture the competing forces
in defining the link between decentralization and controllability in practice. On the one
hand, functionalist arguments would support a positive association. On the other hand,
the expected positive association is expected to be altered or “breached” by the
presence of institutional pressures which either reduce local controllability, or lead to
the need to loosely couple decentralization and controllability. Thus, rather than
appearing to be initially informed by a functionalist perspective (Modell, 2010), the
paper is framed to capture the potential tension between functionalist and interpretive
factors influencing the relationships under study.
Given the ex ante framing of this tension, the functionalist hypothesis in Modell and
Lee (2001) appears to be something of a “straw man”. It is set up only to determine
whether there is a statistically observable relationship between decentralization and the
treatment of non-controllable factors through the budgetary control process. The authors
find only a weak positive statistical relationship. They then establish through the
analysis of qualitative data the more subtle, nuanced relationship between controllability
and decentralization influenced heavily by institutional and political context.
In appearing to create a “straw man” functionalist hypothesis, this paper
demonstrates the challenges of mixing methodologies. The authors cannot suspend
their critical and interpretive understanding of context so it is embedded in their
theorisation of the problem. Knowing that context, we, as readers, are unable to suspend
disbelief enough to subscribe to the functional hypothesis in the first place. Equally, if
the initial framing of this study was completely functionalist, could the authors credibly
argue for a framing of unhypothesised findings as being consistent with a previously
untheorised neo-institutional sociological explanation? As readers, we would most likely
want to see the acknowledgement of that possibility in ex ante theorisation of the
problem. Thus, this paper demonstrates well the challenges in mixing methodologies.
However, if we eschew the methodology question, we can reconceptualise the design of
this paper as a strong illustration of mixed methods. Survey and interview data are
integrated in analysis to provide alternative and complementary perspectives on a
single question within a single study.
7. Conclusions
Our modest objective in this paper was to canvass the literature on mixed methods
research with a view to providing some broader context to the papers appearing in this
special issue. As such, we spent considerable time discussing the nuances in the
definition of mixed methods. Consistent with the preponderance of available literature,
we adopt a broad perspective of mixed methods researching noting two distinguishing
characteristics of this type of research; namely (1) the importance of carefully integrating
the quantitative and qualitative elements of the mixed strategy within (2) a single
program of study. Our definition also recognises the potential for the mixing of
methodologies in this research strategy. The working definitions of mixed methods
adopted by authors throughout this special issue are consistent with ours. In particular,
the contribution of De Loo and Lowe in this issue elaborates our discussion of mixing
methodologies and considers extensively the possibilities and practicalities of
accomplishing this.
Mixed methods
research in
accounting
17
QRAM
8,1
18
While various benefits to the adoption of mixed methods research are proposed the
achievement of these ends is reliant on the careful and considered design, execution and
communication of studies to navigate a range of potential pitfalls. We present the key
rationales and risks in mixing research strands. These themes are elaborated in the three
papers in this issue that reflect on experiences with mixed methods strategies
(Malina et al., Murphy and Maguire, and De Silva).
Our review of four studies published recently in the accounting literature allows us to
illustrate the themes presented in this paper. In our literature search, we noted a paucity
of exemplars of mixed methods studies in the accounting domain. Furthermore, the
examples that do exist are largely limited to the combination of survey and interview
data. While a significant number of studies report adopting a mixed or multiple methods
approach, on closer examination these studies neither evidence the integration of
methods, nor maintain the integrity of a single program of inquiry, which are the
hallmarks of mixed methods research. Instead, these studies reflect what Jick (1979,
p. 603) refers to as “primitive forms“ of mixed methods research, that are typically
“parenthetical, even somewhat patronizing” in their use of qualitative data to support
statistical results.
We see significant potential for the accounting literature to adopt mixed methods
research strategies. Given the acknowledged strengths of mixed methods designs to
enhance both theory testing and theory building through extension, convergence and
contradiction of findings, the lack of use of such methods suggests missed opportunities.
There are potentially several reasons for this under-exploitation of mixed methods. Both
our discussion in Section 5 above and the published studies, we review point to several of
the risks researchers face in undertaking mixed methods studies. In addition to these
risks, there remains further uncertainty regarding journal editor preferences and the
review and publication process. We conjecture that this may have been a factor in
the Davila and Foster (2007) paper. Malina et al. (in this special issue) also point to the
difficulty of meeting the demands of reviewers with divergent quantitative and
qualitative preferences. These effects are likely to be exacerbated where researchers
attempt to mix methodologies. When coupled with the added time and cost of research
design and execution, this renders a mixed method approach a particularly risky research
strategy (Jick, 1979). Potentially, these factors lead to researchers seeking to publish the
quantitative and qualitative strands of their research study independently. Further
inhibiting the broader diffusion of mixed methods research designs is the current lack of
accepted nomenclature and the absence of an established framework for assessing the
quality of such research. The absence of general agreement as to what constitutes mixed
methods research, the scarcity of examples in the literature, and the unresolved question
as to whether the issue is one of mixing methodologies or methods, create a situation of
uncertainty for researchers. Kihn and Ihantola (in this special issue) take up the issue of
reviewing and integrating a range of standards proposed to assess the validity and
reliability of mixed methods. Such analysis is an important first step in the acceptance of
a comprehensive framework by which to judge the rigour of mixed methods research.
Protocols to establish the quality of mixed methods strategies both guide researchers in
the design and execution of studies and promote the confidence of readers.
In spite of these challenges, we hope that this special issue and the examples that it
provides of mixed methods potential help to stimulate interest in the more extensive
adoption of mixed methods in the accounting literature.
Notes
1. Jick (1979) provides examples of within-(quantitative) methods triangulation that include the
use of multiple scales of the same construct on a survey and within-(qualitative) methods
triangulation that include multiple comparison groups in a case study. Within-methods
triangulation is seen to have a less significant effect on promoting convergent and divergent
validity as the inherent weaknesses in the underlying research paradigm prevail (Denzin,
1978). Within-methods triangulation is not widely considered to reflect a mixed methods
research design and does not meet the working definition of mixed methods research
adopted in this paper.
2. Malinaata triangulation involves the use of multiple data sources, investigator triangulation
the use of multiple rather than single observers and theoretical triangulation the use of
multiple theoretical perspectives to interpret the phenomenon (Denzin, 1978; Jack and Raturi,
2006; Modell, 2005). These forms of triangulation are also not considered to reflect mixed
methods research as defined in this paper.
3. Although consistent with Yin (2006), we argue that such integration can be present to
varying degrees at different stages of the research process, as influenced by the “mixing
decision”.
References
Bazeley, P. (2009), “Editorial: integrating data analyses in mixed methods research”, Journal of
Mixed Methods Research, Vol. 3 No. 3, pp. 203-7.
Brannen, J. (2005), “Mixed methods: the entry of qualitative and quantitative approaches into the
research process”, International Journal of Social Research Methodology, Vol. 8 No. 3,
pp. 176-84.
Bryman, A. (2007), “Barriers to integrating quantitative and qualitative research”, Journal of
Mixed Methods Research, Vol. 1 No. 1, pp. 8-22.
Bushman, R.M., Indjejikan, R.J. and Smith, A. (1995), “Aggregate performance measures in
business unit manager compensation: the role of intrafirm interdependencies”, Journal of
Accounting Research, Vol. 35, pp. 101-28 (supplement).
Campbell, D.T. and Fiske, D.W. (1959), “Convergent and discriminant validation by the
multitrait-multimethod matrix”, Psychological Bulletin, Vol. 56, pp. 81-105.
Creswell, J.W. (2009), “Editorial: mapping the field of mixed methods research”, Journal of Mixed
Methods Research, Vol. 3 No. 2, pp. 95-108.
Creswell, J.W. and Plano-Clark, V.L. (2007), Designing and Conducting Mixed Methods Research,
Sage, Thousand Oaks, CA.
Creswell, J.W. and Tashakkori, A. (2007), “Editorial: differing perspectives on mixed methods
research”, Journal of Mixed Methods Research, Vol. 1 No. 4, pp. 303-8.
Davila, A. and Foster, G. (2007), “Management control systems in early-stage startup
companies”, The Accounting Review, Vol. 82 No. 4, pp. 907-37.
Dellinger, A.B. and Leech, N.L. (2007), “Toward a unified validation framework in mixed
methods research”, Journal of Mixed Methods Research, Vol. 1 No. 4, pp. 309-32.
Denscombe, M. (2008), “Communities of practice: a research paradigm for the mixed methods
approach”, Journal of Mixed Methods Research, Vol. 2 No. 3, pp. 270-83.
Denzin, N.K. (1978), The Research Act, McGraw-Hill, New York, NY.
Ferreira, L.D. and Merchant, K.A. (1992), “Field research in management accounting and control:
a review and evaluation”, Accounting, Auditing & Accountability Journal, Vol. 5 No. 4,
pp. 3-34.
Mixed methods
research in
accounting
19
QRAM
8,1
20
Graham, J.R., Harvey, C.R. and Rajgopal, S. (2005), “The economic implications of corporate
financial reporting”, The Journal of Accounting and Economics, Vol. 40 Nos 1-3, pp. 3-73.
Greene, J.C. (2008), “Is mixed methods social inquiry a distinctive methodology”, Journal of
Mixed Methods Research, Vol. 2 No. 1, pp. 7-22.
Guba, E.G. (1990), “The alternative paradigm dialog”, in Guba, E.G. (Ed.), The Paradigm Dialog,
Sage, Newbury Park, CA, Cited in: Creswell, J.W. and Tashakkori, A. (2007), “Editorial:
differing perspectives on mixed methods research”, Journal of Mixed Methods Research,
Vol. 1 No. 4, pp. 303-08.
Hall, B. and Howard, K. (2008), “A synergistic approach: conducting mixed methods research
with typological and systemic design considerations”, Journal of Mixed Methods Research,
Vol. 2 No. 3, pp. 248-69.
Hammersley, M. (1992), “The paradigm wars: reports from the front”, British Journal of Sociology
of Education, Vol. 13 No. 1, pp. 131-43.
Ittner, C.D. and Larcker, D.F. (2001), “Assessing empirical research in managerial accounting:
a value-based management perspective”, Journal of Accounting and Economics, Vol. 32
Nos 1/3, pp. 349-410.
Jack, E.P. and Raturi, A.S. (2006), “Lessons learned from methodological triangulation in
management research”, Management Research News, Vol. 29 No. 6, pp. 345-57.
Jick, T.D. (1979), “Mixing qualitative and quantitative methods: triangulation in action”,
Administrative Science Quarterly, Vol. 24 No. 4, pp. 602-11.
Johnson, R.B. and Onwuegbuzie, A.J. (2004), “Mixed methods research: a research paradigm
whose time has come”, Educational Researcher, Vol. 33 No. 7, pp. 14-26.
Johnson, R.B. and Turner, L. (2003), “Data collection strategies in mixed methods research”,
in Tashakkori, A. and Teddlie, C. (Eds), Handbook of Mixed Methods in Social and
Behavioural Research, Sage, Thousand Oaks, Cited in: Teddlie, C. and Tashakkori, A.
(2009), Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative
Approaches in the Social and Behavioural Sciences, Sage, Thousand Oaks, CA.
Johnson, R.B., Onwuegbuzie, A.J. and Turner, L.A. (2007), “Toward a definition of mixed
methods research”, Journal of Mixed Methods Research, Vol. 1 No. 2, pp. 112-33.
Merchant, K.A. (1989), Rewarding Results: Motivating Profit Centre Managers,
Harvard University Press, Cambridge, MA.
Mertens, D.M. (2007), “Transformative paradigm: mixed methods and social justice”, Journal of
Mixed Methods Research, Vol. 1 No. 3, pp. 212-25.
Modell, S. (2005), “Triangulation between case study and survey methods in management
accounting research: an assessment of validity implications”, Management Accounting
Research, Vol. 16 No. 2, pp. 231-54.
Modell, S. (2009), “In defence of triangulation: a critical realist approach to mixed methods
research in management accounting”, Management Accounting Research, Vol. 20 No. 3,
pp. 208-21.
Modell, S. (2010), “Bridging the paradigm divide in management accounting reserach: the role of
mixed methods approaches”, Management Accounting Research, Vol. 21 No. 2, pp. 124-9.
Modell, S. and Lee, A. (2001), “Decentralization and reliance on the controllability principle in the
public sector”, Financial Accountability & Management, Vol. 17 No. 3, pp. 191-219.
Niglas, K. (2004), “The combined use of qualitative and quantitative methods in educational
research”, dissertation on Social Sciences, Tallin Pedagogical University, Tallin, Cited in:
Bryman, A. (2007), “Barriers to integrating quantitative and qualitative research”,
Journal of Mixed Methods Research, Vol. No. 1, pp. 8-22.
Onwuegbuzie, A.J. and Johnson, R.B. (2006), “The validity issue in mixed research”, Research in
the Schools, Vol. 13 No. 1, pp. 48-63.
Sale, J., Lohfeld, L. and Brazil, K. (2002), “Revisiting the qualitative-quantitative debate:
implications for mixed methods research”, Quality & Quantity, Vol. 36 No. 1, pp. 43-53.
Scott, P.J. and Briggs, J.S. (2009), “A pragmatists argument for mixed methodology in medical
informatics”, Journal of Mixed Methods Research, Vol. 3 No. 3, pp. 223-41.
Shields, M.D. (1997), “Research in management accounting by North Americans in the 1990s”,
Journal of Management Accounting Research, Vol. 9, pp. 3-61.
Tashakkori, A. and Creswell, J.W. (2007a), “Editorial: exploring the nature of reserach questions
in mixed methods research”, Journal of Mixed Methods Research, Vol. 1 No. 3, pp. 207-11.
Tashakkori, A. and Creswell, J.W. (2007b), “Editorial: the new era of mixed methods”, Journal of
Mixed Methods Research, Vol. 1 No. 1, pp. 3-7.
Tashakkori, A. and Teddlie, C. (1998), Mixed Methodology: Combining the Qualitative and
Quantitative Approaches, Sage, Thousand Oaks, CA.
Teddlie, C. and Tashakkori, A. (2003), “Major issues and controversies in the use of mixed methods
in the social and behavioural sciences”, in Tashakkori, A. and Teddlie, C. (Eds), Handbook of
Mixed Methods in Social and Behavioural Research, Sage, Thousand Oaks, CA.
Teddlie, C. and Tashakkori, A. (2009), Foundations of Mixed Methods Research: Integrating
Quantitative and Qualitative Approaches in the Social and Behavioural Sciences, Sage,
Thousand Oaks, CA.
Webb, E.J., Campbell, D.T., Schwartz, R.D. and Sechrest, L. (1966), Unobtrusive Measures:
Non-reactive Research in the Social Sciences, Rand McNally, Chicago, IL.
Wouters, M. and Wilderom, C. (2008), “Developing performance-measurement systems as
enabling formalization: a longitudinal field study of a logistics department”, Accounting,
Organizations and Society, Vol. 33 Nos 4/5, pp. 488-516.
Yin, R.K. (2006), “Mixed methods research: are the methods genuinely integrated or merely
parallel?”, Research in the Schools, Vol. 13 No. 1, pp. 41-7.
About the authors
Jennifer Grafton is a Senior Lecturer in Management Accounting in the Department of
Accounting and BIS at the University of Melbourne. Jennifer Grafton is the corresponding author
and can be contacted at: j.grafton@unimelb.edu.au
Anne M. Lillis is Professor of Management Accounting in the Department of Accounting and
BIS at the University of Melbourne.
Habib Mahama is an Associate Professor of Management Accounting in the School of
Accounting and BIS at the Australian National University.
To purchase reprints of this article please e-mail: reprints@emeraldinsight.com
Or visit our web site for further details: www.emeraldinsight.com/reprints
Mixed methods
research in
accounting
21
Auditing: A Journal of Practice & Theory
Journal of Accounting Literature
Journal of
Managerial Issues
Journal of Accountancy
Accounting, Organizations and
Society
Auditing and assurance services
Understanding regulation: theory, strategy, and practice
Contemporary Accounting Research
Wall Street Journal
International Journal of Auditing
Research in
Accounting Regulation
Public Budgeting and Finance
Accounting, Organizations and
Society
Accounting,
Organizations and Society
Research in Accounting Regulation
The Accounting Review
Accounting Horizons
The Journal of Economic Perspectives
Journal of Accounting, Auditing &
Finance
International Journal of Auditing
Accounting Horizons
Journal of Accounting and Economics
Journal of Accounting and
Economics
Journal of Accounting and Economics
The Accounting Review
Auditing: A Journal of Practice & Theory
The CPA Journal
Behavioral Research in
Accounting
Federal regulation and regulatory reform (Report by the Subcommittee on Oversight and Investigations of the Committee on Interstate
and Foreign Commerce)
The accounting establishment (Subcommittee on Reports, Accounting and Management of the Committee on Government
Operations)
Journal of Accountancy
Research in Accounting Regulation
Accounting, Organizations and Society
The British Accounting Review
Journal of Accounting,
Auditing & Finance
Accounting, Auditing &
Accountability Journal
Research in Accounting Regulation
Accounting Horizons
Speach presented at the Colorado Society of CPAs 2005 SEC Conference
Current Issues in Auditing
Auditing: A Journal of Practice & Theory
Current Issues in
Auditing
Auditing: A Journal of Practice & Theory
The CPA Journal
Presented at the american accounting association annual meeting 2006
Journal of Accounting and Public Policy
The CPA Journal
CPA Journal
Accounting Horizons
Journal of Accounting and
Economics
Strategic Finance
Accounting Horizons
Accounting, Auditing & Accountability Journal
Advances in Public Interest Accounting
Behavioral Research in
Accounting
Auditing: A Journal of Practice &
Theory
Auditing: AJournal of Practice & Theory
Auditing: A Journal of Practice &
Theory
Journal of Business &
Economics Research
Journal of Accounting and
Economics
Accounting Research Journal
The Journal of Accountancy
Accounting, Organizations
and Society
Accounting Today
Journal of Accountancy
Auditing: A Journal of Practice & Theory
Creative accounting, fraud and international
accounting scandals
The CPA Journal
Auditing: A Journal of
Practice & Theory
The Accounting Review
Financial gatekeepers—can they protect investors?
Accounting Horizons
Research in Accounting Regulation
Accounting,
Organizations and Society
Accounting, Organizations and Society
Presented at the 2012 Deloitte Foundation University of Kansas Auditing Symposium
Accounting Horizons
Accounting & the Public Interest
Accounting and the Public Interest
Critical
Perspectives on Accounting
Journal of Accounting, Ethics and Public Policy
Managerial Auditing
Journal
Research in Accounting Regulation
Auditing: A Journal of Practice & Theory
Journal of Accountancy
Accounting, Organizations and Society
Accounting and Business Research
Accounting, Organizations
and Society
Auditing: A Journal of Practice & Theory
Research
in Accounting Regulation
Journal of Corporate Accounting & Finance
Accounting Horizons
Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.
Review of choice-based, matched, and other stratified sample studies in auditing research
Stuart, Iris;Shin, Yong-Chul;Cam, Donald P;Karan, Vijay
Journal of Accounting Literature; 2013; 32, ABI/INFORM Global
pg. 88
Purchase answer to see full
attachment