|
post-autistic
economics review |
|
issue 31
contents
PAE
Review index
home page |
|
Symposium on Reorienting Economics (Part IV) Reorienting Economics Through Triangulation of Methods Paul Downward and Andrew Mearman
Introduction Tony Lawson’s widely acclaimed book Economics
and Reality (1997) provided
a systematic ontological critique of mainstream economic analysis. The
critique focussed upon the inappropriateness of the ‘deductivist
method’ of neoclassical economics, which implicitly assumes a closed-system
ontology, and the open-system nature of reality. Deductivism
is an approach that invokes covering laws of explanation, whether or not
these are derived from formally deductive or inductive premises. This method
in turn assumes a closed-system
ontology, in which strict regularities of the type ‘if event X, then
event Y’ will occur. For Lawson, the emphasis on mathematical modelling and
econometric testing are evidence of this closed-system approach. Further, the
use of econometric testing, so defined, is an example of empirical realism – reflecting a belief in a flat ontology
comprising only observations of events. While essentially a critique, Economics
and Reality also advocated retroduction – the movement in thought from a phenomenon
of interest to the mechanism(s) which (at least in part) caused it – as the
appropriate logic of inference for an open system. The reasons are broadly
two fold. First, cause cannot be associated with the constant conjunction of
events in a non-experimental open-system, but rather the emergence of events
out of causal mechanisms which draw upon human agency and structures. Using
this method to gain an understanding
of events, the researcher then needs to establish the mechanisms that gave
rise to them. Second, our empirical observation of actual events and the
mechanisms that produce them will be related transfactually;
i.e., the causal mechanisms exist and may operate irrespective of the
observed events. Consequently our understanding of them will be fallible, a
fact that follows not only from the complex codetermination of events, but
also because of the hermeneutic issues associated with interpreting and
communicating our understanding of causes. For Lawson, retroduction
is likely to be achieved most successfully through contrast explanation, a method based on the premise that research
should be directed towards explaining ‘surprising’ breakdowns in patterns of
events, because here changes in causal mechanisms may be more apparent.
Contrast explanation thus shares the same structure as experimental reasoning
but, of course, reflects an open-system context in which causal mechanisms
are not isolated and stable. One criticism of Economics and Reality is that its practical guidance to
economists remains limited, beyond informing them what they ought not do. Reorienting Economics (2003)
also emphasises the ontological critique of mainstream method, although in
cautious terms. However, there is an expanded account of ‘contrast
explanation’ and a discussion of the merits of borrowing biological metaphors
in producing social explanation. Lawson demonstrates that heterodoxy’s appeal
to evolutionary biological metaphors as opposed to mathematical systems has
promise, but should be tempered by concern with the ontological relevance of
mathematics to social systems. Lawson then focuses upon a discussion of the
broad, often implicit, ontological similarities between heterodox approaches
to economics and critical realism. As such they offer possibilities for
redefining economics in a much more pluralistic way. We
believe that there is a need for practical guidance in conducting research
projects informed by critical realism, and in this paper we argue, drawing
upon ongoing work, for the adoption of triangulation
as a principle of research design. Triangulation is commitment in research
design to investigation and inference via multiple methods which are not
placed in any a priori hierarchy.
We argue that triangulation allows retroduction to
be made operational, facilitates pluralism – rather than rejecting outright
entire methods – and allows economics to be reorientated
towards other social sciences, as Lawson recommends (Downward and Mearman (2004a, 2004b, forthcoming). Thus, triangulation
can be an important step towards fulfilling Lawson’s project. Triangulation In social research in its broadest sense,
and moving away from the spatial origins of the metaphor, triangulation
implies combining together more than one set of insights in an investigation.
Denzin (1970) offers a taxonomy of triangulation:
1) data triangulation – the combination of different data types; 2)
investigator triangulation – the combination of insights from different
investigators; 3) theoretical triangulation – the combination of different
theoretical perspectives; and 4) methodological triangulation – the
combination of different methods. Denzin further
distinguishes between within-method
triangulation, i.e., combining different cases of the same method; and between-method triangulation, i.e.
combining studies of different methodologies. Clearly, between-method
triangulation is more radical, because it could involve, for example, an
econometric methodology in combination with an ethnographic method. We have
argued elsewhere that Critical Realism can create a philosophical basis for
triangulation. Triangulation is common in other applied
social sciences (see below) but not in economics. There is some use of
triangulation by professional (including government) economists, such as
those at the Bank of England, but its use by academic economists is rarer;
and when used it tends to be for pragmatic reasons. A common reason for
combination of data types is that the main source of data is incomplete. For
example, the Bank of England uses survey and anecdotal evidence from its
Agents as the most up to date picture of economic conditions. This
compensates for the inevitable lag in official quantitative data. Economists
cite epistemological reasons for triangulation, such as the fallibility of
knowledge, relatively rarely. In the literature of social science
research methods there is discussion of the basis for triangulation. It is
common to reject triangulation for
ontological reasons. Positivist
social scientists (or those influenced by positivism) tend to hold that
methods should only be used which conform to positivist principles, such as
objectivity, observability, and precision. All of
these principles are met by quantitative data. For interpretivists, the opposite
position is taken. Interpretivism includes
hermeneutic concerns that social phenomena are intrinsically meaningful; that
meanings must be understood; and that the interpretation of an object or
event is affected by its context. Moreover, for the above reasons, meanings
cannot be measured, counted or understood. Unsurprisingly, interpretivist approaches tend to focus on the
limitations of quantitative analysis in the social arena. Silverman (1993)
offers a typical example of the interpretivist
approach. Silverman argues that quantitative methods retain a positivist
perspective in which data collectors basically follow established protocols
and data providers simply reveal aspects required of the protocol as
‘objective’ entities. In contrast, qualitative methods are ‘interactionist’ and reflect the interviewer creating the
interview context and the interviewee engaging in a dialectic with the
definition of the situation, so that the research reflects social
relationships which are inherently subjective and not objective. On this
basis Silverman rejects quantitative methods as inappropriate to social
research. It remains the case that the justification
for triangulation in social science tends to emphasise pragmatism. This is
clearly unsatisfactory for a Critical Realist, whose concern is for
ontological consistency between method and material. This paper argues that
such consistency can be established because triangulation is essential to retroduction. The argument proceeds by considering
alternative research methods. Triangulation
Rather than Rejection of Methods Lawson (1997) spends considerable time
critically examining the use of econometrics in an open-system reality, in
which event regularities are unlikely to occur. It is easy to form the
impression from Lawson’s work that econometrics is being rejected as
inappropriate practice. Lawson (1999) clarifies this position, arguing that
econometrics is perfectly permissible providing that it is used in closed
systems; however, given that these are held to be unlikely to occur,
econometrics is likely to have extremely limited use. Downward and Mearman (2002) examine such Critical-Realist objections
and find them unsustainable. If methods are to be rejected because they
impose some degree of closure on an open reality, then in fact, all methods
so far proposed must also be rejected. Any method which supposes that an
entity remains fixed for long enough for it to be identified as an object of
study imposes closure on an open reality. Thus quantitative analysis involves
closure, because the act of quantification involves the assumption of
qualitative invariance across subjects. Likewise, if qualitative
investigation is concerned with collating insights and offering stylised
interpretations and narratives, this assumes qualitative invariance – or, in
Critical-Realist terminology, intrinsic
closure. Contrast explanation is thus challenged because, first, it tends
to involve quantitative contrasts; second, because it makes assumptions about
what are surprising, abnormal or significant instances, which in turn
presupposes a notion of a normal deviation from a trend or fixed pattern, and
finally because qualitative endeavours to explain the contrasts are not
fundamentally distinct in terms of maintaining a degree of closure. However,
this is not necessarily problematic per se. As Mearman
(forthcoming) notes, a Critical-Realist abstraction
necessarily involves a focus on what is real and essential to the temporary
exclusion of other factors, regarded as transient or insignificant. Moreover,
by abstracting from other factors, one assumes that those factors are behaving
in particular, consistent ways. Therefore, we argue that for practising
economists, Lawson’s arguments are potentially too destructive, leaving no
techniques available for concrete analysis. In contrast, as reality comprises
parts which have different degrees of openness, these should be explored by
methods which also exhibit different degrees of openness: for example,
statistical inference requires more closure than statistical estimation. The above discussion can be viewed as
reflecting (an aspect of) the fallibilism of all
knowledge claims. More generally, fallibilism of
theory tends to have two geneses: ontological and epistemological.
Epistemologically, humans have limited computational capacity to deal with
the world, even one of a simple structure, particularly when faced with
multiple meanings and incommensurability. Thus, no one theory is likely to be
able to capture adequately these aspects. Furthermore, if the object of
enquiry (and by extension, the world) is highly complex, it is extremely
difficult for any theory to capture its aspects adequately. This concern applies when the social world is
held to comprise open systems, in which, for example, the future is open,
because current agency affects future structures in unpredictable ways. Even
for a super computer, society is complex and unpredictable. In this light, no
one model should be relied upon to give an accurate picture of the object of
enquiry.1 Rather, methodological triangulation requires that in
any act of inference to causal claims about an object of enquiry, two (or more) methods with different
degrees of openness, are combined.. Simultaneously, triangulation provides the researcher with
a means of inference which addresses the concern over open systems and avoids
the rejection of methods. Thus, triangulation offers a way out of the practical impasse found in Reorienting Economics and in Economics and Reality.. Triangulation
and Retroduction As we have argued elsewhere, a
Critical-Realist position can provide an ontological justification for
triangulation; and triangulation is a necessary element of the logic of retroduction and hence is crucial for operationalising
Critical Realism (Downward and Mearman, 2002,
forthcoming). A central element of the Critical-Realist
programme is that every paradigm (and its associated methods and theories),
has an underlying ontology. Thus, the link between a method or theory and its
ontology cannot be avoided (Danermark, et al (2002:
152-3). How far do other philosophical positions meet this criterion? Above
it was argued that both positivist and interpretivist
approaches make a strong pre-commitment to a particular ontology that
explicitly rules out the need for (methodological) triangulation.
Specifically, positivism and interpretivism both
reject methods which do not meet their ontologies.
Consequently, one might imagine that both meet the Critical-Realist criterion.Nevertheless, Critical Realists criticise both
perspectives because each has empirical realist foundations: the empirical is
the basis for, and in fact constitutes, reality. For example, positivism
embraces an inductive view of explanation to which value-free or observation
of objective reality is crucial. Furthermore, either informally or formally,
through statements of initial conditions and assumptions, deduced
consequences or predictions are assessed empirically. Additionally, as noted
earlier, for Critical Realists, there is a factor common to deduction and induction and which characterises
their essential logic: explanations are presented in the form of ‘covering
laws’ which commits an epistemic
fallacy, i.e., to conflate what is experienced with what exists. The
conception or knowledge of phenomena manifest in the theorist’s ideas and
arguments is treated as logically equivalent to the phenomena under review.
In this sense, knowledge is presented as being effectively complete. This is
either in a literal or positivist sense in which theories directly represent
an external world, or in the ‘idealist’ or interactionist
sense that the world is merely the proposed collection of ideas, whether
these are deductive constructs or subjectively specific concepts. Thus, from the Critical-Realist perspective,
both quantitative and qualitative analysis (either based on phenomenology or
on hermeneutics) commit an epistemic fallacy; that is, they conflate the
subject and object of analysis through the invocation of covering laws.
Therefore, the traditional quantitative/qualitative choice can be viewed as
unnecessary and, moreover, reflects fallacious thinking from a critical
realist perspective. This argument opens up the possibility of triangulation. Critical Realists also espouse an ontology
of depth realism, which has two prongs: realism and depth. Realism holds that
the nature of the object drives the process of research. Depth implies that
specific types of methods are necessary for (social) scientific
investigation. Critical Realism implies that, because the objects of social
science are inherently complex and have many important aspects which belong
in various categories, it is not possible to capture some of the broader
aspects of the objects with narrow methods (for instance counting the frequency
of the entity). Furthermore, because, according to Critical Realists, agency
is absolutely crucial, so people’s motives cannot be ignored; and because
reasons are causes, it is essential to explore these different concepts in
analysis. Moreover, because reasons are causes, Critical Realists reject the
traditional equation of, on the one hand, qualitative equals exploratory or
descriptive; and on the other, quantitative equals explanatory methods (Danermark, et al, 2002: 163). Indeed, Critical Realists share
with pragmatists the view that to associate qualitative methods with
hermeneutics (or whoever) and quantitative analysis with positivism (or
whoever) is limiting – instead they are looking to transcend the dichotomy.
Also, research is governed by the need to get to the bottom of a question –
i.e., to uncover generative mechanisms – and whichever are the best methods
to use should be used (Danermark, et al: 162-4).
However, unlike pragmatism, realism has an ontological grounding to that
position. The connection between the ontology and the method – added to the
belief that reality is stratified – can be argued to lead some Critical
Realists to abandon the strict dual of quantitative/qualitative in favour of
one in terms of intensive versus extensive research design (Sayer, 1992: ch.9; Danermark,
et al: 2002, ch. 6). The former design is what
is typically thought of as a characteristic of social science. Research
explores the contextual relations surrounding a particular unit of analysis
(i.e., qualitative research). This is as opposed to the latter research
design which emphasises the formal relations of similarity between units of
analysis, that is, it produces taxonomic descriptions of variables (i.e.,
quantitative research). However, this seems to reinstate the old
dual despite the two types of method being treated as complementary empirical
procedures with complimentary strengths and weaknesses. Thus it is typically
argued that the causal insights from extensive research will be less.
Moreover, one is reminded that the validity of the (qualitative) analysis of
cases does not rely upon broad quantitative evidence. In this sense the
traditional view put forward for triangulation as validating qualitative
insights is not necessarily applicable. The focus on intensive/extensive research
designs in combination seems necessary because of the nature of reality. A
number of authors (for example, Olsen, 2003) have talked in terms of a zoom-lens approach, in which the
investigator necessarily zooms in to focus on critical detail of a case
(intensive research) but, for various reasons (including the often necessary
relations between an object and other objects), the investigator must then
zoom out, to get a wider sense of the object’s context. Yet, the ontological
basis for making such claims seems unclear. In a number of papers we address these
issues and argue that combining methods is central to retroductive activity. The
following discussion briefly restates these arguments. One of the consequences of the above
discussion is that ‘quantitative’ and ‘qualitative’ approaches are not a
dual: they overlap to a degree in underlying logic and can also refer to the
same objects of analysis. They can share, or be conditioned upon, the same
ontological perspective and are not of necessity wedded to particular, and
different, ontological presumptions. Rather, the choice of method is not
paradigmatic or one of ontology, because that ontology is shared by the
methods, but simply reflects the specifics of the question being asked. If
the questions probe different features of a phenomenon then different methods
might be needed. It remains that they focus on the same phenomenon. Two conclusions follow from this
discussion. The first is that different research methods can be logically
employed in a triangulating strategy to reveal different features of the same reality without the presumption
of being exhaustive. The second is that the Critical-Realist perspective
renders a need for the triangulation to have an explicitly ontological
dimension to capture related but different layers of this reality. Figure 1,
adapted from Downward (2003: 298) illustrates potential options. Figure 1 Applied
Critical Realism Philosophical Focus of Applied
Method Position Analysis Critical Realism Cause 'Qualitative' Grounded Theory/Discourse Analysis etc (Meaning/Categories/ Contextual Relations) Retroduction Quasi-closure Triangulation 'Quantitative' Sample Specific Univariate/Bivariate/ Multivariate Parametric/Nonparametric e.g. Descriptive statistics Frequencies, cross-tabulations Correlations Regression
Factor analysis ANOVA Inferential
Univariate/bivariate/ Multivariate
Parametric/Nonparametric e.g. Empirical Realism Events - - - - - - - - - - - - - - - - - (as causes) Tests of mean/variance
differences Tests
of association Tests of overall and specific parameter significance Canonical regression Discriminant Analysis Cluster analysis In the first column are the two opposing
positions identified in the Critical-Realist literature; Critical Realism and
empirical realism. The latter is, of course, the empirical counterpart to deduction,
induction and the hypothetico-deductivist model of
explanation. The next column describes the focus of analysis framed within
these alternative philosophical positions, which are, respectively, real
causes and empirical events. In the case of empirical realism, relationships
between events are purported to reveal causes in the covering law sense as
indicated by the horizontal broken line linked to, say, typical statistical
testing methods. In contrast, Critical Realism maintains
that investigating causes involves moving below the level of events through retroduction. The third column thus reveals that
corresponding to what is typically identified as ‘qualitative’ research
methods, the context-laden meaning of concepts, categories and relationships
can be established and causal narratives constructed. However, this process
of defining shared meanings and categories in essence breaks down the
qualitative orientation of the research and, of necessity, begins to invoke
aspects of closure in seeking a degree of generality and purporting to refer
to the same object of analysis. Hence the direction of ‘quasi closure’
indicated in the column. Thus, categorising phenomena implies assuming
invariant (or a degree of invariance in) qualities. In essence, defining a
variable requires at least intrinsic closure. Increasing the degree to which closure is
invoked thus begins to ‘legitimise’ various statistical procedures. These can
refer to one, two or many variables but be either sample specific or
inferential in orientation, which means that a probability distribution is
referred to in a parametric or non-parametric manner to make claims about the
generality of the purported relationships between variables. It is, of
course, clear that the degree of closure assumed increases as one moves down the column. For example referring
to sample specific descriptive methods of analysis such as averages,
correlations or regression implies that values of variables have consistent
meanings and that these variables can be combined in a relatively constant or
enduring manner as indications of, say, outcomes of causal links. Probabilistic inference assumes, much more
strongly, that the results carry over, in a measurable sense, to contexts
beyond the sample. In this sense one is increasingly invoking the extrinsic condition of closure, and
one can clearly see the strength of the assumptions underpinning the methods
which appear in this part of the column. What the diagram does reveal, in
entirety however, is that in general movements towards statistical methods
naturally shifts attention towards ‘events’ as opposed to ‘causes’ which are
in essence qualitative. On this basis one can argue that rather than
revealing covering laws, in contrast statistical methods can reveal phenomena
from which causal research can begin and combinations of which contribute to
our understanding of the phenomena under investigation (Downward & Mearman, 2004c). Thus the final column reveals that
triangulation between the methods - that is linking the insights gained from
these different research methods captures the retroductive
logic of critical realism. Importantly, any quantitative analysis becomes
merely a scenario whose legitimacy will rest upon the robustness of the
qualitative invariance invoked in causal mechanisms and, of course, the lack
of influence of countervailing causes. The discovery and robustness of such
causal claims, along with their implications, will of necessity always be
open to revision. In this respect, probabilistic inferences are also
conditional upon and should be assessed in connection with analysis of the
nature of the object under investigation. Conclusions This paper has argued that Lawson’s Reorienting Economics still leaves significant
lacunae in its implications for what economists, if they take Lawson
seriously, are to do in their concrete research. Moreover, we interpret
Lawson’s new contribution as reinforcing his previous ones on the proper role
of econometrics and other ‘traditional’ methods in economics: that such
methods should not be used unless in highly specific narrow circumstances. It
has been argued that a commitment in one’s research design to methodological
triangulation, i.e. inference via the combination of methods (informed by
different and perhaps competing philosophies), can contribute several
significant benefits: 1) the rejection of methods apparently implied by
Lawson is avoided; 2) the self-defeating implication of (1), viz., that all
empirical analysis is invalid in open systems, is also avoided; 3)
triangulation is consistent with depth realism; and 4) triangulation is
essential to the operationalisation of the
Critical-Realist logic of retroduction. Moreover, triangulation allows economics
to be reoriented toward the social sciences, as Lawson recommends in Reorienting Economics. In other social
sciences, and implied by Denzin’s taxonomy,
triangulation is much more widespread. For example, Danermark
et al (2002: 152) claim that within the sociological community the view is
widely supported that there is no universal method and that there is a need
for multimethodological approaches. Thus, in the
applied social sciences, triangulation is common in nursing, health and
education, and tourism (see, for example, Downward and Mearman,
forthcoming). Yet in economics,‘scientific status’
is sought, where scientific status is thought to mean systematic explanation,
shaped by empirical evidence, and arrived at via a narrow set of methods.
Hence, economics is typically perceived to be closer to the ‘hard’ sciences
than other social sciences because of the axiomatisation
of the discipline (Hausman, 1998). Economics stands
alone from other branches of social enquiry and, indeed, disciplines. Thus, triangulation raises interesting
questions concerning the nature of social science. If one mixes methods of
research and, in so doing, attempts to bring specific disciplinary tools to
the analysis then such a ‘multi-disciplinary’ approach will entail the
ontological clashes discussed earlier because, by construction the different
disciplines embrace different methods and, as a result, different ontologies as expressed by traditional philosophy of
science. In contrast, to ‘unite’ social science, what is required is an
attempt to transcend the separate disciplines to produce an
‘inter-disciplinary approach’. Social science, so defined, naturally involves triangulation,
because the methods qua disciplinary
boundaries are removed. It is argued in this paper that Critical Realism provides
the methodological apparatus within which such a view of social science can
be constructed. Aspects of the subject matter of the disciplines, if not the
currently expressed nature of the disciplines, thus become branches or fields
of the same domain of investigation brought together by triangulation. Note 1. Bhaskar (1978: 43) supports fallibilism
as concomitant with realism. However, Bhaskar
(1978: 197-9) rejects fallibilism as an overarching
concept, because it can be confused with judgemental relativism, i.e. the
notion that all beliefs or theories have equal merit. Bhaskar
prefers his concept of epistemic relativism, which allows for theories to be
incorrect, but more correct than other theories. However, Sayer
(2000: 20) notes that in open systems, there is always the possibility of
“misattributions of causality”; i.e., fallibilism. Danermark et al (2002: 152-3)’s caution that
investigators must be very careful when making inferences. They also refuse
to rule out a priori any type of
methods; however, this might reflect more strongly their position that any
methods must be informed by the nature of the object under study.
Furthermore, though, Critical Realists argue that agents’ responses are
corrigible, which means that replies to questionnaires, etc. cannot be taken
completely at face value. Thus, the investigator’s evaluation of a subject’s
responses is an important element in research; and given the corrigibility of
responses, other data types or methods should be utilised. More generally,
data might also be fallible, perhaps because it is incomplete or its
collection was difficult. References Bhaskar, R. (1978). A
Realist Theory of Science, London: Verso. Danermark, B., Ekstrom, M., Jakobsen, L. and Karlsson, J. Ch., (2002). Explaining Society: Critical Realism in the Social Denzin, N.K. (1970). The
Research Act in Sociology, Chicago: Aldine. Downward,
P. (2003). ‘Conclusion’, in P. Downward (Ed.) Applied Economics and the Critical Realist Critique, London: Downward,
P. and Mearman, A. (2002). ‘Critical Realism and
Econometrics: A Constructive Dialogue with Post Keynesian __________________________(2004a).
‘Triangulation and Economic Methodology’, paper presented at the conference __________________________
(2004b). ‘Triangulation and Economics: Reorienting Economics into Social
Science’, __________________________
(2004c). ‘Presenting “Demi-regularities” of Pricing
Behavior: the Need for Triangulation,’ _________________________
(forthcoming). ‘On Tourism and Hospitality Management Research: A Critical
Realist Hausman, D. (1998). Economics, Philosophy
of, in Routledge
Encyclopaedia of Philosophy edited by Edward Craig. Lawson,
T. (1997). Economics and Reality,
London: Routledge. _________
(1999). ‘Connections and Distinctions: Post-Keynesianism and Critical
Realism’, Journal of Post Keynesian _________
(2003). Reorienting Economics,
London: Routledge. Mearman, A. (forthcoming). ‘Critical Realism in
Economics and Open-Systems Ontology: A Critique’, Review of Social Olsen, W.
(2003). ‘Triangulation, Time and the Social Objects of Econometrics’, in P.
Downward (Ed.) Applied Economics Sayer, A. (1992). Method
in Social Science: A Realist Approach, London: Routledge.
_______
(2000). Realism and Social Science,
London: Sage. Silverman, D.
(1993) Interpreting Qualitative Data:
Methods for Analysing Talk, Text and Interaction, Sage: London. ___________________________ |