The following is a discussion of the causal-comparative method that occurred in the spring of 1998 on the AERA-D listserv. I have tried to capture as much of this thread as I could, and reproduced it here for class. It is quite extensive, but very interesting. The essence of the disagreement concerns the nature of this research method, how it relates (or differs from) correlational methods, and the whole concept of causality. Enjoy!
Date: Sat, 7 Feb 1998 11:53:35 -0500
From: Bill Castine <wcastine@FAMU.EDU>
Subject: Re: Research Methods Question
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii; x-mac-type="54455854";
x-mac-creator="4D4F5353"
Content-transfer-encoding: 7bit
Donald F. Burrill wrote:
> On Fri, 6 Feb 1998, Burke Johnson wrote:
>
> << snip >>
>
> > Here are my questions:
> >
> > 1. Does anyone know who coined the term "causal comparative"
> > research? (I have never seen the term outside of education.)
> > When was it coined?
> I don't know; but it may be pertinent that have not
> heretofore
> encountered the term, and I did try to keep reasonably current for the
>
> "Introduction to Research in Education" course until my retirement in
> 1993.
I recall this term from at least 25 years ago. It may well have been
mentioned in Campbell and Stanley's work published in the 1963 Handbook
of Research on Teaching and later as a separate paperback. It's at the
office and I'm at home, so I can't verify at the moment.
Many students often miswrite the term as "casual comparative," which may
be an unwittingly accurate statement.
> > 2. Why do educational researchers, such as these textbook authors,
> > believe that evidence about cause and effect will be any stronger in
>
> > causal comparative research than in correlational research?
[[snip]]
> To those suggestions I'm tempted to add "an unawareness, or at
> least a lack of appreciation, of the famous work by Campbell and
> Stanley
> (1966)".
> And it wouldn't surprise me if this belief turned out to be
> related to the regrettably pervasive mathophobia with which we are all
>
> far too familiar, but I'd as soon not speculate in that direction...
> -- DFB.
The notion that educational researchers "believe that evidence about
cause and effect will be any stronger in causal comparative research
than in correlational research" appears to be an overgeneralization.
Many, if not most, of us hold a viewpoint along the following lines:It
is generally accepted that the only way to _establish_ cause and effect
relationships is via experimental research; however, either
causal-comparative or correlational research may indicate _possible_
causes, which can be tested further through experimentation.
In a sense, causal comparative research requires one to "play
detective." Whatever variables have acted, have acted. The
researcher/detective's task is to identify whodunit and/or how.
Correlational research involves a sifting of data to see if two (or
more) variables fall out in the same pile. In either case, potential
causality has been only indicated--not established. Further work is
needed to examine the probability of causal relationships.
_______________
Date: Sat, 7 Feb 1998 17:42:11 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Research Methods Question -Reply -Reply
MIME-version: 1.0
Content-type: text/plain
Darrell L. Sabers (Professor of Educational Psychology) writes:
"I guess I see the differences in studies trying to show causation and
studies trying to establish the size of the correlation coefficient as being
very different."
Burke Johnson's comment begins here:
Similar to Darrell's viewpoint, the leading selling textbook in education
(written by Gay) says: "correlational research...describes conditions that
already exist. Causal-comparative research, however, also attempts to
determine reasons, or causes, for the current status of the phenomena
under study" (4th edition, p.283). "Causal-comparative studies attempt to
identify cause-effect relationships, correlational studies do not" (p. 284).
"The purpose of a correlational study may be to determine relationships
between variables, or to use relationships in making predictions" (p.264).
"...variables that are highly related may suggest CAUSAL-COMPARATIVE
or experimental studies to determine if the relationships are causal"
(p.264).
Burke Johnson's comment to Sabers and Gay follows:
The above is the myth that the scaling of an independent variable has
something to do with attribution of cause and effect. Other things equal,
it does not matter if the independent variable is categorical or if it is
quantitative. Without additional information about how a particular study
was designed and conducted, we simply cannot say whether a
causal-comparative study OR a correlational study provides more or less
evidence about cause and effect, and it is a problem when the leading
textbook does exactly this. I will also assume that if the leading textbook
does this then there must be some teachers of educational research
who are also doing it. BTW, it would obviously be just as wrong to
attribute cause and effect based on a study with a single categorical
Independent variable (with no controls or any attempt to establish time
ordering) as it would with a study with a single quantitative independent
variable (with no controls or attempt to establish time ordering)...
Here is Burke Johnson's comment about Darrell's statement about a
correlation coefficient measuring the magnitude of a relationship:
First, there is MUCH more to correlational research than calculating a
bivariate correlation coefficient...Furthermore, hopefully one day we will
begin teaching that ANOVA and correlation are both cases of the
general linear model. Most methodologists are currently calling for
ANOVA users to provide estimates of effect size in addition to tests of
statistical significance. In fact, whether we are doing an ANOVA OR a
regression, we can all provide 1) p-values for determination of statistical
significance AND 2) estimates of effect size for each independent
variable.
My bottom line is that I hope that we as teachers of educational research
will attempt to debunk the myth that the scaling of the independent
variable (i.e., is it categorical or quantitative) has anything to do with the
attribution of cause and effect, and I hope we will pass our viewpoints
along to our book reps.
Thanks for the open discussion...
Burke Johnson
P.S. For more information about correlational versus causal-comparative
research, you can read the following message from an earlier post...
Burke Johnson said:
2. Why do educational researchers, such as these textbook authors,
believe that evidence about cause and effect will be any stronger in
causal comparative research than in correlational research?
Darrell Sabers said:
There is a major difference in the purpose of the two methods.
Burke Johnson replied:
The answer to the question about cause and effect cannot be in the
form "because we DEFINE causal comparative and correlational
research as having different purposes." My question is, stated
differently, why is your evidence about cause and effect any stronger
for a study about the effect of gender (a nominal variable) on
achievement than a study of self-esteem (a quantitative variable) on
achievement? If I were to accept your definitions of causal comparative
and correlational research, I would be justified in making such a
conclusion.
Darrell Sabers said:
Causal comparative is intended to substitute for experimental research
when one cannot manipulate the independent variable. However, control
over rival hypotheses are attempted in order to provide evidence for
causation. In C_C research, one attempts to a) show correlation
between x and y b) show x precedes y, and c) rules out rival
hypotheses. In correlational research, one shows only a) above and
perhaps the degree to which the variables are correlated.
Burke Johnson replied:
I can show you many examples of published causal comparative
research that do not attempt to control for extraneous variables, and I
can show you many examples of published correlational research that
do attempt to control for extraneous variables...The way to get a read on
time order in nonexperimental research is via theory (which is the
approach often used in structural equation modeling) and by designing
prospective or longitudinal studies. And there is no REQUIREMENT that
the independent variable must be categorical in prospective or in
longitudinal research. Finally, if someone claims they can make a
stronger statement about cause and effect with a categorical
independent variable than with a quantitative independent variable, then I
guess they would always collapse their continuous independent variable
into categories, so that, somehow, their design will now be
stronger--BTW, if they do this, then the difference between causal
comparative and correlational research is irrelevant (i.e., the big
difference suddenly disappears).
DS said:
If the way I present it is accurate, the two are very different, and the
C_C approach can be used to suggest causation whereas correlational
cannot.
BJ said:I don't see the difference, except by definition.
2/8/98
Date: Sun, 8 Feb 1998 10:09:44 MST
From: Norman David Giesbrecht <ndgiesbr@ACS.UCALGARY.CA>
Subject: Causal-Compartive vs. Correlational
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
The differences between causal-comparative vs. correlational studies
_should_ be more than definition or interpretation of results.
In its simplest form, a correlational study provides evidence for
a _relationship_ between some variables - that may or may not
have some theoretical basis (e.g., many correlational studies
may simply be empirical explorations with pragmatic ends).
Causal-comparative, as the name implies, are (or should be)
grounded in causal hypotheses and _designed_ to explore
evidence of causal relationships within the constraints of
a comparative context. Obviously, a comparative analysis
does not provide evidence of causality ... so what to do
within the limitations of a single measurement point ?
There are no truly satisfying solutions, but some approaches
include exploring issues of measurement and analysis. i.e.,
we can recognize that certain observations (or measurments)
reflect a history .. is 'gender' a measurement of one's
current state of 'sex' or does it also reflect a history
of socialization; one can also explore state vs. trait
measures, 'recollected' parenting style etc. (recognizing
the _limitations_ inherent in the measure and the confounding
influence of both history and present vantage point etc);
'age' includes shared cohort experiences (e.g., Vietnam war)
Clearly, there are _major_ limitations and problems with
these assumptions and measurements ... but they reflect
a certain intentionality in design and measurement.
Similarly, in the analysis ... one can use procedures
such as structural equation modelling to test whether
the correlational patterns within the data are
_consistent_ with hypothesized causal relationships.
One is of course limited to a conclusion that 'the
pattern of relationships is consistent with a
specific causal hypothesis'. One can also evaluate
competing causal models and discuss which theoretical
perspective is best 'fitted' by the correlational patterns
in the data. On a simpler level, one can use
multiple-regression based path analysis to explore /
evaluate patterns of covariance. In interpretation,
there are also certain (say time-related) inferences
that one can put forth. A correlational study may
show a relationship between certain cognitive
processing styles and gender or age ... although
this is a correlational results based on a single
measurement point, one would not argue that gender
or age were possibly a 'causal' outcome of cognitive
style ... though one might suggest that the results
are consistent with a causal hypothesis that
developmental influence linked to gender or age
played a role. Covariation / zero-order correlations
also allow us to compare correlational patterns when
we 'control' (statistically remove the variation
associated with) certain measurements / variables.
Norman Giesbrehct, Ph.D.
P.S. Although not specifically stated, major caveats
re: the limitationscausal-comparative measurement
and analytic assumptions hold true. The intention
of the posting being to highlight some implications
for design, measurment and analysis when one
sets causal-comparison as the research objective
(vs. simple correlation)
____________________
2/9/98
Date: Mon, 9 Feb 1998 11:00:32 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Causal-Compartive vs. Correlational -Reply
MIME-version: 1.0
Content-type: text/plain
My comments (Burke Johnson) are after segments of Norman
Giesbrecht's text. Norman's text is in quotes:
>>> Norman David Giesbrecht <ndgiesbr@ACS.UCALGARY.CA>
02/08/98 11:09am >>>
"The differences between causal-comparative vs. correlational studies
_should_ be more than definition or interpretation of results.
In its simplest form, a correlational study provides evidence for
a _relationship_ between some variables - that may or may not
have some theoretical basis (e.g., many correlational studies
may simply be empirical explorations with pragmatic ends)."
--> Causal comparative also examines the relationships between
variables. Doesn't the most simple form of a causal comparative examine
the relationship between a single categorical independent variable and a
single dependent variable?
-->many causal comparative studies may also "simply be empirical
explorations with pragmatic ends"
"Causal-comparative, as the name implies, are (or should be)
grounded in causal hypotheses and _designed_ to explore
evidence of causal relationships within the constraints of
a comparative context. "
-->the name CAUSAL-Comparative is a misnomer. (In my earlier post I
asked who introduced this term into education so that I could learn more
about the history of the term. So far, no one seems to know. The term is
not used outside of education.) Some of my students believe after
reading their educational research methods books that a causal
comparative design provides MORE evidence for cause and effect than
correlational AND EXPERIMENTS. Why should a nonexperimental method,
that happens to have a categorical independent variable, have the word
causal in it? Do I suddenly have a right to think causally if I categorize my
quantitative independent variable? For example, do I have more basis for
thinking causally if I examine high versus low motivation instead of a
quantitatively scaled motivation variable? If any approach should have
the term causal in it, it would be experimental. And in terms of causal
comparative versus correlational, my previous posts should make it
exceedingly clear that the scaling of the independent variable
(categorical in C-C and quantitative in correlational) has NOTHING to do
with one's ability to make cause and effect statements. Do you disagree
with this statement?
"Obviously, a comparative analysis does not provide evidence of
causality"
-->And, again, the degree of truth to this statement has nothing to do
with the scaling of an independent variable (i.e., is it categorical as in C-C
or is it quantitative as in Correlational)
" ... so what to do within the limitations of a single measurement point ?
There are no truly satisfying solutions, but some approaches
include exploring issues of measurement and analysis. i.e.,
we can recognize that certain observations (or measurments)
reflect a history .. is 'gender' a measurement of one's
current state of 'sex' or does it also reflect a history
of socialization; one can also explore state vs. trait
measures, 'recollected' parenting style etc. (recognizing
the _limitations_ inherent in the measure and the confounding
influence of both history and present vantage point etc);
'age' includes shared cohort experiences (e.g., Vietnam war)
Clearly, there are _major_ limitations and problems with
these assumptions and measurements ... but they reflect
a certain intentionality in design and measurement."
-->regarding your points about measurement: they apply equally to
quantitative variables. For example, self-esteem also reflects history,
personality, and even genetic factors. Statements about cause and
effect are not contingent upon the scaling of the independent variable,
and, hence, there is no necessary superiority of causal-comparative
designs over correlational designs.
"Similarly, in the analysis ... one can use procedures
such as structural equation modelling to test whether
the correlational patterns within the data are
_consistent_ with hypothesized causal relationships."
-->this applies to both C-C and correlational designs. Categorical as well
as quantitative variables can be used in structural equation modeling.
You point that we use theory to give evidence about time ordering and to
make predictions to be tested in empirical data is well taken.
"One is of course limited to a conclusion that 'the
pattern of relationships is consistent with a
specific causal hypothesis'."
--> I agree. Furthermore, more than one model specification may
empirically "fit" the data (the theory overidentification problem). Additional
research and open discussion by other researchers in the empirical
literature can help find "holes" in the theory.
" One can also evaluate competing causal models and discuss which
theoretical perspective is best 'fitted' by the correlational patterns
in the data. On a simpler level, one can use multiple-regression based
path analysis to explore / evaluate patterns of covariance. In
interpretation, there are also certain (say time-related) inferences
that one can put forth. A correlational study may show a relationship
between certain cognitive processing styles and gender or age ...
although this is a correlational results based on a single
measurement point, one would not argue that gender
or age were possibly a 'causal' outcome of cognitive
style ... though one might suggest that the results
are consistent with a causal hypothesis that
developmental influence linked to gender or age
played a role. Covariation / zero-order correlations
also allow us to compare correlational patterns when
we 'control' (statistically remove the variation
associated with) certain measurements / variables."
-->Your points in this last section are well taken. And, they more
generally apply to both causal-comparative and correlational designs.
BTW, structural equation modeling can be used on longitudinal as well as
cross-sectional data. More importantly for our discussion, causal
comparative and correlational designs can both be done using
longitudinal or cross-sectional data.
-->Again, my point is that, ceteris paribus, one cannot say whether a
causal comparative design or a correlational design provides more or
less evidence of cause and effect. However, the leading selling textbook
in education (by L.R. Gay) does precisely this, and has done so for over
two decades now. How many students does that add up to having
heard the statement "Causal comparative studies attempt to identify
cause-effect relationships, correlational studies do not" (Gay, 4th ed,
page 284). The correct point is that the scaling of an independent
variable has nothing to do with statements about cause and effect.
"Norman Giesbrehct, Ph.D. P.S. Although not specifically stated, major
caveats re: the limitationscausal-comparative measurement and analytic
assumptions hold true. The intention of the posting being to highlight
some implications for design, measurment and analysis when one sets
causal-comparison as the research objective (vs. simple correlation)"
-->your implied comparison is a straw man. Why would one want to
compare a causal-comparative approach where attempts have been
made to control extraneous variables and establish time order versus a
correlational design that only examines a simple correlation? This is
exactly what many educational research methods books have been
doing for years. They act as if correlational researchers know how to do
little more that examine bivariate correlations. Books need to compare the
simple form of causal-comparative versus the simple form of
correlational, and the more complex forms versus the more complex
forms.
-->I hope others will become involved in our open discussion; perhaps
we can have an impact on the authors of our textbooks.
-->Regards.
-->Burke Johnson
__________________
_________________
Date: Mon, 9 Feb 1998 11:39:23 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Research Methods-Reply -Reply
MIME-version: 1.0
Content-type: text/plain
Following is Bill's comment to mine. My reply to his post is given at the
end...
>>> Bill Castine <wcastine@famu.edu> 02/09/98 09:48am >>>
Burke Johnson wrote:
> Why wouldn't a researcher conducting correlational research have to
> "play detective" just as much as a researcher conducting
> causal-comparative research? The same rules of evidence apply to
both
>
> approaches if one is attempting to gather any evidence of cause and
> effect, right?
Indeed you are correct, Burke. I guess I didn't emphasize that point
because one generally undertakes a causal comparative project with the
express notion of rooting out possible causative variables leading to
some previously observed effect. Too many correlation studies appear
to
be undertaken from the standpoint that "we have all these data; let's
run a correlation and see what falls out." In other words, the primary
motive often is to seek _any_ relationship, irrespective of cause. This
may be a minor point--and a flaw in the researcher rather than the
method. In any event, I guess your last sentence includes a premise
which is not always the case: "if one is attempting to gather any
evidence of cause and effect...." Sometimes that's a big "if."
Thanks for your feedback.
Bill
This is my comment to Bill's post...
-->Bill, The purpose of my original post was to point out that educational
research methods books are mistaken in saying that causal comparative
research can be used to probe cause and effect while correlational
cannot. That is simply not true. You point out in your post that you think
causal-comparative researchers are better at thinking about cause and
effect than researchers doing correlational research. This seems to me
to be a stereotype that might be true for some researchers, but it is
certainly not true for many other researchers. For example, examine the
journal called American Sociological Review, which reports mostly
nonexperimental research, and then tell me if you believe most
correlational research pays little or no attention to control. Obviously the
quality of a journal is typically related to the quality of the articles. I
contend that if we taught how to do good correlational AS WELL AS
good causal-comparative research, then our field would be better off. It
only hurts our field when some of our books reproduce stereotypes.
That is not the purpose of a textbook, and we can do better. Good books
should teach good practice. It does not help our field to suggest as LR
Gay does that causal-comparative research can be used to explore
cause and effect but correlational research cannot. The scaling of the
independent variable has nothing to do with cause and effect evidence.
Regards...
Burke J.
________________
2/10/98
Date: Tue, 10 Feb 1998 00:30:42 -0500
From: Bill Castine <wcastine@FAMU.EDU>
Subject: Re: Research Methods-Reply -Reply
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii; x-mac-type="54455854";
x-mac-creator="4D4F5353"
Content-transfer-encoding: 7bit
Burke Johnson quoted my personal post to him, which said in part:
> . . . one generally undertakes a causal comparative project with the
> express notion of rooting out possible causative variables leading to
> some previously observed effect. Too many correlation studies appear
> to be undertaken from the standpoint that "we have all these data;
> let's
> run a correlation and see what falls out." In other words, the
> primary
> motive often is to seek _any_ relationship, irrespective of cause.
>
> This is my [Burke Johnson's] comment to Bill's post...
>
> <<snip>>
> You point out in your post that you think
> causal-comparative researchers are better at thinking about cause and
> effect than researchers doing correlational research.
<<snip>>
To which I (Bill) reply: Please reread my note. I never said I thought
anyone was "better at thinking about cause and effect...." I simply
said the basic premise varied amongst various researchers. In a much
earlier post I pointed out that experimentation was the only accepted
way to _establish_ causality, though various other methods might
_indicate_ it. Most of us accept that "correlation does not imply
causality."
Bill
________________
Date: Tue, 10 Feb 1998 16:53:43 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Causal-Compartive vs. Correlational -Reply -Reply
MIME-version: 1.0
Content-type: text/plain
My most recent comments are in CAPS OR I BEGIN THE PARAGRAPH
WITH CAPS . . .
>>> "Norman David Giesbrecht" <ndgiesbr@acs.ucalgary.ca> 02/10/98
11:26am >>>>
> Recently "Burke Johnson" wrote ...
> --> Causal comparative also examines the relationships between
> variables. Doesn't the most simple form of a causal comparative
examine the relationship between a single categorical independent
variable and a
> single dependent variable?
Causal-comparative is _not_ (essentially) "a categorical independent
variable and a ... dependent variable".
Burke Johnson's reply:
->IT IS ACCORDING TO LR Gay's textbook, and that's what my comment
was about. Her textbook teaches through words and examples that
causal comparative research has at least one categorical IV and
correlational research is based on quantitative variables. We can both
think of exceptions to this rule, but, this is what is being taught to our
students via their textbooks. I find myself having to constantly correct the
textbook. Below, I will I provide some quotes from the textbook by L.R.
Gay. Also, take a look at Slavin's textbook. He does not buy into the
artificial distinction between correlational and causal-comparative
research. He titles his chapter "nonexperimental quantitative research"
and he says: "When correlational studies use categorical as independent
variables, they are often called _causal-comparative designs_...In
concept, causal-comparative studies are not any different from other
correlational studies using categorical variables, [note that Slavin also
doesn't hide the point that correlation coefficients are available for
categorical variables] except that researchers using causal-comparative
designs often use statistics (such as t-tests and F's) to compare the
different groups..."(p. 55). BTW, also note that Kerlinger dropped his use
of the term ex-post facto in his 1986 3rd edition of his well known book.
He chose the new term "nonexperimental."
Burke Johnson said earlier that:
> -->many causal comparative studies may also "simply be empirical
> explorations with pragmatic ends"
Norman David Giesbrecht replied...
As can _any_ research design (including experimental). A good
research design is more a function of the researcher than
the approach ... _however_, I would still maintain that
certain designs (in which I would include both experimental
and causal-comparative) can or should push a researcher
to greater methodological rigor.
-->AND I WOULD ADD CORRELATIONAL DESIGNS TO YOUR
REFERENCE TO EXPERIMENTAL AND CAUSAL-COMPARATIVE in your
note in parentheses . IF we are interested in explanatory research, then
people conducting correlational research also need " "GREATER
METHODOLOGICAL RIGOR."
> Burke Johnson from earlier post-->the name CAUSAL-Comparative is a
misnomer. (In my earlier post I asked who introduced this term into
education so that I could learn more about the history of the term. So far,
no one seems to know. The term is > not used outside of education.)
Some of my students believe after> reading their educational research
methods books that a causal comparative design provides MORE
evidence for cause and effect than > correlational AND EXPERIMENTS.
>why should a nonexperimental method, that happens to have a
categorical independent variable, have the word> causal in it? Do I
suddenly have a right to think causally if I categorize my > quantitative
independent variable? For example, do I have more basis for > thinking
causally if I examine high versus low motivation instead of a >
quantitatively scaled motivation variable?
Norman David Giesbrecht replied...
>A categorical independent variable doesn't make anything causal ...
and _no_ researcher would ever claim this. Scaling is a characteristic
of measurement and has _nothing_ to do with causal statement. No one
would disagree with your statement and no one makes that claim.
Reply By Burke Johnson.
-->ON THE CONTRARY, BASED ON A READING OF LR GAY's textbook
on research methods, ONE WOULD HAVE TO DISAGREE WITH YOUR
STATEMENT. TRY asking your students sometime after reading Gay if
more evidence can be obtained from a C-C design than from a CORR
design. I know what I read in Gay, and I know what my students have
told me when I have asked them. GAY STATES THAT ONE CAN MAKE
STATEMENTS ABOUT CAUSE AND EFFECT WITH C-C RESEARCH BUT
NOT WITH CORR RESEARCH. Obviously, both you and I know that this is
certainly not the case, and this is the point I have been addressing in all
of my previous posts. If you look at any example in the Gay book, you
will see that the difference between a causal-comparative design and a
correlational design is the scaling of the independent variable. Note the
Slavin quote above, and note the following quotes.
HERE ARE A FEW QUOTES FROM GAY FOR A REMINDER ABOUT WHAT
IS STATED IN THE TEXTBOOK BY GAY: "CORRELATIONAL
REESEARCH...describes conditions that already exist.
Causal-comparative research, however, also attempts to determine
reasons, or causes, for the current status of the phenomena under
study" (4th edition, p.283). "Causal-comparative studies attempt to
identify cause-effect relationships, correlational studies do not" (p. 284).
"Corrrelational research attempts to determine whether, and to what
degree, a relationship exists between two or more quantifiable variables"
(p.14). "Causal-comparative and experimental research...both attempt to
establish cause-effect relationships; bot involve group comparisons"
(p.15). In a correlational study "each variable must be experssible in
numerical form, that is, must be quantifiable" (p.279). "The purpose of a
correlational study may be to determine relationships between variables,
or to use relationships in making predictions" (p.264). "...variables that
are highly related may suggest CAUSAL-COMPARATIVE or experimental
studies to determine if the relationships are causal" (p.264).
WHAT I AM CONTENDING IS 1) OTHER THINGS EQUAL, ONE CANNOT
MAKE ANY STRONGER STATEMENTS about cause and effect with a
causal-comparative design than with a correlational design. My
distinction between causal-comparative and correlational designs is that
the former has a categorical IV whereas the latter has a quantitative IV.
This distinction is based on Gay and many other Educational Research
textbooks. I personally believe that the distinction between C-C and
CORR should be dropped (i.e., why don't we talk about nonexperimental
research rather than C-C versus Correlational) and/or that statements
that C-C provides greater causality evidence than correlational should be
dropped. That's all I am trying to say. What do you think about these two
points?
An earlier comment by Burke Johnson:
>If any approach should have
> the term causal in it, it would be experimental.
Norman David Giesbrecht replied...
I believe that here is where we start to get at the heart of the matter.
Perhaps the reason that causal-comparative is well-rooted in education
is linked to the inadeqacy of experimental designs in addressing
complex psychosocial processes. It is simply _not_ possible to
"control" the complex dynamics that are involved. Causal-comparative
is often more sensitive (or should be) to the multiple influences
than experimental designs _In these contexts_ . Experiments are
not the be all and end all (in "causal" statements or otherwise)
_in these complex human arenas_ ... they cannot possibly control
all relevant variables and randomly select / assign subjects
matched on relevant characteristics. Performance in a
controlled "laboratory" settings under lab conditions is _NOT_
the same as performance in "real-life"settings ... when
the context iteself is an important variables (i.e., as it is
in almost all educational endeavours). Experimental designs
cannot isolate human beings for the _length_ of time one
typically needs to observe developmental processes. You can
to some degree control and observe some sensation and perception
processes in a controlled lab setting but you cannot
experimentally control, measure and analyze the process whereby
the perceived information is interpreted and integrated into
broader thinking processes. Note that we also move into
the arena of longitudinal vs. single-measurement point ...
and causal-comparative designs are not _by definition_
limited to a single measurement point any more than
they are by definition forced into a single categorical/interval
variable.
REPLY BY BURKE JOHNSON:
YOU SAID THAT historically, educational variables could not be
manipulated, hence, the development of "causal comparative" research.
Aren't many IVs in education also quantitative and nonmanipulatable?
You also say that education has to deal with complex real life settings. I
wonder can't correlational research deal just as well with complex real
life settings as causal-comparative? Finally, correlational designs are not
limited to measurement at a single time point, just as C-C research is not
limited in this way. (You did agree with this last point later in this post) My
question is: why do we use textbooks again and again that state that
causal-comparative research can provide evidence about cause and
effect but correlational research cannot? Granted, the Gay textbook has
much going for it (including very a very enjoyable and lucid writing style),
but why did we allow it to get to the 5th edition without correcting the
statement that causal-comparative research can provide some evidence
about cause and effect but correlational research cannot????? Several
years ago, I sent my comments to Gay via the Merrill rep. (I identified the
rep and mailed my comments to him. He said she received the
comments.)
<snip example>
Norman David Giesbrecht also said...
What causal-comparative _attempts_ (or can attempt to do)
is sample from real-world domains and identify possible patterns
of influence (based on correlational-related techniques)
REPLY by Burke Johnson:
Can I conclude that you believe that causal-comparative research is a
correlational research method (as Slavin teaches)? If yes then can I also
conclude that the distinction between causal-comparative and
correlational research is artificial and misleading, at least in terms of
providing evidence about cause and effect (other things equal)? If yes,
then we agree don't we?
Norman David Giesbrecht said...
By the way, it should be noted that ANOVA and correlation are
simply two faces of the same general linear model. Similarly,
_any_ categorical variable can be turned into (n-1) binary
_interval_ level variables.
Burke Johnson's reply:
I TOTALLY GREE WITH YOU and made the same point in an earlier post.
Burke Johnson said:
> cross-sectional data. More importantly for our discussion, causal
> comparative and correlational designs can both be done using
> longitudinal or cross-sectional data.
Norman David Giesbrecht replied...
Agreed ... good point ... which helps highlight some other
important methodlogical concerns in making causal statements.
Burke Johnson said:
We agree again. Our differences are probably a lot smaller than may
seem to be the case.
Burke Johnson said in an earlier post:>
> in education (by L.R. Gay) does precisely this, and has done so for
over
> two decades now. How many students does that add up to having
> heard the statement "Causal comparative studies attempt to identify
> cause-effect relationships, correlational studies do not" (Gay, 4th ed,
> page 284).
Norman David Giesbrecht replied...
The key word is _attempt_ and perhaps it would be more accurate
to say that they attempt to find consistencies between observed
patterns and hypothesized causal relationships and then argue
that this suggests or supports the validity of the hypothesized
relationships.
Burke Johnson replies:
WHAT IS your point? Mine was that Gay clearly states (numerous times)
that causal-comparative attempts to demonstrate cause and effect but
correlational does not. My points are 1) experimental research is (ceteris
paribus) better than nonexperimental research, and 2) Ceteris paribus,
causal comparative research is neither better nor worse regarding
cause and effect than correlational research. I also pointed out that the
Gay textbook (that has sold thousands and thousands of copies) clearly
and directly says that greater evidence can be obtained from
causal-comparative than from correlational research. It just ain't so.
THANKS FOR THE DEBATE Norman and others...
If you have any more comments, I will find time to reply.
Sincerely,
Burke Johnson
____________________
Date: Tue, 10 Feb 1998 17:22:04 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain
It is popular in educational research books for the authors to state that
"correlation does not imply cause." Why don't we also emphasize:
"Neither does a difference between two or more groups in a
causal-comparative design imply cause." I find this second point just as
important as the first point. BTW, correlation or relationship is a
necessary but not sufficient condition for causation, right? As I tell my
students, "it ain't nearly enough but you gotta have it."
Cheers,
Burke Johnson
_____________________
Date: Tue, 10 Feb 1998 18:57:38 EST
From: Michael Scriven <Scriven@AOL.COM>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
In a message dated 2/10/98 4:26:44 PM, Burke Johnson wrote:
<< Why don't we also emphasize:
"Neither does a difference between two or more groups in a
causal-comparative design imply cause." I find this second point just as
important as the first point.>>
1. If assignment is random, and the groups large enough, the difference
virtually guarantees causation. Whereas correlation, no matter how perfect,
does not even make it likely.
You continue: >>BTW, correlation or relationship is a necessary but not
sufficient condition for causation, right? As I tell my students, "it ain't
nearly enough but you gotta have it."<<
2. Not at all; most causation is not associated with correlations. Heart
attacks are a common cause of death, but the correlation is low; same with
virtually all historical causation, clinical causation, etc.
Michael Scriven
_________________
Date: Tue, 10 Feb 1998 10:26:10 MST
From: Norman David Giesbrecht <ndgiesbr@ACS.UCALGARY.CA>
Subject: Re: Causal-Compartive vs. Correlational -Reply
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
>
>
> Recently "Burke Johnson" wrote ...
>
>
> --> Causal comparative also examines the relationships between
> variables. Doesn't the most simple form of a causal comparative examine
> the relationship between a single categorical independent variable and a
> single dependent variable?
Causal-comparative is _not_ (essentially) "a categorical independent
variable and a ... dependent variable". Scaling is a function of
the variable being measured - not the design. One could just as
easily (and inappropriately) say that an experimental design
in "the most simple form ... examine(s) the realtionship between
a single categorical independent variable and a single dependent
variable" (i.e., an ANOVA analysis). Variables that do not lend
themselves to ordered interval measurement (e.g., control vs treatment)
are by definition categorical- irregardless of the research design.
>
> -->many causal comparative studies may also "simply be empirical
> explorations with pragmatic ends"
>
As can _any_ research design (including experimental). A good
research design is more a function of the researcher than
the approach ... _however_, I would still maintain that
certain designs (in which I would include both experimental
and causal-comparative) can or should push a researcher
to greater methodological rigor.
>
> -->the name CAUSAL-Comparative is a misnomer. (In my earlier post I
> asked who introduced this term into education so that I could learn more
> about the history of the term. So far, no one seems to know. The term is
> not used outside of education.) Some of my students believe after
> reading their educational research methods books that a causal
> comparative design provides MORE evidence for cause and effect than
> correlational AND EXPERIMENTS.
Why should a nonexperimental method,
> that happens to have a categorical independent variable, have the word
> causal in it? Do I suddenly have a right to think causally if I categorize my
> quantitative independent variable? For example, do I have more basis for
> thinking causally if I examine high versus low motivation instead of a
> quantitatively scaled motivation variable?
A categorical independent variable doesn't make anything causal ...
and _no_ researcher would ever claim this. Scaling is a characteristic
of measurement and has _nothing_ to do with causal statement. No one
would disagree with your statement and no one makes that claim.
Any design that compares "groups" tends to have categorical variables
to describe the "groups" (including experimental) simply because
one doesn't intervally measure groups (e.g., control, treatment 1,
treatment 2, treatement 1+2)
>If any approach should have
> the term causal in it, it would be experimental.
I believe that here is where we start to get at the heart of the matter.
Perhaps the reason that causal-comparative is well-rooted in education
is linked to the inadeqacy of experimental designs in addressing
complex psychosocial processes. It is simply _not_ possible to
"control" the complex dynamics that are involved. Causal-comparative
is often more sensitive (or should be) to the multiple influences
than experimental designs _In these contexts_ . Experiments are
not the be all and end all (in "causal" statements or otherwise)
_in these complex human arenas_ ... they cannot possibly control
all relevant variables and randomly select / assign subjects
matched on relevant characteristics. Performance in a
controlled "laboratory" settings under lab conditions is _NOT_
the same as performance in "real-life"settings ... when
the context iteself is an important variables (i.e., as it is
in almost all educational endeavours). Experimental designs
cannot isolate human beings for the _length_ of time one
typically needs to observe developmental processes. You can
to some degree control and observe some sensation and perception
processes in a controlled lab setting but you cannot
experimentally control, measure and analyze the process whereby
the perceived information is interpreted and integrated into
broader thinking processes. Note that we also move into
the arena of longitudinal vs. single-measurement point ...
and causal-comparative designs are not _by definition_
limited to a single measurement point any more than
they are by definition forced into a single categorical/interval
variable.
For example, let's say we wanted to study people's responses
to Clinton's alleged activities. How could one possibly
design and implement an experimental design ? i.e., without
consciously eliminating important influences such as peer
discussions, family interactions, the changing media response,
etc. ... also, how could you possibly control / match / assign
individuals without eliminating or not considering the host
of psychological variables etc. that might influence their response.
What causal-comparative _attempts_ (or can attempt to do)
is sample from real-world domains and identify possible patterns
of influence (based on correlational-related techniques)
And in terms of causal
> comparative versus correlational, my previous posts should make it
> exceedingly clear that the scaling of the independent variable
> (categorical in C-C and quantitative in correlational) has NOTHING to do
> with one's ability to make cause and effect statements. Do you disagree
> with this statement?
No one makes this claim ... it is "self-evident".
>
> -->regarding your points about measurement: they apply equally to
> quantitative variables. For example, self-esteem also reflects history,
> personality, and even genetic factors. Statements about cause and
> effect are not contingent upon the scaling of the independent variable,
> and, hence, there is no necessary superiority of causal-comparative
> designs over correlational designs.
My examples included both categorical (gender) and quantitative (age)
variables ... I agree with your example re: self-esteem.
By the way, it should be noted that ANOVA and correlation are
simply two faces of the same general linear model. Similarly,
_any_ categorical variable can be turned into (n-1) binary
_interval_ level variables.
> cross-sectional data. More importantly for our discussion, causal
> comparative and correlational designs can both be done using
> longitudinal or cross-sectional data.
Agreed ... good point ... which helps highlight some other
important methodlogical concerns in making causal statements.
>
> in education (by L.R. Gay) does precisely this, and has done so for over
> two decades now. How many students does that add up to having
> heard the statement "Causal comparative studies attempt to identify
> cause-effect relationships, correlational studies do not" (Gay, 4th ed,
> page 284).
The key word is _attempt_ and perhaps it would be more accurate
to say that they attempt to find consistencies between observed
patterns and hypothesized causal relationships and then argue
that this suggests or supports the validity of the hypothesized
relationships.
_____________________
Date: Tue, 10 Feb 1998 23:21:00 -0500
From: "Bryan W. Griffin" <bwgriffin@GSVMS2.CC.GASOU.EDU>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
>Michael, thank you for an elegantly succinct and cogent argument! 'Nuff
>said!
I disagree; in fact, I think the two statements introduce problems that
must be addressed.
>
>Bill Castine
>
>
>Michael Scriven wrote:
>
>> 1. If assignment is random, and the groups large enough, the
>> difference
>> virtually guarantees causation. Whereas correlation, no matter how
>> perfect,
>> does not even make it likely.
The difference guarantees causation only if you assume nothing else
differed between the two groups except for the experimental manipulation (a
tenuous assumption no matter how well controlled, at least in education).
I don't understand your use of the word correlation. Are you saying that
calculating a correlation for the group difference you mention above does
not imply causation, but using a t-test or ANOVA does? Or are you using the
term correlation synonymously (and incorrectly, I think) with the notion of
non-experimental research?
I think this type of loose language is exactly at the heart of the matter
that Burke has highlighted. The statistical procedure one uses is
irrelevant for determining causation -- keep in mind that the results of a
two group t-test or ANOVA can very easily be converted to a correlation, so
it should be obvious that reporting r = .85 (p < .05) can provide as much
information about causation as reporting t = 3.00 (p < .05) or F = 9.00 (p
< .05).
Yes, if one uses the term correlation to refer to all non-experimental
research, then correlation does not imply causation. However, since
correlation coefficients (and multiple regression, path analysis, etc.) can
be used to analyze experimental data, I think this double usage of the term
--
correlation = a type of research (i.e., non-experimental)
correlation = a set of statistical analysis procedures
confuses the issue.
For clarity of discussion, we should not confuse correlation with
non-experimental research. I think it would be much clearer to eliminate
the terms correlational research and causal-comparative research and use
instead non-experimental research or ex post facto research (as a number of
authors have already done, e.g., Kerlinger, Pedhazur and Schmelkin).
In your statement-- "Whereas correlation, no matter how perfect,
does not even make it likely" -- seems to imply that you are referring to
the statistical procedure since you refer to the size of the coefficient
("no matter how perfect"), so you appear to be confusing the statistical
analysis procedure with the research design. If the data were collected
from an experiment, then the correlation does, in fact, imply causation
just as much as the group difference expressed by a t-test or ANOVA.
>> You continue: >>BTW, correlation or relationship is a necessary but
>> not
>> sufficient condition for causation, right? As I tell my students, "it
>> ain't
>> nearly enough but you gotta have it."<<
>>
>> 2. Not at all; most causation is not associated with correlations.
>> Heart
>> attacks are a common cause of death, but the correlation is low; same
>> with
>> virtually all historical causation, clinical causation, etc.
>>
>> Michael Scriven
Again, your response to Burke appears problematic. According to Kerlinger,
Pedhazur and Schmelkin, and every other text I've ever read, to show
causation one must show a relationship. So even with heart attacks, the
association with death is low, but it is clearly present.
How one chooses to present that relationship in statistical form -- such as
with correlation coefficients, means and t or F ratios, chi-squares,
odds-ratios, etc.
-- is irrelevant to showing causation. That "most causation is not
associated with correlations" is simply a matter of choice or ignorance.
Cohen and Cohen, in their text on multiple correlations/regression, show
that anything presented with means and F-ratios can also be presented with
correlations, partial correlations, and part correlations. In fact,
correlations may actually be a better presentation form since they are a
standardized form and hence are effect sizes, something that is gaining in
importance thanks to a growing interest in meta-analysis.
___________________________________________________________________
Bryan W. Griffin
Phone: 912-681-0488
E-Mail: bwgriffin@gsvms2.cc.gasou.edu
WWW: http://www2.gasou.edu/edufound/bwgriffin/bgriffin.htm
__________________
Date: Wed, 11 Feb 1998 09:24:41 -0500
From: Gunapala Edirisooriya <edirig@ACCESS.ETSU.EDU>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7bit
I have been following this discussion with keen interest. Please let me
make a couple of comments:
Why do we talk about correlation and cuasation in terms of statistical
techniques, research design, and scales of measurement?
If we know X ===> Y (X causes Y), use any statistical technique
(regression, correlation, ANOVA, ...); use any design (experimental,
quasi-experimental, time-series, ...); use any scale (nominal, interval,
...). Any of these will still show us X ===> Y. Does any of these show
us a relationship between X and Y? Of course, they do. If the theory
says, the existing body of knowledge says that X ==> Y, then use any
statistical technique, design, or scale, you do not miss causation.
If we do not know whether X ==> Y or Y ==> X, then we cannot speak of
causation. But, the data show us a relationship between X and Y and Y
and X (X <==> Y). In the positivistic tradition, (true) experiments can
be used to find causation. OK, but we know we can use not only ANOVA,
but regression, correlation, etc with such data. It does not disprove
causation.
In causal-comparative (CC), we cannot experiment. So, we take two
samples one variable (factor) absent from one sample, ceteris paribus,
and the effect on the criterion or dependent or outcome variable is
highly correlated with the factor. Do an ANOVA, correlation, regression;
that relationship will be there. Call it causation, correlation,
ANOVAtion, whatever. What is important is that whatever we call it, it
MUST make sense. Nonsensical causation, nonsensical correlation simply
makes no sense.
If we use double arrow (<==>) to mean correlation (I think it makes
sense), then there is no point in talking about heart disease and death
in terms of X <==> Y. Because there is a relationship between heart
disease and death and NOT vice versa. If we cannot figure out which
causes which, then call it a relationship (correlation).
It is NOT the statistical technique that determies whether X ==> y or Y
==> X. It depends on the the best understanding of the existing body of
knowledge on the subject matter one is investigating. Or, common sense.
Let us hear more about on this topic. Gunapala.
----------------------------------------------------------------------
Burke Johnson wrote:
>
> It is popular in educational research books for the authors to state that
> "correlation does not imply cause." Why don't we also emphasize:
> "Neither does a difference between two or more groups in a
> causal-comparative design imply cause." I find this second point just as
> important as the first point. BTW, correlation or relationship is a
> necessary but not sufficient condition for causation, right? As I tell my
> students, "it ain't nearly enough but you gotta have it."
>
> Cheers,
> Burke Johnson
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
2/11/98
Date: Wed, 11 Feb 1998 08:59:52 -0800
From: dennis roberts <dmr@PSU.EDU>
Subject: Re: eliminating items
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
len says:
>I would imagine that most of us do not specify how examinations will be
>graded in our syllabi. Usually, students are just interested in what
>proportion of the final grade will be determined by each grade producing
>activity during the semester.
i would not be so sure about the above .. usually ... this is about ALL we
give them though .... even if that much. in general ... i find the
information that instructors give to students about testing/grading to be
rather ambiguous ... and they do it that way deliberately i am sure ...
but, i would suggest that is not being very open/honest/fair with students.
i think students have a right to know HOW they will be graded and by what
STANDARDS. is this so difficult for an instrutor to figure out and convey?
>
>Any test scoring strategy that can be reasonably theoretically or
>empirically defended on the basis of reasonable professional practice must
>be defended on the basis of maintaining academic and professional freedom
>at institutions of eduation.
i would suggest that there is no good overall definition of "best practice"
... but, in the case of class tests ... that are supposed to be directly
connected to course content ... there is an overriding concern ... and that
the content ON the test items would have to be defended FIRST and ABOVE
ANYTHING ELSE. for example ... if the students point out (and are correct)
that an item on the test covers material not specifically covered in the
course/class ... or told by the instructor that would be on the test ...
then the students have a legitimate complaint ... and such an item should
be tossed out ... EVEN IF SOMEONE IN THE CLASS HAPPENED TO KNOW THE ANSWER.
items that turn out to be very ambiguous ... such that all kinds of
students misinterpret (legitimately from the rhetoric of the item) WHAT is
being asked ... then perhaps that might be a candidate for exclusion ...
but, i would suggest that using item analysis data .. the difficult and/or
disc ... by itself ... with no reference to some serious content flaw ...
is NOT sufficient evidence on which to make a decision to toss out an item.
>>And the people who determine the
>reasonablness of a scoring practice are the professionals within the
>discipline, not a bunch of lawyers and administrators!
>
i agree in general with the above ... but, i would point out that
capricious elimination of items on tests ... after the fact ... if it can
be demonstrated have some negative impact on one or more examinees .. could
be an instance that brings in those "lawyers and administrators" ... so if
we engage in this practice ... it should be for a DEFENSIBLE reason ...
>Len
>
>Leonard B. Bliss, Ph.D.
>Professor, Department of Leadership and Educational Studies
>Reich College of Education
>Appalachian State University
>Boone, NC 28608 USA
>blisslb@conrad.appstate.edu
>
>
Dennis Roberts, Penn State University, 208 Cedar Bldg., University Park, PA
16802
Email: dmr@psu.edu ... Phone: AC 814-863-2401 .. FAX: AC 814-863-1002
WWW: http://www2.ed.psu.edu/espse/staff/droberts/drober~1.htm
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 09:04:54 -0800
From: dennis roberts <dmr@PSU.EDU>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
actually ... what determines whether we can LOGICALLY AND REASONABLY
attribute cause of X to the outcome of Y ... is a METHOD that we use in the
collecting of data on X and Y ... what CONTROL we have over WHO gets what
level of X ...
i am not so sure that understanding of the "existing body of knowledge ...
" is the key ... though that could certainly help in being able to week out
extraneous noise from the causative chain you are trying to establish ...
>
>It is NOT the statistical technique that determies whether X ==> y or Y
>==> X. It depends on the the best understanding of the existing body of
>knowledge on the subject matter one is investigating. Or, common sense.
>
>Let us hear more about on this topic. Gunapala.
>
>----------------------------------------------------------------------
>
>Burke Johnson wrote:
>>
>> It is popular in educational research books for the authors to state that
>> "correlation does not imply cause." Why don't we also emphasize:
>> "Neither does a difference between two or more groups in a
>> causal-comparative design imply cause." I find this second point just as
>> important as the first point. BTW, correlation or relationship is a
>> necessary but not sufficient condition for causation, right? As I tell my
>> students, "it ain't nearly enough but you gotta have it."
>>
>> Cheers,
>> Burke Johnson
>
>
Dennis Roberts, Penn State University, 208 Cedar Bldg., University Park, PA
16802
Email: dmr@psu.edu ... Phone: AC 814-863-2401 .. FAX: AC 814-863-1002
WWW: http://www2.ed.psu.edu/espse/staff/droberts/drober~1.htm
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 09:18:00 CDT
From: Robert Arden Terry <rterry@OUPSY.PSY.OU.EDU>
Subject: Re: eliminating items
> but ... even if it is announced ... it still puts examinees at a
> disadvantage ... in the sense that they NEVER know if the items they will
> be spending time on .. will count. besides .. what is a bad item anyway ...
> ? until you discuss what that is ... what the parameters are for being
> called, after the fact, a bod item .. i don't think the instructor has
> adequately done his/her job. besides ... most faculty don't really
> understand item analysis statistics ... and would just use some arbitrary
> rule ... when the rule really has no SOUND psychometric basis ...
>
It is true that the very nature of taking a test can change the
results in ways unintended. Perhaps the inclusion of bad items may
have effects that are difficult to measure. If I understand Dennis
correctly, perhaps what is needed is to make some mechanism
available to the students that allows them an opportunity to deal
with such difficult questions.
One possibility is to allow students, at their discretion, to
"give" short-answers to MC questions that they feel cannot be
answered without further elaboration. This allows the good students
a chance to take poorly worded or vague questions and clarify their
response. I usually limit this to two questions, on the assumption
that my tests would rarely have more than 2 items which had "bad"
item statistics :-).
What I have found is that the good students are usually the ones
who identify the "bad" items by choosing to supplement their MC
response with a short-answer. In essence, there is usually no need
to throw out the item, because the students themselves are making the
item better by recognizing the limitation of the item itself, and
modifying it's effect by choosing to answer the question via
short-answer.
Then, on the next administration, one could either a) fix the
item, or b) get rid of it altogether.
Cheers,
Robert
Quantitative Pychology
University of Oklahoma
Norman, OK 73019
Ph: (405)-325-4593
Fax: (405)-325-4737
E-Mail: rterry@ou.edu
"A man's got to know his limitations" Clint Eastwood
as Dirty Harry in Magnum Force
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 08:39:04 -0700
From: "Kevin F. Spratt" <Kevin-Spratt@UIOWA.EDU>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
At 05:22 PM 2/10/98 -0600, you wrote:
>It is popular in educational research books for the authors to state that
>"correlation does not imply cause." Why don't we also emphasize:
>"Neither does a difference between two or more groups in a
>causal-comparative design imply cause." I find this second point just as
>important as the first point. BTW, correlation or relationship is a
>necessary but not sufficient condition for causation, right? As I tell my
>students, "it ain't nearly enough but you gotta have it."
>
>Cheers,
>Burke Johnson
>
>
Correlation is *not* a necessary condition for causation. Causation need
not be a linear phenomenon.
Another scenario where linear causation is involved, but where a zero order
correlation might not reflect this can occur when the nature of that linear
causation is dependent on another factor. For example, suppose x and a are
causally related in a linear fashion, and that rxa is positive for males
and rxa is negative for females: rxa could be minimal when computed across
a set including males and females.
___________________________________________________________
Kevin F. Spratt
Iowa Testing Programs &
Spine Diagnostic & Treatment Center
224-D Lindquist Center
University of Iowa
Iowa City, Iowa 52242
(319) 335-5572 (voice)
(319) 335-6399 (fax)
Kevin-Spratt@Uiowa.edu (e-mail)
_____________________
Date: Wed, 11 Feb 1998 10:56:12 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Causal comparative and cause -Reply
MIME-version: 1.0
Content-type: text/plain
My comments follow Michael's...
Michael Scriven writes: <Scriven@AOL.COM> 02/10/98 05:57pm >>>
In a message dated 2/10/98 4:26:44 PM, Burke Johnson wrote:
<< Why don't we also emphasize:
"Neither does a difference between two or more groups in a
causal-comparative design imply cause." I find this second point just as
important as the first point.>>
1. If assignment is random, and the groups large enough, the difference
virtually guarantees causation. Whereas correlation, no matter how
perfect,
does not even make it likely.
Burke Johnson's reply
-->Michael, You missed the point. Sorry if I did not make the point
clearly... The point was, WITHOUT any control or random assignment or
manipulation, etc. one can no more attribute cause from a difference
between two groups on some variable than one can from a bivariate
correlation. Whether one does an independent t-test or a bivariate
correlation has no bearing on statements of causality. I was comparing
the simple case of what educational research methods books call
causal-comparative research and correlational research. The only
difference between the simple case (i.e., the case to which I was
referring) of a causal-comparative design and the simple case of a
correlational design is the scaling of the independent variable. And in
both simple cases, one should not jump to causation. Obviously, in none
of these cases can one jump to cause. More investigative work is
needed...
It should also be interesting to note that one can move from the simple
case of correlational research to the simple case of causal-comparative
research simply by categorizing one's independent variable. In this case
one ends up with an ordinal variable (e.g., high IQ, medium IQ, and low
IQ). This process of collapsing a quantitative variable into a "categorical"
variable was common practice during the heyday of ANOVA. BTW, I am
not saying that one should NEVER collapse a quantitative variable into a
"categorical" variable...I'm just saying that I see no difference between
the SIMPLE CASES of C-C and CORR with regard to judgements of
causality.
Michael also says:
2. Not at all; most causation is not associated with correlations. Heart
attacks are a common cause of death, but the correlation is low; same
with virtually all historical causation, clinical causation, etc.
My reply is:
Most group differences are, likewise, not causal. The point is that we
need MORE than to observe group differences on some variable OR
observe a simple correlation to make a causal attribution. Don't you
agree with this statement?
Burke Johnson
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 16:53:29 0000
From: H F TANNER <H.F.Tanner@SWANSEA.AC.UK>
Subject: Re: Performance assessment and student evaluation
> The problem is this; it is not all that uncommon to receive a
> score on the appeal process that is one point off. When I read the
> comments on the student evaluations from last semester, two students
> who had appealed suggested that because we didn't demonstrate that
> the grade obtained was exactly the same.
> Despite the fact that we covered reliability, a number of
> students complained because we couldn't "consistently" grade the
> papers.
> To me, this is a real problem with using performance assessments,
> You have to admit that although independent judges can give quite
> similar grades, empirically they will NEVER be the same.
> On the other hand, even though we as measurement experts "know"
> that the same holds true for MC exams (when considered as a sample
> from some domain), the students DON"T KNOW THIS. To them, they
> either missed the item or got it right, and they don't question the
> reliability of the MC exam in the same way they question the essay
> or paper grade.
> Since student evalutions matter (another story :-)), it is
> probably in my best interest not to use these kinds of assessments
> since a) 20% of the class do not like them, and b) a substantial
> number thinks they can't be reliably assessed.
>
> Cheers,
>
> Robert
>
> Quantitative Pychology
> University of Oklahoma
> Norman, OK 73019
>
> Ph: (405)-325-4593
> Fax: (405)-325-4737
> E-Mail: rterry@ou.edu
>
>
> "A man's got to know his limitations" Clint Eastwood
> as Dirty Harry in Magnum Force
>
Howard Tanner
University of Wales Swansea
Department of Education
Hendrefoelan
Swansea SA2 7NB
tel: (+44) (0)1792 518642
fax: (+44) (0)1792 290219
"Life is what happens to you
while you're busy making other plans"
John Lennon
Why not visit my homepage on:
http://www.swan.ac.uk/education/waw/Howard.htm
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 11:46:49 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Causal comparative and cause -Reply
MIME-version: 1.0
Content-type: text/plain
Burke Johnson said:
<snip>
> BTW, correlation or relationship is a
>necessary but not sufficient condition for causation, right? As I tell my
>students, "it ain't nearly enough but you gotta have it."
Kevin F. Spratt replied:
Correlation is *not* a necessary condition for causation. Causation need
not be a linear phenomenon.
My reply is: I think you misread my statement. . . The statement says
"correlation OR relationship." Certainly a relationship may be curvilinear.
Hence, I totally agree with your statements about curvilinear
relationships...
Burke
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 11:50:48 -0600
From: Betsy Bizot <EBizot@BALLFOUNDATION.ORG>
Subject: Re: eliminating items
MIME-version: 1.0
Content-type: text/plain
> One possibility is to allow students, at their discretion, to
>"give" short-answers to MC questions that they feel cannot be
>answered without further elaboration. This allows the good students
>a chance to take poorly worded or vague questions and clarify their
>response. I usually limit this to two questions, on the assumption
>that my tests would rarely have more than 2 items which had "bad"
>item statistics :-).
This sounds good to me too. Reminiscient of the usual quantitative vs. qualitative discussions, the item statistics will tell you only that *something* is wrong with the item. Without this kind of qualitative feedback, you have to rely on your ability t
o second-guess yourself to figure out what in particular is wrong.
It also helps resolve the dilemma about credit/no credit for a problematic item. Anyone who (a) gets the original, intended answer with no comment, or (b) provides a convincing rationale for his/her choice of answer, can be given credit
Betsy
Elizabeth B. Bizot, Ph.D.
Project Manager
====================================
The Ball Foundation
Discovering and Developing Human Potential
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 12:08:44 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Causal comparative and cause -Reply
MIME-version: 1.0
Content-type: text/plain
Bill Castine said in reply to Michael Scriven's reply:
>>> Bill Castine <wcastine@FAMU.EDU> 02/10/98 09:38pm >>>
Michael, thank you for an elegantly succinct and cogent argument! 'Nuff
said!
Bill Castine
My reply is:
Let's try to debate the issues with evidence, Bill, no via an appeal to
authority...Furthermore, let's examine Michael's reply to my reply. He
made some assumptions that were clearly not intended in my original
message, and since you have been following this debate I'm quite sure
that you are well aware of those assumptions... I think it is quite unlikely
that Michael and I will end up disagreeing... The fact is that one's
evidence for cause is no greater when comparing two groups (without
any manipulation, control, random assignment, etc.) than it is when
examining a bivariate correlation coefficient...I look forward to your
future replies based on good solid logical argument...
In the spirit of open debate,
Burke
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 13:41:41 -0500
From: "William W. Pendleton" <socwwp@EMORY.EDU>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: TEXT/PLAIN; charset=US-ASCII
On Wed, 11 Feb 1998, Kevin F. Spratt wrote:
> Correlation is *not* a necessary condition for causation. Causation need
> not be a linear phenomenon.
> Another scenario where linear causation is involved, but where a zero order
> correlation might not reflect this can occur when the nature of that linear
> causation is dependent on another factor. For example, suppose x and a are
> causally related in a linear fashion, and that rxa is positive for males
> and rxa is negative for females: rxa could be minimal when computed across
> a set including males and females.
>
A great deal of confusion could be eliminated if authors of textbooks
would resist the temptation of creating new labels where none are needed
and distinguish carefully between the analysis of the data and
what that analysis means with respect to some theory. Unfortunately, that
task seems to confound some writers. With respect to the reply quoted
above, I suggest we use association to refer to the relation between
variables in general and specify which correlation measure we have in
mind if we use that term,i.e. Pearson product moment correlation
coefficient, Spearman's, Kendall's, etc. By the way, in reporting
experimental or ''ex post facto esperimental results" eta-squared has
much more meaning than F or t. I wonder where so many students get the
idea that knowing that an association (difference) is likely real is more
important than knowing how great it is.
Wm W. Pendleton
Department of Sociology
Emory University
Atlanta, Ga. 30322
socwwp@emory.edu
404 727-7524
_______________________
Date: Wed, 11 Feb 1998 14:18:56 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Professor Johnson -Reply
MIME-version: 1.0
Content-type: text/plain
Here is Jack Fraenkel's distinction between causal-comparative and
correlational research. As he points out in his message, his book (How
to Design and Evaluate Research in Education) is now the leading selling
research methods book in education. As you can see, he gave
permission for me to post his statement to this discussion group.
Burke J.
>>> jack Fraenkel <jrf@sfsu.edu> 02/11/98 12:30pm >>>
>Jack,
>There is a discussion about causal-comparative versus correlational
>research going on the AERA-D listserv. Do I have your permission to
>post your reply to me so that they can see how you explain the
>difference?
>Thanks
>Burke J.
>
>>>> jack Fraenkel <jrf@sfsu.edu> 02/11/98 12:06pm >>>
>Dear Professor Johnson,
> Thanks for your e-mail. Here are some answers to your
>questions:
>
>>1. Do you know who coined the term "causal comparative"
>>research? (I have never seen the term outside of education.) When
>was
>>it coined?-- REGRETFULLY, I DO NOT, ALTHOUGH IT IS USED
>COMMONLY IN
>>EDUCATIONAL RESEARCH. PERHAPS YOU KNOW IT BY ITS'
>SYNONYM "EX-POST-FACTO"
>>RESEARCH.
>>
>>2. Why do educational researchers, such as these textbook authors,
>>seem to believe that evidence about cause and effect will be any
>>stronger in causal comparative research than in correlational
research?
>>
> The reason is as follows: CAUSAL-COMPARATIVE RESEARCH
>involves
>COMPARING (thus the "COMPARATIVE" aspect) TWO groups in order
to
>explain
>existing differences between them on some variable or variables of
>interest. The only difference between CC and experimental research is
>that
>the groups being compared in CC research have ALREADY been
>formed, and any
>treatment (if there was a treatment) has ALREADY been applied. Of
>necessity, the researcher must examine the RECORDS of the two
>groups to see
>if he or she can offer a reasonable explanation (i.e., what "CAUSED")
>the
>existing differences between the two groups. (Example: Seeking to
>explain
>why some automobile drivers have three or more accidents in a year
>while
>apparently other, similar drivers have none. The researcher would try
to
>explain this difference by examining their characteristics as described
in
>their automobile insurance company records. If the researcher can
>identify
>a reasonable explanation for the difference (i.e., the difference in the
>number of accidents) and eliminate other, alternative explanations, he or
>she can reasonbly conclude that he or she has identified the CAUSE of
>the
>difference in accidents.
> CORRELATIONAL RESEARCH, on the other hand, does not NOT
>look at
>differences differences between groups. Rather, it looks for
>relationships
>within a SINGLE group. THIS IS A BIG DIFFERENCE! In doing correlational
>research,a researcher obtains data from two instruments for each
>member of
>the SINGLE group being studied.Should one find a relationship (e.g.,
that
>scores on variable A increase along with scores on variable B, or
>vice-versa), one is only entitled to conclude that a relationship of some
>sort exists, NOT that variable A caused some variation in variable B, or
>vice-versa. (Some third variable(s) might be the causal factor.
> In sum, CC research does allow one to make reasonable
>inferences
>about causation; correlational research does not. (Correlational
research
>does often suggest follow-up experimental research, however)
> Hope this helps.
>
>BRUCE,My rep tells me that our book now outsells Gay, and we are #1
in
>the
>country.
> Anyway, thanks for your inquiry.
>Best wishes,
> Jack Fraenkel
Bruce,
Yes, you may.
Cordially,
JRF
--Boundary_(ID_pgwaVpwgA7GNXKYcHW44MQ)
Date: Wed, 11 Feb 1998 15:47:35 EST
From: Michael Scriven <Scriven@AOL.COM>
Subject: Re: Causal comparative and cause -Reply
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
In a message dated 2/11/98 10:57:51 AM, Burke Johnston wrote:
<<we need MORE than to observe group differences on some variable OR observe a
simple correlation to make a causal attribution. Don't you agree with this
statement?>>
Sure; but your original remark referred to group differences "in a causal-
comparative design" and a design would hardly qualify as causal-comparative if
there was no manipulation OR 'natural experiment' variation of the independent
variable, I would think. And if that's present, then the inference to
causation is highly plausible.
Michael Scriven
_____________________
Date: Wed, 11 Feb 1998 20:27:55 -0500
From:
"Bryan W. Griffin (by way of \"Bryan W. Griffin\" <bwgriffin@gsvms2.cc.gasou.edu> by way of \"Bryan
W. Griffin\" <bwgriffin@gsvms2.cc.gasou.edu>)" <bwgriffin@GSVMS2.CC.GASOU.EDU>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Sorry to send this message again, but someone stated that only part of the
message got through, so here is the complete form (I hope).
I welcome, and encourage, comments.
**************************************
>Bill Castine wrote:
>
>Michael, thank you for an elegantly succinct and cogent argument! 'Nuff
>said!
I disagree; in fact, I think the two statements introduce problems that
must be addressed.
>Michael Scriven wrote:
>
>> 1. If assignment is random, and the groups large enough, the
>> difference
>> virtually guarantees causation. Whereas correlation, no matter how
>> perfect,
>> does not even make it likely.
The difference guarantees causation only if you assume nothing else
differed between the two groups except for the experimental manipulation (a
tenuous assumption no matter how well controlled, at least in education).
I don't understand your use of the word correlation. Are you saying that
calculating a correlation for the group difference you mention above does
not imply causation, but using a t-test or ANOVA does? Or are you using the
term correlation synonymously (and incorrectly, I think) with the notion of
non-experimental research?
I think this type of loose language is exactly at the heart of the matter
that Burke has highlighted. The statistical procedure one uses is
irrelevant for determining causation -- keep in mind that the results of a
two group t-test or ANOVA can very easily be converted to a correlation, so
it should be obvious that reporting r = .85 (p < .05) can provide as much
information about causation as reporting t = 3.00 (p < .05) or F = 9.00 (p
< .05).
Yes, if one uses the term correlation to refer to all non-experimental
research, then correlation does not imply causation. However, since
correlation coefficients (and multiple regression, path analysis, etc.) can
be used to analyze experimental data, I think this double usage of the term
--
correlation = a type of research (i.e., non-experimental)
correlation = a set of statistical analysis procedures
confuses the issue.
For clarity of discussion, we should not confuse correlation with
non-experimental research. I think it would be much clearer to eliminate
the terms correlational research and causal-comparative research and use
instead non-experimental research or ex post facto research (as a number of
authors have already done, e.g., Kerlinger, Pedhazur and Schmelkin).
In your statement-- "Whereas correlation, no matter how perfect,
does not even make it likely" -- seems to imply that you are referring to
the statistical procedure since you refer to the size of the coefficient
("no matter how perfect"), so you appear to be confusing the statistical
analysis procedure with the research design. If the data were collected
from an experiment, then the correlation does, in fact, imply causation
just as much as the group difference expressed by a t-test or ANOVA.
>> You continue: >>BTW, correlation or relationship is a necessary but
>> not
>> sufficient condition for causation, right? As I tell my students, "it
>> ain't
>> nearly enough but you gotta have it."<<
>>
>> 2. Not at all; most causation is not associated with correlations.
>> Heart
>> attacks are a common cause of death, but the correlation is low; same
>> with
>> virtually all historical causation, clinical causation, etc.
>>
>> Michael Scriven
Again, your response to Burke appears problematic. According to Kerlinger,
Pedhazur and Schmelkin, and every other text I've ever read, to show
causation one must show a relationship. So even with heart attacks, the
association with death is low, but it is clearly present.
How one chooses to present that relationship in statistical form -- such as
with correlation coefficients, means and t or F ratios, chi-squares,
odds-ratios, etc.
-- is irrelevant to showing causation. That "most causation is not
associated with correlations" is simply a matter of choice or ignorance.
Cohen and Cohen, in their text on multiple correlations/regression, show
that anything presented with means and F-ratios can also be presented with
correlations, partial correlations, and part correlations. In fact,
correlations may actually be a better presentation form since they are a
standardized form and hence are effect sizes, something that is gaining in
importance thanks to a growing interest in meta-analysis.
___________________________________________________________________
Bryan W. Griffin
Phone: 912-681-0488
E-Mail: bwgriffin@gsvms2.cc.gasou.edu
WWW: http://www2.gasou.edu/edufound/bwgriffin/bgriffin.htm
___________________
Date: Wed, 11 Feb 1998 20:57:57 -0500
From: "William W. Pendleton" <socwwp@EMORY.EDU>
Subject: Re: Professor Johnson -Reply
MIME-version: 1.0
Content-type: TEXT/PLAIN; charset=US-ASCII
I have left below professor Hohnsons message which includes the comments
of Fraenkel. Here the problem is posed in a way that demonstrates the
error of the formulation which seems to be in the best selling book in
the field as well as others. Consider teh case (a simple one) were a
researcher wishes to see if studying causes higher grades on a test.
Three possibilities for making some inference about that possibility
illustrate the point. Students may be separated into two groups on the
basis of how much they study and the means of their test scores
compared using a statistical test such as Student's t. The groups may
be scored 0 ond 1 (dummy variable) as the Pearson product
moment correlation coefficient may be tested. The results will be
essentially the same. (Recalling that t squared with k degree of freedom
is F with k and 1 degree of freedom and the 1-r squared represents
unexplained variation as the within group sum of squares reflects
the unexplained variation in the comparison of means may make that
clearer). There is no basis for claiming that the first gives better
support to a causal claim than the second. A third possibility would be
to measure the time studying and correloate it with the score on the test
using the PPMCC and that linaer effect is sufficiently great, infer a
(possible causal effect) None of these is a strong basis for infering
cause, but the third would have the advantage of showing that more of the
presumed cause gives more of the presumed effect which seems slightly
stronger evidence. Of course in any ex post facto design, infer cause is
tricky because relevant factors have not been controlled by randomization.
Correlation and associated regression analysis allow for easier control by
statistical means that do classification based methods,
hence correlation has an advantage in the practical sense,
but in either case, inferring cause rests on confidence in the theory
being used, especially the identification of the etiological mechanisms assumed
to be at work and the temporal sequence. Having groups that look like a
random experiment when in fact they are not gives no basis for being more
confident about cause than having measured variables. It is a great
disservice to researchers in training to direct them toward such designs
on that basis, especially when very powerful statistical controls are
easily available in regression, correlation, structural equations, and
hierarchical linear models. Teachers and textbook writers and those who
edit and select textbooks should think again about such contentions.
I think most of this has been said by others and said better, but I could
not let what seems to me be utter rubbish stand as the last word.
On Wed, 11 Feb 1998, Burke Johnson wrote:
> Here is Jack Fraenkel's distinction between causal-comparative and
> correlational research. As he points out in his message, his book (How
> to Design and Evaluate Research in Education) is now the leading selling
> research methods book in education. As you can see, he gave
> permission for me to post his statement to this discussion group.
> Burke J.
>
> >>> jack Fraenkel <jrf@sfsu.edu> 02/11/98 12:30pm >>>
> >Jack,
> >There is a discussion about causal-comparative versus correlational
> >research going on the AERA-D listserv. Do I have your permission to
> >post your reply to me so that they can see how you explain the
> >difference?
> >Thanks
> >Burke J.
> >
> >>>> jack Fraenkel <jrf@sfsu.edu> 02/11/98 12:06pm >>>
> >Dear Professor Johnson,
> > Thanks for your e-mail. Here are some answers to your
> >questions:
> >
> >>1. Do you know who coined the term "causal comparative"
> >>research? (I have never seen the term outside of education.) When
> >was
> >>it coined?-- REGRETFULLY, I DO NOT, ALTHOUGH IT IS USED
> >COMMONLY IN
> >>EDUCATIONAL RESEARCH. PERHAPS YOU KNOW IT BY ITS'
> >SYNONYM "EX-POST-FACTO"
> >>RESEARCH.
> >>
> >>2. Why do educational researchers, such as these textbook authors,
> >>seem to believe that evidence about cause and effect will be any
> >>stronger in causal comparative research than in correlational
> research?
> >>
> > The reason is as follows: CAUSAL-COMPARATIVE RESEARCH
> >involves
> >COMPARING (thus the "COMPARATIVE" aspect) TWO groups in order
> to
> >explain
> >existing differences between them on some variable or variables of
> >interest. The only difference between CC and experimental research is
> >that
> >the groups being compared in CC research have ALREADY been
> >formed, and any
> >treatment (if there was a treatment) has ALREADY been applied. Of
> >necessity, the researcher must examine the RECORDS of the two
> >groups to see
> >if he or she can offer a reasonable explanation (i.e., what "CAUSED")
> >the
> >existing differences between the two groups. (Example: Seeking to
> >explain
> >why some automobile drivers have three or more accidents in a year
> >while
> >apparently other, similar drivers have none. The researcher would try
> to
> >explain this difference by examining their characteristics as described
> in
> >their automobile insurance company records. If the researcher can
> >identify
> >a reasonable explanation for the difference (i.e., the difference in the
> >number of accidents) and eliminate other, alternative explanations, he or
> >she can reasonbly conclude that he or she has identified the CAUSE of
> >the
> >difference in accidents.
> > CORRELATIONAL RESEARCH, on the other hand, does not NOT
> >look at
> >differences differences between groups. Rather, it looks for
> >relationships
> >within a SINGLE group. THIS IS A BIG DIFFERENCE! In doing correlational
> >research,a researcher obtains data from two instruments for each
> >member of
> >the SINGLE group being studied.Should one find a relationship (e.g.,
> that
> >scores on variable A increase along with scores on variable B, or
> >vice-versa), one is only entitled to conclude that a relationship of some
> >sort exists, NOT that variable A caused some variation in variable B, or
> >vice-versa. (Some third variable(s) might be the causal factor.
> > In sum, CC research does allow one to make reasonable
> >inferences
> >about causation; correlational research does not. (Correlational
> research
> >does often suggest follow-up experimental research, however)
> > Hope this helps.
> >
> >BRUCE,My rep tells me that our book now outsells Gay, and we are #1
> in
> >the
> >country.
> > Anyway, thanks for your inquiry.
> >Best wishes,
> > Jack Fraenkel
>
> Bruce,
> Yes, you may.
> Cordially,
> JRF
>
Wm W. Pendleton
Department of Sociology
Emory University
Atlanta, Ga. 30322
socwwp@emory.edu
404 727-7524
--Boundary_(ID_C/5M99+ViuGHmsgbolz9Pg)
Date: Wed, 11 Feb 1998 20:52:05 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Causal comparative and cause -Reply
MIME-version: 1.0
Content-type: text/plain
Jack Fraenkel, the lead author of How to Design and Evaluate Research
in Education (a leading textbook in terms of sales) said he would like to
join this discussion. I told him how to get on the listserv and I have sent
him all previous messages (all causal comparative posts and all research
methods questions posts).This is good science isn't it...we are all
debating some important issues in our field.
Burke Johnson
--Boundary_(ID_C/5M99+ViuGHmsgbolz9Pg)
Date: Wed, 11 Feb 1998 22:29:57 -0500
From: "Bryan W. Griffin" <bwgriffin@GSVMS2.CC.GASOU.EDU>
Subject: Re: Professor Johnson -Reply
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
>I think most of this has been said by others and said better, but I could
>not let what seems to me be utter rubbish stand as the last word.
I agree with Dr. Pendleton's comments about Fraenkel's book. In fact, I've
made the exact same points several times in other posts.
Where is the idea that causal-comparative (ex post facto) studies provide
something different from correlational studies coming from? And why is this
myth so difficult to eliminate even from those of us who teach research
methods and statistics?
___________________________________________________________________
Bryan W. Griffin
E-Mail: bwgriffin@gsvms2.cc.gasou.edu
WWW Page: http://www2.gasou.edu/edufound/bwgriffin/bgriffin.htm
Phone: 912-681-0488
_____________________
2/12/98
Date: Thu, 12 Feb 1998 08:33:30 -0500
From: Gunapala Edirisooriya <edirig@ACCESS.ETSU.EDU>
Subject: Causal Comparative and ...
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7bit
Of course, I agree with the following statement of Professor William W.
Pendleton. Quite a number of us have been explaining this point. But,
the message does not seem to get through. Thanks. Gunapala.
----------------------------------------------------------------------
Here the problem is posed in a way that demonstrates the error of the
formulation which seems to be in the best selling book in
the field as well as others. Consider teh case (a simple one) were a
researcher wishes to see if studying causes higher grades on a test.
Three possibilities for making some inference about that possibility
illustrate the point. Students may be separated into two groups on the
basis of how much they study and the means of their test scores
compared using a statistical test such as Student's t. The groups may
be scored 0 ond 1 (dummy variable) as the Pearson product
moment correlation coefficient may be tested. The results will be
essentially the same. (Recalling that t squared with k degree of freedom
is F with k and 1 degree of freedom and the 1-r squared represents
unexplained variation as the within group sum of squares reflects
the unexplained variation in the comparison of means may make that
clearer). There is no basis for claiming that the first gives better
support to a causal claim than the second. A third possibility would be
to measure the time studying and correloate it with the score on the
test
using the PPMCC and that linaer effect is sufficiently great, infer a
(possible causal effect) None of these is a strong basis for infering
cause, but the third would have the advantage of showing that more of
the
presumed cause gives more of the presumed effect which seems slightly
stronger evidence. Of course in any ex post facto design, infer cause
is
tricky because relevant factors have not been controlled by
randomization.
Correlation and associated regression analysis allow for easier control
by
statistical means that do classification based methods,
hence correlation has an advantage in the practical sense,
but in either case, inferring cause rests on confidence in the theory
being used, especially the identification of the etiological mechanisms
assumed
to be at work and the temporal sequence. Having groups that look like a
random experiment when in fact they are not gives no basis for being
more
confident about cause than having measured variables. It is a great
disservice to researchers in training to direct them toward such designs
on that basis, especially when very powerful statistical controls are
easily available in regression, correlation, structural equations, and
hierarchical linear models. Teachers and textbook writers and those who
edit and select textbooks should think again about such contentions.
I think most of this has been said by others and said better, but I
could
not let what seems to me be utter rubbish stand as the last word.
_______________________
Date: Thu, 12 Feb 1998 21:58:19 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Causal comparative and cause -Reply -Reply
MIME-version: 1.0
Content-type: text/plain
I looked at Kevin Spratt's comment to my statement that "Correlation or
relationship is a necessary but not sufficient condition for causality" and I
noticed that he made a good point that I missed.
He said " Another scenario where linear causation is involved, but
where a zero order correlation might not reflect this can occur when the
nature of that linearcausation is dependent on another factor. For
example, suppose x and a are causally related in a linear fashion, and
that rxa is positive for malesand rxa is negative for females: rxa could
be minimal when computed across a set including males and females."
My reply is that I agree that we can be misled by examining the
relationship between two variables using statistical tools like One-Way
ANOVA or a bivariate correlation. In Kevin's case, we would miss
causation altogether even if we did a single factor EXPERIMENT WITH
RANDOM ASSIGNMENT! Statistically speaking, our model would have
been misspecified because we excluded the moderator variable gender.
Kevin's relationship would have only showed up in the interaction effect
in a model that included gender.
THE QUESTION IS SHOULD WE DROP THE FIRST REQUIREMENT IN THE
POPULAR "THREE CRITERIA FOR CAUSATION" HEURISTIC?
1. There must be a relationship or association or correlation,
2. There must be time ordering of the variables, i.e., if a causes b then a
must come before b
3. And, alternative hypotheses must be ruled out, i.e., the third variable
problem.
OR
SHOULD WE CALL THE RELATIONSHIP that Kevin pointed out an
"interactive relationship" and keep the statement that there must be a
relationship (of some kind) in order to have causation??
IF YOU THINK we should keep criterion one, how would you word it?
Regards,
Burke
>>> Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU> 02/11/98
11:46am >>>
Burke Johnson said:
<snip>
> BTW, correlation or relationship is a
>necessary but not sufficient condition for causation, right? As I tell my
>students, "it ain't nearly enough but you gotta have it."
Kevin F. Spratt replied:
Correlation is *not* a necessary condition for causation. Causation need
not be a linear phenomenon.
My reply is: I think you misread my statement. . . The statement says
"correlation OR relationship." Certainly a relationship may be curvilinear.
Hence, I totally agree with your statements about curvilinear
relationships...
Burke
_________________________
2/13/98
Date: Fri, 13 Feb 1998 08:02:45 PST
From: "Fetler, Mark" <MFetler@SMTP.CDE.CA.GOV>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Michael -
Would you care to define "causation." It strikes me that we've seen
definitions of the various research designs and methods, but no
definition of the relationship that some wish to demonstrate.
Most social scientists, from a very early age have had drummed in that
"correlation does not imply, much less entail, causation." However, it
seems to me that few of these introductory courses delve into the reasons
behind this dogma.
Doubtless, you are more familiar with the philosophical details than I,
but is it not true that "causation" has always been more more comfortably
an epistemological concept than an empirical one? In a scientific context
can talk of causality be anything more than a sloppy shorthand for
describing the results of a body of research?
As to the matter of randoml assignment to to sufficiently large groups
... doesn't this relate more to the certainty of statements about the
equivalence group means than to matters of causality?
- Mark Fetler
_________________
2/14/98
Date: Sat, 14 Feb 1998 03:36:55 -0500
From: "Donald F. Burrill" <dburrill@USER.XTDL.COM>
Subject: Re: Causal comparative and cause -Reply -Reply
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
On Thu, 12 Feb 1998, Burke Johnson wrote in part:
> THE QUESTION IS SHOULD WE DROP THE FIRST REQUIREMENT IN THE
> POPULAR "THREE CRITERIA FOR CAUSATION" HEURISTIC?
> 1. There must be a relationship or association or correlation,
Only if "relationship or association or correlation" is narrowly
interpreted to mean _only_ bivariate relationships (&c), excluding
multiple and/or partial correlations (&c) and nonlinear functions of
the variables of interest. I would consider any such narrow
interpretation indefensible (and other, less courteous, adjectives).
> 2. There must be time ordering of the variables, i.e., if a causes b
> then a must come before b
> 3. And, alternative hypotheses must be ruled out, i.e., the third
> variable problem.
>
> OR
>
> SHOULD WE CALL THE RELATIONSHIP that Kevin pointed out an
> "interactive relationship" and keep the statement that there must be a
> relationship (of some kind) in order to have causation??
This seems to be confounding statements 1 and 3.
>
> IF YOU THINK we should keep criterion one, how would you word it?
Doesn't much matter how you word it, somebody can always be
found who will insist on misinterpreting it.
------------------------------------------------------------------------
Donald F. Burrill, Professor Emeritus 416-923-6641 ext 2460
The Ontario Institute for Studies in Education
Toronto, Canada M5S 1V6 dburrill@oise.utoronto.ca
184 Nashua Road, Bedford, NH 03110 603-471-7128 dburrill@user.xtdl.com
------------------------------------------------------------------------
____________
2/18/98
Date: Wed, 18 Feb 1998 12:33:04 -0800
From: jack Fraenkel <jrf@SFSU.EDU>
Subject: The correlation/causal-comparative controversy
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Colleagues,
Thanks to Burke Johnson for sending me many of the postings on this
discussion; it's made me think about a topic that I had previously not paid
much attention to. Here are some of my thoughts on the issue:
*Both correlational and causal-comparative studies explore
relationships; it should not be implied that this is true more so for
correlational than for causal-comparative. In general, though, I think that
causal-comparative research is better for examining relationships involving
categorical variables, while correlational research is better for
examinaing relationships between continuous variables.
*Very often the methods used are indistinguishable; in both types,
corelational coefficients can and often are computed; chi-square tests are
performed; multiple regression analyses occur; and/or mean differences are
calculated.
*Both seek to identify possible (stress on the possible) causes of
phenomena that can be followed up by experiments;
*Interpretations of results in both (with regard to causality)
should be limited, because in neither case can the researcher say
conclusively (emphasis on the "conclusively") whether a particular factor
is a cause or result of that which is being observed (or if some third
factor is at work); the question remains as to whether one type offers a
stronger case for causality than the other (My thoughts on this below).
*Neither type of research is "better" than the other; however,
causal-comparative studies lend themselves to some kinds of investigations
better than correlational studies do, to wit:
1. when time, cost, ethics, or other factors prevent an experiment,
it often makes sense to conduct a causal-comparative study as an
alternative, but where a correlational study would either make little sense
or be less effective in finding out what the researcher is interested in.
An example would be where the Director of Curriculum in a school district
is thinking about implementing a new curriculum. He could conceivably
locate two districts, one where the new curriculum is being used and
another, similar (emphasis on the similar) district that is not, and then
compare student achievement in the two districts. Results of the comparison
would give the curriculum director some basis for making a decision about
whether to implement the new curriculum or not. I think this is often the
case in education, because the variables of interest simply cannot be
manipulated.
2. when one of the variables is categorical. Much
causal-comparative research (especially in medicine) that involves
categorical variables continues to be done and I don't see much point in
re-formulating it as correlational (i.e., point biserial). Example:
Comparing men and women with regard to their speaking ability could be
re-cast and analyzed as the correlation between gender and speaking
ability, but why, when the former is more straightforward and delta is as
useful to report as r?
3. when there are more than two categories involved. Research with
more than two categories does not lend itself to correlational analysis.
Example: A comparison of several ethnic groups in terms of their "attitude
toward government." Assigning numerical values to the categories in such
instances makes no sense, whereas a comparison of averages would be
straightforward.
*The key question in both types of research, for me, is always: Can
alternative explanations for outcomes be eliminated? This leads me to the
question as to whether causal-comparative research offers a stronger case
for causality than correlational research (or vice-versa). After thinking
this one over, I now really don't see how one could in any strong sense
"prove" such one way or the other. Burke Johnson's point, as well as that
of a few others, is that causal-comparative research has no necessary
advantage over correlational research in terms of causation. I think I'll
have to concede this point (By the way, Burke, I think this is what our
text says, both in Chapter one, particularly page 10, last paragraph, and
in Chapter 14. If your students are getting a different impression, I'd
like to know why or how; perhaps the wording should be changed, but it
looks okay to me).
However, I do think that there are times when the findings of a
causal-comparative study do offer strong evidence for causality (in fact,
almost as strong as the results of a carefully controlled experiment). I
would say this would be the case when the following conditions exist: (a)
the sequence of events is such that although variable A could cause
variable B, the reverse is not possible. This is true in much medical
research (e.g.,the research on the smoking/cancer link); (b) when there is
a comparison group that rejects (i.e., eliminates) other reasonable
alternative explanations for the results; and (c) when there have been
other causal-comparative studies conducted by different investigators using
different samples in different settings, and consistent results emerge from
these studies (as in much of the smoking research). When the combined
evidence from such studies is consistent, such results offer (to my way of
thinking) a strong case for causality.
A problem I have with much reported correlational research is that
the investigator often does not discuss the plausibility of (often does not
even identify) alternative explanations for results, whereas I find that
such are usally discussed in causal-comparative studies (take a look at
Carlsmith's study on the effects on math and verbal aptitude of boys when
their fathers were absent in the boys' early years).
*Finally, I think it might be better if we dropped the term
"causal-comparative," and replaced it with a term like "after-the fact"
experiments or some such (I find that Krathwohl suggests "after-the-fact
natural experiments" in his '93 text).
Cordially,
Jack R. Fraenkel
Research and Development Center
Burk Hall 254, College of Education
San Francisco State University
__________________
Date: Wed, 18 Feb 1998 20:58:27 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: The correlation/causal-comparative controversy -Reply
MIME-version: 1.0
Content-type: text/plain
Thanks to Jack Fraenkel for his comments and thoughts! I want to point
out that I use Fraenkel and Wallen's book and consider it far superior to
the previous best selling textbook. I also think the book by Gall, Borg, and
Gall is quite good, but it is a little too long for my purposes.
I have just a couple of comments...
<snip>
Jack Fraenkel says:
*The key question in both types of research, for me, is always:
Can alternative explanations for outcomes be eliminated? This leads
me to the question as to whether causal-comparative research offers
a stronger case for causality than correlational research (or vice-versa).
After thinking this one over, I now really don't see how one could in any
strong sense "prove" such one way or the other. Burke Johnson's point,
as well as that of a few others, is that causal-comparative research has
no necessary advantage over correlational research in terms of
causation. I think I'll have to concede this point.
Burke Johnson's comment
--I think that we all seem to agree on this basic point about causality.
Basically, when we compare apples to apples (e.g., the simple cases of
CC and Corr) and when we compare oranges to oranges (e.g., the
cases of CC and Corr where several variables have been controlled),
and so forth (e.g., prospective study to prospective study...), it just
doesn't matter whether one has a categorical or a quantitative
independent variable in terms of cause and effect.
--Now, the question is what to do about the fact that we have been
teaching for years (in our best selling books) that causal-comparative
research provides stronger evidence about cause and effect...
--As we know, the term causal comparative is the educational
researcher's term for ex post facto research. Therefore, it may be
instructive to note what Fred Kerlinger did with the term Ex Post Facto...
He included a chapter on ex post facto research in the 1973 version of
his seminal methods book (Foundations of Behavioral Research).
However, in the last edition (1986) of his book before his death, Kerlinger
eliminated the term ex post facto and used the term nonexperimental
research. BTW, if you look at Kerlinger's examples you will see that
some of the studies he uses have quantitative IVs and some have
categorical IVs; hence, he was not concerned about the scaling of the IV
in terms of cause and effect.
--My question is: would educational research methods teachers across
the United States (e.g., how about the ones on this list) accept the
change in terms (i.e., use the term "nonexperimental" rather than CC and
Corr) or would they simply find a book that kept saying what had been
said for many previous years (i.e., "One can use CC to explore cause
and effect; one can only use Corr to examine relationships" AND CC and
Corr are very different METHODS of research)? More specifically, do
you recommend that textbook writers jump directly to "nonexperimental"
as Kerlinger did, or should there be a transitional period (treat both CC
and Corr as cases of Nonexperimental)? Or, finally, do you have some
other, better, alternative that you would recommend?
Thanks.
Burke Johnson
--Boundary_(ID_HlWCXYLWI3UsMfcCQBVZrA)
Date: Thu, 19 Feb 1998 00:30:51 EST
From: Michael Scriven <Scriven@AOL.COM>
Subject: The correlation/causal-comparative controversy
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
Sorry to spoil the accord, but I can't buy the idea that CC is no stronger
than Corr. The issue is exactly as Fraenkel puts it; which approach does the
most to eliminate alternative explanations? And 'natural experiments', a.k.a.
ex post, a.k.a. non-experimental, are a lot better than mere correlations,
simply because they involve variables that have a pedigree as causal in this
kind of context. We don't call a study a natural experiment (CC, etc.) unless
this condition is met, and when it is, that's an edge over merely
correlational studies, which are the ones where one has no reason to call the
connection directly causal rather than a byproduct of other causal
connections.
Michael Scriven
Claremont Graduate University
____________________
Date: Thu, 19 Feb 1998 03:49:33 -0500
From: "Bryan W. Griffin" <bwgriffin@GSVMS2.CC.GASOU.EDU>
Subject: Re: The correlation/causal-comparative controversy
MIME-version: 1.0
Content-type: text/plain; charset="iso-8859-1"
Content-transfer-encoding: quoted-printable
Clearly there exists differences of opinion on this issue. Dr. Fraenkel
still leans toward causal-comparative (CC) studies (i.e., natural
experiments) as providing more evidence of causality than correlational
studies.
Dr. Scriven stresses this even more by stating:=20
>Sorry to spoil the accord, but I can't buy the idea that CC is no stronger
>than Corr. The issue is exactly as Fraenkel puts it; which approach does=
the
>most to eliminate alternative explanations? And 'natural experiments',=
a.k.a.
>ex post, a.k.a. non-experimental, are a lot better than mere correlations,
>simply because they involve variables that have a pedigree as causal in=
this
>kind of context. We don't call a study a natural experiment (CC, etc.)=
unless
>this condition is met, and when it is, that's an edge over merely
>correlational studies, which are the ones where one has no reason to call=
the
>connection directly causal rather than a byproduct of other causal
>connections.
>
>Michael Scriven
>Claremont Graduate University
I think Thomas Cook and Donald Campbell chapter 7 (pp. 295 - 340,
"Quasi-experimentation: Design & analysis issues for field settings, 1979,
Houghton Mifflin) addressed these issues well, but it appears many believe
that natural experiments eliminate (i.e., control) confounding variables
better than "merely correlational studies" (to use Dr. Scriven's words). I
cannot see the rationale for this belief, but I can see possible sources
for this confusion. Below I try to explain the issues that lead to the
judgement that correlational studies are inferior to CC studies in
providing control and, hence, the ability to identify possible causal
linkages.=20
Issue #1: Prediction (forecasting) vs. causal inference
One problem, I think, is that many confuse correlational techniques with
prediction only techniques. It is important to note that correlational
procedures (e.g., multiple regression) may be used for both prediction and
for investigating causal linkages. See Blalock (1964, Causal inference in
non-experimental research, UNC Press), or Pedhazur (1982, Multiple
regression in behavioral research: Explanation and prediction, HBJ) for
lucid discussions on this.
Issue #2: Group comparisons vs. quantitative variables
It appears that there are some who believe that categorical,
non-manipulated independent variables (i.e., CC studies) provide better
evidence of causality than quantitative, non-manipulated independent
variables (i.e., correlational studies). For example, one might argue that
sex (male vs. female) provides better evidence of causality than a
quantitative measure of intelligence, that is:
Math ach. =3D sex (male vs. female) + error (a CC study)
Math ach. =3D IQ + error (a correlational study)
Since sex is categorical and therefore a natural experiment, it provides
better evidence of causality that the model with IQ? Okay, what if
intelligence, like sex, consisted of two common levels -- high and low. Now
the model is:
Math ach. =3D IQ (high vs. low) + error (a CC study)
What does this study help us understand that the correlational counterpart
did not? Have we lost important information by dichotomizing IQ? Since we
are comparing two groups, does this make things easier to understand? Since
this is now a CC study, may we infer better evidence of causality?
Here's another study to consider. Suppose we wish to investigate the
effects of class size on achievement. We could use a natural experiment and
compare two (or three or four, etc.) class sizes. For convenience, let's
compare class sizes of 12 and 30 (keeping in mind that the unit of analysis
here should be classes, not students). Perhaps I would collect class
achievement means from 50 classes of size 12, and 50 more for classes of
size 30. The model might look like the following:
Math ach. =3D class size (12 vs. 30) + error (a CC study)
So we may find that students in classes of size 12 have slightly higher
achievement. We may be tempted to say, from this CC study, that smaller
classes result (for whatever reason) in better achievement.=20
Now, being inquisitive, I would prefer to check for more interesting
relationships. First, I would try to find naturally occurring class sizes
that ranged from, say, 10 to 30. As in the above CC study, I would collect
as many observations as possible, so I would have a number of classes of
size 10, a number of classes of size 11, of size 12, of size 13, =85, up to
size 30. Now, since this can be analyzed using correlations, and since my
independent variable is quantitative (a ratio), I have a correlational
study. Here's the possible model:
Math ach. =3D class size + error (a correlational study)
However, what's more interesting with this study is that I can now check,
using correlational methods of analysis (multiple regression) whether a
curvilinear relationship exits between class size and achievement using
polynomial terms. Of course this could also be done if there were enough
class sizes present in the CC study above using ANOVA, but that would
require far more degrees of freedom which would mean considerably less
power and precision, thus an increase in the likelihood of making a Type II
error. I'll assume that there is only one inflection point, so only the
first polynomial (quadratic) term is needed:
Math ach. =3D class size + (quadratic term) + error (a corr. study)
With this correlational study, I can see two things, whether class size is
related to mathematical achievement, and whether there are diminishing
returns for smaller class sizes (as reflected by the quadratic term).
Now, is this study weaker than the CC counterpart? Since I am not directly
comparing groups, as is traditionally done in CC studies, is this
correlational study somehow inferior to the CC counterpart? I'd argue that
it is far superior since it provides so much more information using the
same two degrees of freedom.
I think this brings me to the next issue -- CC studies resemble experiments
and correlational studies do not.
Issue #3: Group comparison exist in experimental and CC studies, but not in
correlational studies
Students in education and psychology are taught that classical experiments
have a treatment group (or two) and a control group. After collecting data
from the experiment, we analyze this data using ANOVA or ANCOVA procedures
and then interpret the results.
Since both CC and true experiments have group comparisons, then it is
understandable for one to say CC studies, since they resemble experiments,
do a better job of controlling confounding variables than do correlational
studies. After all, except for manipulation of the independent variable
(and possibly random assignment), an experiment and a natural experiment
(i.e., CC study) look alike. So does this make the CC better equipped to
rule out confounding variables as Dr. Scriven suggests?
First, let's question the idea of group comparisons. Must randomized
experiments have a treatment and control group to compare (like natural
experiments)? It seems that educators and psychologists only conceive of
experiments with a treatment and a control group.=20
Is it possible to have a randomized experiment in which the manipulated
independent variable is quantitative rather than categorical? If it is,
then that means the important distinction between CC and correlational
studies -- group comparisons -- is no longer important. After all, if a
randomized experiment can look like a correlational study, then...
Consider the familiar class size and achievement example. I wish to
determine, using randomized procedures, whether class size has an effect
upon achievement. I decide to have 20 experimental treatments --- a class
of size 10, 11, 12, etc., up to 30. I randomly assign students and teachers
to one of these 20 conditions --- both students and teachers are randomly
assigned to a class of size 10, 11, etc.=20
I'd probably treat the class mean level of achievement as the unit of
analysis again, so I'd need to have several classes of size 10, and so on.
After collecting the data, I would have the following model (with
appropriate polynomial terms to check for trends):
Math ach. =3D class size + (quadratic term) + error=20
Note that this study looks EXACTLY like the correlational study described
above, but this study is a true experiment with random assignment=
throughout!
I hope that it is clear now that the sole difference between CC and
correlational studies --- a categorical independent variable --- doesn't
provide any type of real justification for isolating causality. One does
not need to make group comparisons to find evidence of causality! Even TRUE
EXPERIMENTS can also be formulated in a manner that resembles correlational
research, and through replications can provide the same level of evidence
for causality that true experiments with group comparisons can provide.
The point I am trying to make here is that CC studies are not special and
don't grant any type of evidence of causality that correlational studies
cannot address. For years agricultural researchers have been using
quantitative and categorical independent variables for true experiments
(response surfaces), and for years sociologists have been using
quantitative and categorical variables in causal models (see Blalock).=20
With mere correlational studies, I can test alternative hypotheses and I
can attempt to control confounding variables. So what can be done with a CC
study that cannot be done with a correlational study in terms of
identifying possible causal relationships?=20
I think the correct answer to this question is: nothing.
_______________
Date: Thu, 19 Feb 1998 06:51:23 -0600
From: "J. Philip Miller" <phil@WUBIOS.WUSTL.EDU>
Subject: Re: The correlation/causal-comparative controversy
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
> Date: Thu, 19 Feb 1998 00:30:51 -0500 (EST)
> From: Michael Scriven <Scriven@aol.com>
> Subject: The correlation/causal-comparative controversy
> To: AERA-D@asuvm.inre.asu.edu
>
> Sorry to spoil the accord, but I can't buy the idea that CC is no stronger
> than Corr. The issue is exactly as Fraenkel puts it; which approach does the
> most to eliminate alternative explanations? And 'natural experiments', a.k.a.
> ex post, a.k.a. non-experimental, are a lot better than mere correlations,
> simply because they involve variables that have a pedigree as causal in this
> kind of context. We don't call a study a natural experiment (CC, etc.) unless
> this condition is met, and when it is, that's an edge over merely
> correlational studies, which are the ones where one has no reason to call the
> connection directly causal rather than a byproduct of other causal
> connections.
>
This seems to be a new claim, using a definition of CC which is based on
the variable's pedigree rather than whether it is of a categorical nature.
Could we please be enlightened as to an operational definition of what is a
good pedigree in order to claim CC rather than just Corr?
-phil
> Michael Scriven
> Claremont Graduate University
>
--
J. Philip Miller, Professor, Division of Biostatistics, Box 8067
Washington University School of Medicine, St. Louis MO 63110
phil@wubios.WUstl.edu - (314) 362-3617 [362-2693(FAX)]
http://www.biostat.wustl.edu/~phil
__________________
Date: Thu, 19 Feb 1998 10:56:42 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: The correlation/causal-comparative controversy -Reply
MIME-version: 1.0
Content-type: text/plain
Michael,
First, I will try not to repeat in any depth the points outlined by Bryan
Griffin. Second, I think it will be helpful if you provide some BASIS for
your assertions. Third, I think it will be helpful if you will reply to the
comments to your assertions so that we can DISCUSS these important
issues. I think you should start with Bryan Griffin's, and then I hope you
will move on to mine. This discussion has been going on for almost two
weeks now, and I would like to know which SPECIFICS Michael Scriven
disagrees with.
For example, do you disagree that we can change a quantitative variable
into a categorical variable and then move from a correlational study to a
causal comparative design? That's what one would be obliged to
conclude based on what our books teach. More importantly, now that
we have a categorical variable in this example, do you honestly believe
that one's evidence of causality has IMPROVED? That is what one would
be obliged to conclude based on what our books teach. If you think one's
evidence has improved in this example, then WHAT is your basis? I have
studied research in multiple disciplines, and I was shocked when I was
given the book by LR Gay to use in my educational research methods
course. Here are a few quotes from the late LR Gay's text for a reminder
of what is being taught in our textbooks:
"Correlational research ... describes conditions that already exist.
Causal-comparative research, however, ALSO ATTEMPTS to determine
reasons, or causes, for the current status of the phenomena under
study" (4th edition, p.283). "Causal-comparative studies attempt to
identify cause-effect relationships, correlational studies do NOT" (p.
284). "Corrrelational research attempts to determine whether, and to
what degree, a relationship exists between two or more quantifiable
variables" (p.14). "Causal-comparative and experimental research...both
attempt to establish cause-effect relationships; bot involve group
comparisons" (p.15). In a correlational study "each variable must be
experssible in numerical form, that is, must be quantifiable" (p.279). "The
purpose of a correlational study may be to determine relationships
between variables, or to use relationships in making predictions" (p.264).
"...variables that are highly related may suggest CAUSAL-COMPARATIVE
or experimental studies to determine if the relationships are causal"
(p.264).
Michael, is this what you would teach?
I agree with Fred Kerlinger that:
"Nonexperimental research is systematic empirical inquiry in which the
scientist does not have direct control of independent variables because
their manifestations have already occurred or because they are
inherently not manipulable. Inferences about relations among variables
are made, without direct intervention, from concomitant variation of
independent and dependent variables" (p.348, Foundations of Behavioral
Research, 1986).
AND it makes no difference (as Kerlinger shows with his examples) if
one's IV is quantitative or categorical.
That is what I teach. Please point out the mistake you apparently believe
we are making.
Are you contending that nature ONLY experiments with categorical
independent variables? Do you have a definition of causal-comparative
other than what is used in the books that we have been discussing? Do
you not believe that it is specious to argue that since a causal
comparative "looks like an experiment" it somehow must provide stronger
evidence for cause and effect than a similar correlational study? That
sounds (to me) like a statement someone just beginning their first
research course might make, not one that a leader in the field of program
evaluation would make. Do you know something that we don't know? If
yes, please explain it to us. My additional comments are embedded...
>>> Michael Scriven <Scriven@AOL.COM> 02/18/98 11:30pm >>>
said:
Sorry to spoil the accord, but I can't buy the idea that CC is no stronger
than Corr. The issue is exactly as Fraenkel puts it; which approach does
the most to eliminate alternative explanations?
My comment:
--Franekel is not the only one to put it this way. It has been put this way
10 or 15 times already in earlier posts. We all agree that to make
statements about cause and effect we must "eliminate alternative
explanations." What is your point? We are discussing the difference
between causal comparative research and correlational research. My
point is this: "When we compare apples to apples (e.g., the simple cases
of CC and Corr, with no controls) and when we compare oranges to
oranges (e.g., the cases of CC and Corr where several variables have
been controlled), and so forth (e.g., a prospective CC versus a
prospective Corr . . . ), it just DOESN'T MATTER whether one has a
categorical or a quantitative variable in terms of cause and effect." What
is your comment on this contention, Michael?
Michael Scriven also said:
And 'natural experiments', a.k.a. ex post, a.k.a. non-experimental, are a
lot better than mere correlations,simply because they involve variables
that have a pedigree as causal in this kind of context. We don't call a
study a natural experiment (CC, etc.) unless this condition is met, and
when it is, that's an edge over merely correlational studies, which are
the ones where one has no reason to call the connection directly causal
rather than a byproduct of other causal connections.
My comment?
--First, you seem to have missed the distinction between
causal-comparative and correlational as taught by our textbooks. BOTH
are nonexperimental. The only difference is the scaling of the IV (unless
one wants to teach stereotypes such as "CC researchers attempt to
control for third variables, but those Correlational researchers don't"). Do
you believe that nature ONLY experiments with categorical causal
variables? Do you believe that noticing that males and females differ on
some variable provides more evidence for cause and effect than noticing
that level of motivation or the amount of self-esteem is related to some
variable? This is what we have been discussing for quite some time
now.
In the spirit of open discussion...
Burke Johnson
Michael Scriven
Claremont Graduate University
___________________
Date: Thu, 19 Feb 1998 12:22:12 -0500
From: Gregory Camilli <camilli@RCI.RUTGERS.EDU>
Subject: Re: The correlation/causal-comparative controversy -Reply
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
I think it's wise to make a distinction between non-experimental studies
and quasi-experimental studies. That is, a quasi-experimental study can be
prospective and consequently many issues of control (e.g., selection bias)
can be worked out in advance. For example, suppose a treatment is to be
delivered to students having trouble (of some kind) with reading. An
ex-post-facto approach might be used to study such children in terms of
their exposure to different reading environments and opportunity-to-learn.
However, a different approach is to prospectively design a program based on
the best current understandings, and then to assign students to treatment
and comparison based on some measure of need. This is Campbell's
regression discontinuity design.
Of course, naturally-occurring treatments -- given a thick data base of
descriptive information -- might also uncover effective treatments. Yet
theories formed retrospectively (regardless of cross-validation) do not
carry the weight of prospective investigations. No matter how eloquently
my friend might argue that he "knew" it was going to rain today based on
his extensive meteorological expertise, I would still require a set of
corroborated predictions to satisfy my skeptical nature.
Part of the difference might depend on the planned use of a treatment.
Thus, part of the design process would be to create a description that
replications could be based on. After Cronbach, controls are not selected
to obtain a "causal connection" in which ALL alternative hypotheses have
been ruled out. Rather, controls are sought primarily with reproducibility
in mind, and the degree of effort required to obtain a control must be
taken into consideration. As Cronbach wrote "Imagination enables one to
consider the reproducibility of a statement about a historical event that
could not possibly be repeated" and "A reader who accepts a conclusion
without reading all the procedures and thinking hard about them is
accepting the conclusion on the basis of prior beliefs." (Designing
Evaluations, pp. 121, 122). I find it difficult to believe that in
ex-post-facto research the necessary information has been preserved for
considering reproducibility.
I might agree that correlational research has no "necessary" advantage, but
this is a technical point. Practical advantage, especially with regard to
reproducibility, is another matter.
Gregory Camilli
Department of Educational Psychology
Graduate School of Education
Rutgers University
10 Seminary Place
New Brunswick, NJ 08903
phone: 732-932-7496 ext. 343
fax: 732-932-6829
Visit the GSE Website: http://www.gse.rutgers.edu
_________________
2/19/98
Date: Thu, 19 Feb 1998 16:25:17 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: The correlation/causal-comparative controversy -Reply -Reply
MIME-version: 1.0
Content-type: text/plain
Gregory, I have a couple of thoughts that I think are relevant to our
discussion. They are at the end of each paragraph...
>>> Gregory Camilli <camilli@RCI.RUTGERS.EDU> 02/19/98 11:22am >>>
I think it's wise to make a distinction between non-experimental studies
and quasi-experimental studies. That is, a quasi-experimental study can
be prospective and consequently many issues of control (e.g., selection
bias) can be worked out in advance. For example, suppose a treatment
is to be delivered to students having trouble (of some kind) with reading.
An ex-post-facto approach might be used to study such children in terms
of their exposure to different reading environments and
opportunity-to-learn. However, a different approach is to prospectively
design a program based on the best current understandings, and then to
assign students to treatment and comparison based on some measure of
need. This is Campbell's regression discontinuity design.
My thought is:
--Nonexperimental research can also be retrospective OR
PROSPECTIVE. Let's look at Kerlinger's chapter on ex post facto
research in his 1973 book (i.e., when he still used the term ex post facto)
where he says "The usual ex post facto study uses groups that exhibit
differences in the dependent variable. In some longitudinal-type studies
the groups are differentiated first on the basis of the independent
variable. But the two cases are basically the same [with regard to the
problem of selection], since group membership on the basis of a variable
always brings selection into the picture" (p.383).
--The leading selling book in educational research (for many years) by
LR Gay says (when discussing CC research), "The basic approach is
sometimes referred to as retrospective causal-comparative research
(since it starts with effects and investigates causes), and the variation
as prospective causal-comparative research (since it starts with causes
and investigates effects). Retrospective causal-comparative studies are
far more common in educational research" (Gay, fourth edition, p.284).
--Finally, obviously some survey researchers (e.g., how about many
ot the sociologists and political scientists at the Institute for Social
Research in Ann Arbor) conduct prospective research to investigate
cause and effect (explanatory research), and they make no distinction
based on the scaling of the independent variable.
--My point is this: both causal-comparative and correlational research
can be prospective, and that neither CC nor Corr research is necessarily
better or worse than its nonexperimental rival.
Gregory also said:
Of course, naturally-occurring treatments -- given a thick data base of
descriptive information -- might also uncover effective treatments. Yet
theories formed retrospectively (regardless of cross-validation) do not
carry the weight of prospective investigations. No matter how
eloquently my friend might argue that he "knew" it was going to rain
today based on his extensive meteorological expertise, I would still
require a set of corroborated predictions to satisfy my skeptical nature.
My thought is
--You make a good point, but don't forget that even IF someone is doing a
retrospective study, that does not mean that they have necessarily fallen
victim to the post hoc, ergo propter hoc fallacy. (As an example, Paul
Lazarsfeld's technique of elaboration does not fall into this trap.) The
point I want to make is that retrospective research can also be
hypothesis and theory DRIVEN, focusing on confirmation. That is, there is
no reason that retrospective studies must be data-then-hypotheses.
They can also be of the form hypotheses-then-data. I agree with you
that prospective studies are potentially more convincing than
retrospective ones... Again, however, at least according to Gay,
Kerlinger, and me, there is no reason why nonexperimental or
causal-comparative or correlational research must be retrospective;
hence, the point I think you are trying to make is somewhat moot. (I'm not
quite sure which earlier comment you were reacting to, if any.) BTW, do
you contend that correlational research CAN be prospective? If yes, then
I guess you would conclude that correlational has an advantage over
causal-comparative, other things equal. You have found a point where
some viewpoints do differ. In particular, Fraenkel said in an earlier
message that causal-comparative research is retrospective research.
Remember that our debate is over the strength of the two
NONexperimental methods, causal comparative versus correlational
research (as defined in our textbooks); that excludes quasi-experimental
research and experimental research.
<snip>
Sorry if I missed some of your points...
Regards,
Burke J.
____________________
Date: Fri, 20 Feb 1998 09:13:10 MST
From: Norman David Giesbrecht <ndgiesbr@ACS.UCALGARY.CA>
Subject: S.E.M. as Causal-Comparative
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
I would like to suggest that some of the causal-comparative
vs. correlational discussion has been: a)limited by an out-dated
view of causal-comparative, b) that causal-comparative is
defined by purpose, design, and analysis rather than
variable measurement, and c) that structural equation modeling
is a much more current expression of causal-comparative design.
a) Borg & Gall (5th ed) describe the causal-comparative approach
as "one approach to epxloring cause-and-effect relaitonships
... a particular way of analyuzing relational data" and (but ?)
go on to describe it in terms of t-tests and anova/manova.
I would content that the comparison of means across groups
reflects an out-dated approach. i.e., even Borg & Gall
state "the causal-comparative method was more widely used
years ago than it is now, because the statistical techniques
associated with this method - primarily the t-test and
analysis of variance - were well known to researchers
then. In recent years, however, researchers have
dsicovered that correlational statistics ... can do
everything that the t-test and anlsysis of variance
can do and more" (p. 536-537)
My contention is that the earlier incarnation of
causal-comparative "was more widely used years ago"
because it was (as others have said) a "psuedo"
experimental approach without the rigor of
an experimental design but utilizing anlaytic
techniques and treating the results as similar
to experimental findings.
b) as others have pointed out, variable measuremnt
(i.e., categorical) is a red herring ... it is
only when "comparative" is used to refer to
a comparison between two+ groups that a
categorical variable enters the anlaysis
c) I would contend that "comparative" need
not refer to two groups of subjects but can
(more appropriately) refer to comparison
between theory and data (hence "causal-comparison")
Structural equation modeling techniques utilize
a comparison between hypothesized theoretical
relationships (both at the measurement and
factor-relaitonship or path level) and the
correaltion/covariance patterns in the actual
data. Categorical variables may or may not
be present, multiple groups may or may not
be present (i.e., multi-sample analyses)
... one could say the technique uses
advanced correlational procedures to
evaluate causal hypotheses through comparison
of "real data" with "theoretically-grounded
causal hypotheses". The data can be longitudinal
(which addresses some of the causal pre-conditions
re: time-ordering of events / variables) but
does not typically use and experimental design.
Typical caveats re: causal claims hold, of course.
I think there is value in distinguishing an
sem-oriented causal-comparative approach from
a correlational approach .. .though the difference
is more of degree than character.
My 2 cents,
Norman Giesbrecht, Ph.D.
__________________
2/23/98
Date: Mon, 23 Feb 1998 15:32:04 EST
From: Michael Scriven <Scriven@AOL.COM>
Subject: Re: The correlation/causal-comparative controversy
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
In a message dated 2/19/98 5:51:50 AM, you wrote:
<<Could we please be enlightened as to an operational definition of what is a
good pedigree in order to claim CC rather than just Corr?>>
Yes, sorry; I'm going back to the original idea behind CC, which was taken to
be equivalent to the use of categorical variables, and I'm bypassing the
latter move - which I think was a mistaken analysis - in order to save the key
methodological point, which is that there are non-experimental studies that
modestly support (but do not deductively entail) causal connection better than
studies about which we only know that some of the variables are correlated.
Guilty of not clarifying this before. I see this discussion as something like
bypassing the interpretation of factor analysis that leads people to talk
about factors 'explaining' the variance - which they don't in general because
f.a. is only correlational analysis - and pointing out that, nonetheless, if
the factors are prescreened for causal potency in like contexts, one can
generate plausibility-increasing arguments for causation from f.a.
Michael Scriven
--Boundary_(ID_NJXVVseJBml4C16DAoYYgQ)
_____________
Date: Mon, 23 Feb 1998 16:44:42 EST
From: Michael Scriven <Scriven@AOL.COM>
Subject: Re: Causal comparative and cause
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
In a message dated 2/19/98 9:26:47 AM, Mark Fetler wrote:
<<Would you care to define "causation." >>
Ah, with friends like Mark, who needs enemies!
Cause is an absolute primitive, i.e., not definable in terms of other, simpler
and different notions. There have been many attempts at the definition and
probably the closest one can come is to say that a cause is an essential
element of a group of factors which together comprise an empirically
sufficient condition for the effect. However, (i) in most contexts, some of
the remaining factors are not known, one only infers their existence; (ii)
this definition fails because of cases of overdetermination (since there would
be at least two such entities present, but only one is in fact the cause). So,
the notion is irreducible, in the same way as the concept of knowledge or
number is irreducible, but clear enough to all of us that we can use it.
It is of course intimately connected to the primitive notion of agency, of
doing something, which we acquire around two years of age; hence the
importance of random assignment in the classical design. So, that's not just a
matter of certainty, it's connected to the essence of the concept - random
assignation is our lab equivalent of us 'just doing' whatever it is the
independent variable is doing, i.e., from free choice. That's what breaks the
link to other prior causes that might otherwise be the real cause.
Michael Scriven
Claremont Graduate University
--Boundary_(ID_NJXVVseJBml4C16DAoYYgQ)
Date: Mon, 23 Feb 1998 21:00:41 EST
From: Michael Scriven <Scriven@AOL.COM>
Subject: Re: The correlation/causal-comparative controversy -Reply
MIME-version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7bit
Wish I had time to take up all of Burke Johnson's points, but I'm about 400
messages backlogged and must get to the ones from current students, and
perhaps my earlier replies to Mark Fetler and others will help, along with
this one.
1. I'm bypassing what I consider the mistaken excursion into the modality of
the IV. Your arguments about that are fine. But that's not the main point,
it's a distraction.
2. Focussing on the main point, whether there are two classes of research
(which overlap, but have distinct non-overlapping parts), plain correlational
vs. the well-done ex post facto, then the comments you quote from L.R. Gay are
all fine, much to your horror. Try reading them again without thinking about
the quantitative/categorical issue as central.
3. The Kerlinger quote is fine, but has to be interpreted rather carefully: he
says "Inferences about relations among variables are made, without direct
intervention, from concomitant variation of independent and dependent
variables". Won't quite do, unless we add (or treat as implicit), "with
varying degrees of plausibility, depending on the causal pedigree and the
particular context". That's why the Utah hospital study of the smoking/lung
cancer (population was adult converts to Mormonism) connection was so
important, and why previous correlational studies were inadequate: the
intervention (stopping smoking) was quasi-random, although this was not an
experimental study.
Sorry that I seemed not to be giving you any basis for my conclusions; it's
hard to know how far back to go in these discussion without losing most of the
readers. Perhaps this comes nearer.
Cheers,
Michael Scriven
--Boundary_(ID_NJXVVseJBml4C16DAoYYgQ)
Date: Mon, 23 Feb 1998 23:02:37 -0500
From: "Bryan W. Griffin" <bwgriffin@GSVMS2.CC.GASOU.EDU>
Subject: Correlational and Causal-Comparative, and retro/prospective studies
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Hello Michael
After reading your messages, I think perhaps I understand what you are
trying to say, but your points are difficult to understand. The reason, I
think, is because you appear to be confusing causal-comparative and
correlational with the distinction between retrospective (case-control) and
prospective (cohort) non-experimental investigations.
With some of your comments ("well-done ex post facto," and the reference to
a smoking study), it appears that you are referring to advantages of
prospective non-experimental studies relative to retrospective studies, not
with inherent differences between CC and correlational studies. If this is
the case, then I agree that "well-done" prospective studies are better than
retrospective studies.
Now, both prospective and retrospective studies can be either CC or
correlational, so the question still remains -- why do we need the
distinction between CC and correlational studies? And does the difference
between CC and correlational amount only to the scaling of the IV?
*******************
>2. Focussing on the main point, whether there are two classes of research
>(which overlap, but have distinct non-overlapping parts), plain correlational
>vs. the well-done ex post facto, then the comments you quote from L.R. Gay
are
>all fine, much to your horror. Try reading them again without thinking about
>the quantitative/categorical issue as central.
>3. The Kerlinger quote is fine, but has to be interpreted rather
carefully: he
>says "Inferences about relations among variables are made, without direct
>intervention, from concomitant variation of independent and dependent
>variables". Won't quite do, unless we add (or treat as implicit), "with
>varying degrees of plausibility, depending on the causal pedigree and the
>particular context". That's why the Utah hospital study of the smoking/lung
>cancer (population was adult converts to Mormonism) connection was so
>important, and why previous correlational studies were inadequate: the
>intervention (stopping smoking) was quasi-random, although this was not an
>experimental study.
>
>
___________________________________________________________________
Bryan W. Griffin
Phone: 912-681-0488
E-Mail: bwgriffin@gsvms2.cc.gasou.edu
WWW: http://www2.gasou.edu/edufound/bwgriffin/bgriffin.htm
--Boundary_(ID_NJXVVseJBml4C16DAoYYgQ)--
__________
2/24/98
Date: Tue, 24 Feb 1998 15:52:52 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: A Correlational/C-C Questionnaire
MIME-version: 1.0
Content-type: text/plain
Michael Scriven said:
<snip>
The key methodological point...is that there are non-experimental studies
that modestly support (but do not deductively entail) causal connection
better than studies about which we only know that some of the variables
are correlated.
Here is my comment and general reply to Michael Scriven:
I believe that this particular comment is obvious to everyone on this list.
Elaborating, I assume that you mean that some non-experimental studies
are better than others depending on one's search for alternative
hypotheses, statistical control of plausable confounding variables,
attempts to establish time order based on theoretical considerations and
on design control such as the use of prospective instead of
cross-sectional data, etc. These are the kinds of considerations we all
make when we are interested in cause and effect. We don't seem to be
getting anywhere in our replies to each other SO FAR... Perhaps, we
can locate our differences using a DIFFERENT approach...
I BELIEVE THAT WE CAN CLARIFY ANY DIFFERENCES WE MIGHT HAVE
WITH EACH OTHER, OR WITH THE CONTENTIONS MADE IN OUR
TEXTBOOKS, IF YOU WILL DEFINE THESE TWO WORDS AND ANSWER
MY "ASSUMPTIONS QUESTIONNAIRE."
BTW, I HOPE THE _OTHERS_ FOLLOWING THIS DISCUSSION WILL ALSO
DO THE FOLLOWING...
First, please define these two words:
1. Correlational research.
2. Causal-comparative research.
Second, fill out the following brief questionnaire:
1. The independent variable in a causal-comparative design can be:
a. categorical
b. quantitative
c. both a and b
2. The independent variable in a correlational design can be:
a. categorical
b. quantitative
c. both a and b
3. A causal-comparative study can be:
a. retrospective
b. prospective
c. both a and b
4. A correlational study can be:
a. retrospective
b. prospective
c. both a and b
5. A researcher doing a causal-comparative research study can control
for some alternative explanations or confounding variables.
a. true
b. false
6. A researcher doing a correlational research study can control for
some alternative explanations or confounding variables.
a. true
b. false
7. In causal-comparative research, the researcher can state his or her
hypotheses a priori and then test those hypotheses with empirical data.
a. true
b. false
8. In correlational research, the researcher can state his or her
hypotheses a priori and then test those hypotheses with empirical data.
a. true
b. false
9. Please list any differences between CC and Corr research that were
not addressed in this questionnaire but do affect one's ability to make
causal attributions.
_____________________________________________________
_____________________________________________________
_____________________________________________________
I suspect that if you will tell us your definitions of C-C and Corr and fill out
the questionnaire then we will be able to see how C-C and correlational
DIFFER according to you. THEN we should be able to determine whether
the differences identified in the definitions, the questionnaire, and your
additional responses affect one's ability to make causal attributions.
Does it make sense to claim that one can make claims about cause and
effect with causal-comparative but NOT with correlational? I contend that
once you clear out the smoke, you will not find any difference that
affects one's ability to establish evidence of causality. If there is an
important difference then someone on this list should surely be able to
identify it for us.
Before I end, I want to make it clear that I believe the following argument
is illogical... I call it the "stereotype argument" (and it is insulting, I
suspect, to people trained in rigorous correlational research methods). It
goes something like this, "Causal-comparative research can be used to
gather some evidence about causality; however, correlational research
cannot. This is true because...you know...THOSE CORRELATIONAL
RESEARCHERS don't know how to control for confounding variables or
provide evidence about the time ordering of their variables...but THOSE
CAUSAL-COMPARATIVE RESEARCHERS certainly DO know how...they
just TEND to know a little more about good research than correlational
researchers..."
Let's see if we can all get to the bottom of this!
Cheers, and thanks in advance for your reply.
Burke Johnson
_____________
Date: Tue, 24 Feb 1998 17:02:58 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: The correlation/causal-comparative controversy -Reply -Reply
MIME-version: 1.0
Content-type: text/plain
My replies are embedded:
>>> <Scriven@aol.com> 02/23/98 08:00pm >>>
Wish I had time to take up all of Burke Johnson's points, but I'm about 400
messages backlogged and must get to the ones from current students,
and perhaps my earlier replies to Mark Fetler and others will help, along
with this one.
My reply:
Michael, I provided a very brief questionnaire in my previous post. I also
ask one question below. It should not take much time.
Michael said:
1. I'm bypassing what I consider the mistaken excursion into the modality
of the IV. Your arguments about that are fine. But that's not the main
point, it's a distraction.
My reply:
I also consider the modality of the IV to be mistaken. It just happens to be
what is taught in many of our texts--therefore, it seems hardly "a
distraction." I have been suggesting for quite some time now that the
modality of the IV is irrelevant for causality. Hence, we agree on this
point. Let's go on to the next point and determine whether we agree...
Michael also said:
2. Focussing on the main point, whether there are two classes of
research (which overlap, but have distinct non-overlapping parts), plain
correlational vs. the well-done ex post facto, then the comments you
quote from L.R. Gay are all fine, much to your horror. Try reading them
again without thinking about the quantitative/categorical issue as central.
My reply:
Okay. Eliminating the quantitative/categorical issue, the difference
between causal-comparative and correlational research dissappears.
They are both nonexperimental approaches, and both may provide some
evidence of causality to the degree that they 1. establish the existence
of relationships, 2. provide evidence of time ordering of variables, and 3.
eliminate alternative hypotheses.
Once again, the STRAW MAN appears in your argument about why C-C
is better than Corr. You compared "plain correlational vs. the well-done
ex post facto". Why do you insist on making this comparison? The only
reason that I can think of is because of a stereotype you may have of
"correlational researchers." ("Those causal-comparative researchers
attempt to eliminate alternative explanations...but THOSE
CORRELATIONAL RESEARCHERS do not.") I remind you that the early
developers of path analysis and SEM woud probably call themselves
correlational researchers BEFORE they would call themselves
causal-comparative researchers--I'm referring to economists and
sociologists here, who, have likely never heard the term
CAUSAL-comparative research. Where am I going wrong here? I remind
you of an important question I posed earlier but you did not answer: (I
modified the question, using the words causal-comparative and
correlational rather than the words categorical variable and quantitative
variable, since you agreed that the scaling of the IV is irrelevant.)
Do you agree or disagree with the following statement?
"When we compare apples to apples (e.g., the simple cases of CC and
Corr, with no controls) and when we compare oranges to oranges (e.g.,
the cases of CC and Corr where several variables have been
controlled), and so forth (e.g., a prospective CC versus a
prospective Corr . . . ), it just DOESN'T MATTER whether one has a
causal-comparative design or a correlational design in terms of cause
and effect."
Please answer these two questions.
Q1. Do you agree or disagree with that statement (about apples and
oranges and causality)?
Q2. If there is a flaw in the statement, what is the flaw?
<snip>
Michael said:
Sorry that I seemed not to be giving you any basis for my conclusions;
it's hard to know how far back to go in these discussion without losing
most of the readers. Perhaps this comes nearer.
My comment:
If you take a moment to answer the two questions posed above (one of
the questions only requires a yes or a no answer) and the BRIEF
questionnaire in my previous post to you (I think there were 9 questions,
8 of which were multiple choice), I'm pretty sure we can get to the heart
of this debate rather QUICKLY. Then, perhaps, we can find an agreeable
solution.
Thanks for your thoughts!
Burke Johnson
Cheers,
Michael Scriven
____________
2/26/98
Date: Wed, 25 Feb 1998 18:11:08 -0800
From: DEANNA NIELSON <nielsond@WORLDNET.ATT.NET>
Subject: Re: 10 Reasons Causal-Comparative is better than Correlational
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7bit
Burke Johnson wrote:
>
> Here are some arguments about "why causal-comparative research is
> better than correlational research" that just don't work:
>
> 1. I wish it was better, so it is.
> 2. I was told that it was better, so it must be.
> 3. I read somewhere that it was better, so it must be.
> 4. I feel that it is better, so it must be.
> 5. I had a dream . . .
> 6. It is better because those correlational people don't know how to
> control for third variables or think about causality as well as us true
> scientists.
> 7. It LOOKS more like an experiment, so it must be better.
> 8. It is better because we DEFINE it to be better.
> 9. "Comparative" is a better word than "correlational."
> 10. It is just better.
>
> Sorry to be silly, but I was bored...
>
> Burke Johnson
Thanks, Burke. While I've enjoyed all the discussion and debate, I
appreciate the sentiments expressed here.
Deanna Nielson
--Boundary_(ID_9tlioJ6gpHp3Fvm4CQX2Zg)
Date: Wed, 25 Feb 1998 21:33:51 -0500
From: Maria Pennock-Roman <mjp12@PSU.EDU>
Subject: Re: 10 Reasons Causal-Comparative is better than Correlational
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Although I'm a little tired of this discussion, I do agree with Burke that
the distinction between causal-comparative and correlational studies is
superficial and it involves the statistics used, and not the strength of
causal inferences that can be made from it. (The Gall et al book is very
clear on this issue.)
At least two persons on the list have already mentioned one superior
correlational method---structural equation modeling which is probably the
best way to control of extraneous and rival explanations in non-experimental
designs. Most well done correlational studies today do not merely report
the strength of an association-- they fit several path models to test which
model or rival hypothesis is most consistent with the data found. Because
SEM models can include both categorical and continuous variables, they are a
powerful method for examining differences between existing groups composed
of individuals who have not been randomly assigned to their group (as is
typical of non-experimental data). Such an analysis can give us a much
better handle on what is a more plausible "cause" than another, because some
competing explanations are ruled out.
Perhaps the persons still claiming that CC designs are more powerful for
establishing causes are unfamiliar with well done SEM studies. The EQS
manual by Bentler gives a list of studies that have used SEM for those
interested in looking up what has been done with this method. You'll see
that they go far beyond examining merely the strength of association between
variables.
Maria
At 03:07 PM 2/25/98 -0600, you wrote:
>Here are some arguments about "why causal-comparative research is
>better than correlational research" that just don't work:
>
>1. I wish it was better, so it is.
>2. I was told that it was better, so it must be.
>3. I read somewhere that it was better, so it must be.
>4. I feel that it is better, so it must be.
>5. I had a dream . . .
>6. It is better because those correlational people don't know how to
>control for third variables or think about causality as well as us true
>scientists.
>7. It LOOKS more like an experiment, so it must be better.
>8. It is better because we DEFINE it to be better.
>9. "Comparative" is a better word than "correlational."
>10. It is just better.
>
>Sorry to be silly, but I was bored...
>
>Burke Johnson
>
>
Maria Pennock-Roman
Associate Prof. of Educational Psychology
232 CEDAR Bldg.
The Pennsylvania State University
University Park, PA 16802-3109
tel. (814) 865-4368
mjp12@psu.edu
_____________
2/28/98
Date: Fri, 27 Feb 1998 15:56:07 EST
From: Scriven <Scriven@AOL.COM>
Subject: Re: The correlation/causal-comparative controversy -Reply
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sorry, Burke, can't do the quiz because I'm running 200 messages a day and it
is now well buried. But here's a simple enough answer to your underlying
question.
1. There is a perfectly respectable kind of correlational research that
provides zero basis for causal ascription. It only looks at the correlation,
which is useful for valuable tasks such as prediction. Let's call this basic
correlational (BC) research.
2. There is another kind of research, experimental causal (EC) research, which
provides the best possible basis for ascription of causes, especially but not
only if it has randomly assigned treatment to matched groups.
3. There's another kind of research, which can provide a prima facie basis for
cautious causal ascription, although not experimental. It makes sense to call
this causal correlational (CC) research. The strongest form of it involves
apparently random application of a known possible cause in a 'natural
experiment', which is found to be highly correlated with the occurrence of a
potential effect.
It does not seem helpful to insist that 1 and 3 are simply (sophisticated)
correlational; the distinction has long been recognized by researchers in a
score of fields.
Michael Scriven
Claremont Graduate University
------------------------------
Date: Fri, 27 Feb 1998 19:11:00 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: The correlation/causal-comparative controversy -Reply -Reply
MIME-Version: 1.0
Content-Type: text/plain
Michael,
Perhaps someone will pick up on your solution. Here is my paraphrase
of your comment: Delineate two forms of nonexperimental research and
call them: basic correlational (BC) and causal correlational (CC). Causal
thinking and evidence are important in the latter approach.
I wonder what others think about this? I have no major problems with it,
as long as no one claims that the independent or predictor variable must
be quantitative in basic correlational and the independent variable must
be categorical in causal correlational. I do, however, think the term
"correlational" has a lot of baggage attached to it and should probably not
be used in either case. Cook and Campbell argued in 1979 that we
should drop the term "correlational" research. We should probably have
listened to them. Perhaps the following terms would be better for your
two cases: "basic relationship" research and "causal relationship"
research (i.e., BR and CR). I'm doing some writing on this topic, and I am
leaning toward these words: descriptive research, predictive research,
and nonexperimental explanatory research. Causal thinking becomes
important in the last case (i.e., explanatory). BTW, if you are wondering
where survey research is...I treat it as a method of "data collection" that
can be used in descriptive, predictive, or explanatory research. (I have
learned that there is no perfect research methods typology.)
Does anyone have any thoughts on these nonexperimental research
classification systems:
1. Basic correlational research versus causal correlational research
2. Basic relationship research versus causal relationship research
3. Descriptive research, predictive research, and nonexperimental
explanatory research.
Does anyone have a different typology of NONexperimental research
that they think works well?
Thanks for the discussion,
Burke Johnson
University of South Alabama
Michael Scriven said:
Sorry, Burke, can't do the quiz because I'm running 200 messages a day
and it is now well buried. But here's a simple enough answer to your
underlying question.
1. There is a perfectly respectable kind of correlational research that
provides zero basis for causal ascription. It only looks at the correlation,
which is useful for valuable tasks such as prediction. Let's call this basic
correlational (BC) research.
2. There is another kind of research, experimental causal (EC) research,
which provides the best possible basis for ascription of causes,
especially but not only if it has randomly assigned treatment to matched
groups.
3. There's another kind of research, which can provide a prima facie
basis for cautious causal ascription, although not experimental. It makes
sense to call this causal correlational (CC) research. The strongest form
of it involves apparently random application of a known possible cause in
a 'natural experiment', which is found to be highly correlated with the
occurrence of a potential effect.
It does not seem helpful to insist that 1 and 3 are simply (sophisticated)
correlational; the distinction has long been recognized by researchers in
a score of fields.
Michael Scriven
Claremont Graduate University
------------------------------
Date: Fri, 27 Feb 1998 20:44:25 -0500
From: Maria Pennock-Roman <mjp12@PSU.EDU>
Subject: Re: The correlation/causal-comparative controversy -Reply -Reply
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Hi, Burke, and Michael, I vote for the last choice: descriptive,
predictive, and non-experimental explanatory research. Maria
>Does anyone have any thoughts on these nonexperimental research
>classification systems:
>1. Basic correlational research versus causal correlational research
>2. Basic relationship research versus causal relationship research
>3. Descriptive research, predictive research, and nonexperimental
>explanatory research.
>
>Does anyone have a different typology of NONexperimental research
>that they think works well?
>
>Thanks for the discussion,
>Burke Johnson
>University of South Alabama
>
>
>
Maria Pennock-Roman
Associate Prof. of Educational Psychology
232 CEDAR Bldg.
The Pennsylvania State University
University Park, PA 16802-3109
tel. (814) 865-4368
mjp12@psu.edu
______________
Date: Sat, 28 Feb 1998 14:25:06 -0500
From: Gregory Camilli <camilli@RCI.RUTGERS.EDU>
Subject: Correlation/causal-comparative: Reply to Burke
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Looking over the aera-d message archive, I realized I had missed the
comment by
Burke Johnson:
>My thought is:
--Nonexperimental research can also be retrospective OR PROSPECTIVE.
I used the term "quasi-experimental" which poses a kind of "what if"
question, e.g., if a control group subject had received a treatment, what
would the increment (decrement) in outcome have been? To carry out a
quasi-experimental study (either prospective or retrospective), it must be
theoretically possible for a case to be (or to have been) assigned to a
treatment or control condition. "Nonexperimental" more appropriately
refers to the situation in which this is not possible, e.g., sex difference
studies. Now this discussion might be somewhat moot to the original point
of contention which was limited solely to nonexperimental methods. BTW, I
completely agree that the scaling of the independent variable has no
necessary connection to the quality of an inference.
-- BTW, do you contend that correlational research CAN be prospective? If
yes, then I guess you would conclude that correlational has an advantage over
causal-comparative, other things equal.
Sorry, I can't parse the (compound) question. If you care to rephrase,
I'll take a whack.
One other point I would like to make. Burke Johnson said:
<snip>
> BTW, correlation or relationship is a
>necessary but not sufficient condition for causation, right? As I tell my
>students, "it ain't nearly enough but you gotta have it."
There is a well-know situation in which a causal effect exists, but the
correlation is zero: the suppressor effect. Sufficiently tangled
relationships might also result in an controlled coefficient with a
near-zero estimate when the true effect is nonzero (i.e., when key
independent variables are poorly measured or highly correlated). I think
the right way to acknowledge the correlation-cause connection is "causation
is a necessary, but not sufficient cause for observed correlation."
Gregory Camilli
Department of Educational Psychology
Graduate School of Education
Rutgers University
10 Seminary Place
New Brunswick, NJ 08903
phone: 732-932-7496 ext. 343
fax: 732-932-6829
Visit the GSE Website: http://www.gse.rutgers.edu
--Boundary_(ID_D1OHwRYQxPHmZZ07XlJF0A)
Date: Sat, 28 Feb 1998 16:28:31 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Correlation/causal-comparative: Reply to Burke -Reply
MIME-version: 1.0
Content-type: text/plain
Gregory Camilli replied to my earlier contention that "correlation or
relationship is a necessary but not sufficient condition for causation" as
follows:
<snip>
There is a well-know situation in which a causal effect exists, but the
correlation is zero: the suppressor effect. Sufficiently tangled
relationships might also result in an controlled coefficient with a
near-zero estimate when the true effect is nonzero (i.e., when key
independent variables are poorly measured or highly correlated). I think
the right way to acknowledge the correlation-cause connection is
"causation is a necessary, but not sufficient cause for observed
correlation."
My comment:
We discussed this issue (suppressor effects, moderated effects,
curvilinear effects, etc.) and decided that the three commonly cited
criteria for direct causation (i.e., 1. relationship of some form, 2. time
ordering of the variables, and 3. rule out alternative explanations) should
still stand. You are right, though, that condition number one should not
say "correlation." In my defense, I did originally say "correlation OR
relationship." I think the word "correlation" should probably be dropped
from condition one because the word brings to mind a statistical index.
How's this: Condition 1 "a relationship between the variables must exist
in order for direct causation to occur." Condition 1 (as rewritten) stands
as a necessary but not sufficient condition for causation.
Burke J.
______________
3/2/98
Date: Mon, 2 Mar 1998 13:30:25 -0500
From: Gregory Camilli <camilli@RCI.RUTGERS.EDU>
Subject: Re: Correlation/causal-comparative: Reply to Burke -Reply-Reply
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Burke wrote (in a private message that he has agreed to let me post):
>"A relationship of some form between events or variables A and B is
necessary but not sufficient for direct causation."
>I interpret the statement to say that it must be theoretically possible to
observe a relationship of some form.
I think the problem is this: the relationship in question must be measured
in some way that does not beg the causal question. Moreover, the statement
depends on having a conceptually sharp notion of causation. Without these
prereqs, there is no way to verify the truth of the statement. As an
opinion, however, it seems to be as good, if not better, than many other
opinions about cause-effect; though many such opinions cannot be falsified.
For example, "All things are caused by an undetectable gremlin." So,
despite the beating that Hume takes from Michael Scriven, there is much
wisdom IMO about the critical role of experience in providing coherence
between understandings of causes and effects.
Gregory Camilli
Department of Educational Psychology
Graduate School of Education
Rutgers University
10 Seminary Place
New Brunswick, NJ 08903
phone: 732-932-7496 ext. 343
fax: 732-932-6829
Visit the GSE Website: http://www.gse.rutgers.edu
--Boundary_(ID_STJaP/EZ8K4WySIUGA/D+Q)
Date: Mon, 2 Mar 1998 14:45:32 -0500
From: Gregory Camilli <camilli@RCI.RUTGERS.EDU>
Subject: Re: Correlation/causal-comparative: Reply to Burke -Reply
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Burke wrote:
>My comment:
>We discussed this issue (suppressor effects, moderated effects,
>curvilinear effects, etc.) and decided that the three commonly cited
>criteria for direct causation (i.e., 1. relationship of some form, 2. time
>ordering of the variables, and 3. rule out alternative explanations) should
>still stand. You are right, though, that condition number one should not
>say "correlation." In my defense, I did originally say "correlation OR
>relationship." I think the word "correlation" should probably be dropped
>from condition one because the word brings to mind a statistical index.
>How's this: Condition 1 "a relationship between the variables must exist
>in order for direct causation to occur." Condition 1 (as rewritten) stands
>as a necessary but not sufficient condition for causation.
Still doesn't work. If you reject the "correlation" content but retain
"relationship," the statement is dangerously close to a tautology (since
"cause" could be operationalized as "direct relationship"). As I noted in
my post both correlational indices and measures of direct relationship can
be within a stone's throw of zero given a true causal effect. Some of this
has to do with the imperfections of measurement, but some of it also has to
do with model or specification error.
What I did realize at the time is that there is plenty of experiences in my
field where direct nozero estimates (statistically significant beyond
argument) are also not indications of a true effects. For example, with
incorrectly specified models in DIF analysis, nonzero effects are LIKELY to
be found when true effects are zero. (Rebecca Zwick shows this clearly in
her 1990 JES article.) So what we are left with is
1. Causation is neither necessary nor sufficient for observed nonzero
correlation.
2. Observed nonzero correlation is neither necessary nor sufficient for
causation.
This may be the reason why you hear people saying that evidence and theory
are "loosely coupled." Thus, some scientists continue to work on problems
after "definitive" solutions have been demonstrated because they mistrust
the evidence or models used to draw conclusions. For example, the
continuing controversy over the universe expansion constant (Alan Sandage
and the French guy) could not be resolved. (I have heard recently that the
issue has been settled in favor of an ever-expanding universe, but can't
recall the source.)
"Cause and effect" is a tough nut to crack as Michael Scriven has noted. I
agree with his statement -- given the limitations -- that " a cause is an
essential element of a group of factors which empirically comprise an
empirically suffficient condition for the effect." Still, I believe that
his limitations are understated for a number of reasons. First, effects
are not measured independently of theories, and thus cannot be verified in
an absolute sense. Second, individual studies are typically not equal to
the task of sorting through and ordering sufficient conditions -- or
establishing a definitive measurement procedure for determining an effect.
Third, even given Michael's definitional framework, cause-effect is not
unrelated to purpose. For example, in some situations it might be
sufficient to say that a person is sick because he has a fever. For other
purpose, the pathogen must me identified; and still for other purposes,
specific proteins produced by the pathogen must be understood.
In a larger sense, CC and CR studies both contribute to a research process.
While CR studies have great flexibility, CC studies often have a greater
simplicity. No doubt individual preference will continue to play a large
role in selection methodology, but so might a substantive question.
Finally, Michael's remark on cause reminds me of a story I heard about an
executive who was bemusing the fact that one half of the agency's
advertizing budget was a waste. Problem was, the exec didn't know which
half.
Gregory Camilli
Department of Educational Psychology
Graduate School of Education
Rutgers University
10 Seminary Place
New Brunswick, NJ 08903
phone: 732-932-7496 ext. 343
fax: 732-932-6829
Visit the GSE Website: http://www.gse.rutgers.edu
--Boundary_(ID_STJaP/EZ8K4WySIUGA/D+Q)
Date: Mon, 2 Mar 1998 15:17:33 -0600
From: Burke Johnson <BJOHNSON@USAMAIL.USOUTHAL.EDU>
Subject: Re: Correlation/causal-comparative: Reply to Burke -Reply -Reply
MIME-version: 1.0
Content-type: text/plain
Gregory Camilli points out the following:
1. Causation is neither necessary nor sufficient for observed nonzero
correlation. 2. Observed nonzero correlation is neither necessary nor
sufficient for causation.
My comment:
I totally agree with both of these statements.
Gregory also contends that the following statement is FALSE:
"A relationship of some form between events or variables A and B is
necessary but not sufficient for direct causation."
(The statement is criterion one in the commonly cited three criteria for
establishing causation: 1. a relationship of some form between two
events or variables, 2. time ordering, and 3. lack of alternative
explanations).
My comment:
I have pointed out that the term "relationship" includes but is not limited (in
my view) to a relationship as measured by a statistical index, and I totally
agree that one can be misled by simple statistical indices. The statement
says only that the two events or variables must be related in some way
(not necessarily causal) for direct causation to occur. I interpret the
statement to say that it must be theoretically possible to observe a
relationship of some form (i.e., in a world where all other factors can be
stripped away, where we have as much time as possibly needed,
where we have perfect eyesight, where we can see even causally
unconnected relationships such as an insignificant event on Mars and
one on earth, etc.). Based on yours and others comments, I have come
to view the statement in more of an ontological than methodological way.
Let's try this approach: IF THE STATEMENT IS FALSE, THEN PROVIDE
AN EXAMPLE THAT FALSIFIES IT.
Identify an example where some event is the result of another but they
are not related in any way. I don't think that an example has been
provided yet, do you?
Burke J.
_______________