What have we learned from 20 years of CHC COG-ACH
relations research? What are the implications for the changing
landscape of school psychology assessment practice? We present a
half dozen primary conclusions and implications (in no order of
implied importance) below:
-
The current review should supplant the earlier Flanagan et al.
(2006) research synthesis. The Flanagan et al. (2006) CHC
COG-ACH research synthesis served a valuable function during the
formative stages of bridging the gap between CHC intelligence
theory and the practice of intellectual assessment. We choose
not to devote pages to detailed comparisons of the similarities and
differences between the conclusions of the current review and that
of Flanagan et al. (2006). The results and conclusions of the
current review are intended to improve upon the specificity of the
CHC COG-ACH relations literature to better inform assessment
practice. The current review reveals a much more nuanced set of CHC
COG- ACH relations as a function of: (a) breadth of cognitive
abilities and measures (broad vs. narrow), (b) breadth of
achievement domains (e.g., basic reading skills and reading
comprehension vs. broad reading), and (c) developmental (age)
status.
-
Many analyses have been completed. A large number of CHC
COG- ACH analyses have been completed since the CHC model of human
cognitive abilities was first operationalized in an applied
intelligence battery in 1989. Assessment professionals now
have a more solid empirically-based foundation upon which to make
CHC based COG- ACH related assessment decisions and
interpretations.
-
Almost all of the available research is limited exclusively to
one cognitive battery. Almost all (94%) analyses have been
completed with the WJ battery (WJ-R; WJ III). This is good news for
the WJ III as it sits alone as the only individually administered
intelligence battery with an empirical knowledge base from which to
inform CHC-based assessment, diagnosis, and intervention planning.
Until additional CHC COG-ACH research is completed with other (non-
WJ) intelligence batteries, users of these other batteries must
proceed with caution when forming COG- ACH relations-based
diagnostic, interpretative, and intervention hypotheses. Given the
brief evidence we presented for the lack of direct 1-1 broad or
narrow CHC construct measure equivalence and predictive validity
(of achievement) across measures, which is supported by Floyd,
Bergeron, McCormack, Anderson and Hargrove-Owens (2005), additional
CHC composite exchangeability analyses across major intelligence
batteries is needed. It is recommended that independent
researchers, test authors, or publishers of other CHC-based or
interpreted batteries begin completing similar studies, preferably
in age- differentiated subgroups of test battery standardization
sample data. Studies with students with clinically diagnosed
learning disabilities are also needed for all intelligence
batteries.
-
The primary action is at the narrow ability level. Our
discussions of broad and narrow abilities were lengthy and
difficult to write—due to the need to explain why some broad
CHC abilities did not display strong relations (as expected based
on theory and the prior Flanagan et al., 2006, review). A
resolution of these findings typically occurred when we examined
the narrow ability results. It is our conclusion that the most
important focus for CHC COG-ACH relations is at the narrow ability
level. Broad CHC composites may demonstrate the best average
predictive validity across a broad array of academic and
non-academic criterion measures, but when attempting to understand
and develop potential interventions for sub- areas of reading
(e.g., word attack; sight vocabulary; reading comprehension) and
math (e.g., learning math facts; solving applied math problems),
narrow is better. Space does not allow for further explanation, but
this finding is consistent with the classic
“bandwidth-fidelity tradeoff” (Cronbach & Gleser,
1957) and the “attenuation paradox” (Boyle, 1991;
Loevinger, 1954) issues in the validity and reliability measurement
literature, respectively. Broad best predicts and explains
broad. Narrow best predicts and explains narrow. We believe that
validated narrow cognitive ability indicators need to be the focus
of assessment personnel working in the schools and should be
featured in future cognitive battery test development. This
conclusion was not apparent in the prior Flanagan et al. (2006) CHC
COG-ACH research review.
-
œIntelligent” intelligence testing. The extant
CHC COG-ACH literature, even the more conservative latent variable
studies that include a general intelligence factor (g; full scale
score proxy), confirms the conclusion that a number of broad and
narrow CHC abilities are important above and beyond the influence
of g when predicting school achievement. We believe this argues for
more judicious, flexible, selective, “intelligent”
(Kaufman, 1979) intelligence testing where practitioners select
sets of tests most relevant to each academic referral. Unless there
is a need for a full scale IQ g-score for diagnosis (e.g., MR;
gifted), professionals need to break the habit of “one
complete battery fits all” testing. The CHC COG- ACH
research summarized here should assist assessment professionals in
making better decisions regarding which measures from an
intelligence battery may provide the most diagnostic and
instructionally relevant information for different academic domains
at different ages or grades. Selective referral-focused
assessments, with branching tree decision-rules for follow-up
testing, need to be encouraged in practice and pre- and
post-professional training (see McGrew, 2009 for
examples). Before conducting assessments for reading and math
problems, practitioners need to ask the following questions when
designing their initial assessment: What is (are) the subdomain(s)
of concern? What is the age of the student? What CHC abilities does
research suggest are most related to this (these) domain(s) at this
age level? Our findings suggest that the design of
assessments requires œintelligent” decisions that
recognize CHC cognitive domain-by-achievement subdomain-by-age
level interactions. Keith (1994) stated that œintelligence is
important, intelligence is complex. Kaufman (1979) argued for
“intelligent” intelligence testing. We agree and add
that the intelligent design of individual assessments is critically
important and complex (but not difficult or impossible) and must
recognize the complexity of the domains of human cognitive
abilities and achievements, and the nuanced differential
interactions between different CHC abilities and achievement
domains. The intelligent design of assessments does not come from a
higher power—it comes from integrating the research synthesis
presented here with professional and clinical experience.
-
There is a future for intelligence testing. We believe the
current results are consistent with the call for an integration of
RtI and intelligent intelligence testing (see Hale, Kaufman,
Naglieri & Kavale, 2006). The cognitive markers mentioned by
RtI early screening advocates correspond nicely with many CHC broad
and narrow abilities identified in the current review. A variety of
researchers have argued for early screening for “at
risk” students based on cognitive
“markers”(Berninger, 2006; Fuchs, Compton, Fuchs,
Paulsen, Bryant, &Carol, 2005; Torgesen, 2002) that may be
œprecursors to manifest disabilities” (Fletcher et al.,
2002). In addition to the more academic variables (e.g.,
letter identification or knowledge of concepts of print), cognitive
markers often mentioned as relevant to reading by proponents of
some RtI models (Fletcher et al., 2002; Torgesen, 2002) have
included (with our corresponding CHC broad or narrow ability
classification) picture naming or receptive and expressive
vocabulary (Gc-VL; lexical knowledge), sentence recall or verbal
short-term memory (Gsm-MS; memory span), phonological awareness
skills or processing (Ga-PC; phonetic coding), rapid automatic
naming of objects, numbers or letters (Glr-NA; naming facility;
Gs-P; perceptual speed), working memory (Gsm- MW; working memory),
general oral language comprehension and development (Gc-LD:
language development) and verbal knowledge (Gc- K0; general
information). Although less researched, some (but not all) math
cognitive markers mentioned by those supporting some RtI models
(e.g, Fuchs et. al., 2005, 2006, 2008) have included efficiency of
execution of cognitive tasks (Gs; processing speed), short-term
memory (Gsm- MS; memory span), working memory (Gsm-MW; working
memory), fluid intelligence (Gf; fluid intelligence), language
ability (Gc; comprehension-knowledge), and vocabulary knowledge
(Gc- VL; lexical knowledge). Interestingly, most of these reading
and math markers were identified as significant COG-ACH relations
in the current review. A number of today’s CHC-based
intelligence batteries include reliable and valid tests that can
serve as psychometrically sound markers as articulated by RtI
advocates. We believe those who argue against the use of any
cognitive ability tests in the new RtI environment either have: (a)
failed to examine the abilities measured by many contemporary CHC
intelligence batteries, (b) have not taken the time to do the RtI
marker-to- CHC ability terminology “crosswalk” (as
demonstrated above) or, (c) may have an agenda that is more
sociopolitical than empirical.
The times and tests have changed during the past
20 years. Progress has been made in constructing more comprehensive
(broader array of broad and narrow abilities sampled) CHC-based
cognitive assessment batteries. Contemporary intelligence tests
should be viewed as valuable tool boxes, with each tool carefully
selected by intelligent craftsman to match the presenting problem.
The current research synthesis provides empirical evidence to help
guide assessment practices. We do not believe the current review is
the end of the journey, but rather an important step toward a more
complete understanding of the relationships between cognitive
abilities and school achievement.
|
|
1. Introduction
