7. Summary and implications
What have we learned from 20 years of CHC COG-ACH relations research? What are the implications for the changing landscape of school psychology assessment practice? We present a half dozen primary conclusions and implications (in no order of implied importance) below:
  • The current review should supplant the earlier Flanagan et al. (2006) research synthesis. The Flanagan et al. (2006) CHC COG-ACH research synthesis served a valuable function during the formative stages of bridging the gap between CHC intelligence theory and the practice of intellectual assessment.  We choose not to devote pages to detailed comparisons of the similarities and differences between the conclusions of the current review and that of Flanagan et al. (2006). The results and conclusions of the current review are intended to improve upon the specificity of the CHC COG-ACH relations literature to better inform assessment practice. The current review reveals a much more nuanced set of CHC COG- ACH relations as a function of:  (a) breadth of cognitive abilities and measures (broad vs. narrow), (b) breadth of achievement domains (e.g., basic reading skills and reading comprehension vs. broad reading), and (c) developmental (age) status. 
  • Many analyses have been completed. A large number of CHC COG- ACH analyses have been completed since the CHC model of human cognitive abilities was first operationalized in an applied intelligence battery in 1989.  Assessment professionals now have a more solid empirically-based foundation upon which to make CHC based COG- ACH related assessment decisions and interpretations. 
  • Almost all of the available research is limited exclusively to one cognitive battery. Almost all (94%) analyses have been completed with the WJ battery (WJ-R; WJ III). This is good news for the WJ III as it sits alone as the only individually administered intelligence battery with an empirical knowledge base from which to inform CHC-based assessment, diagnosis, and intervention planning. Until additional CHC COG-ACH research is completed with other (non- WJ) intelligence batteries, users of these other batteries must proceed with caution when forming COG- ACH relations-based diagnostic, interpretative, and intervention hypotheses. Given the brief evidence we presented for the lack of direct 1-1 broad or narrow CHC construct measure equivalence and predictive validity (of achievement) across measures, which is supported by Floyd, Bergeron, McCormack, Anderson and Hargrove-Owens (2005), additional CHC composite exchangeability analyses across major intelligence batteries is needed. It is recommended that independent researchers, test authors, or publishers of other CHC-based or interpreted batteries begin completing similar studies, preferably in age- differentiated subgroups of test battery standardization sample data. Studies with students with clinically diagnosed learning disabilities are also needed for all intelligence batteries.
  • The primary action is at the narrow ability level.  Our discussions of broad and narrow abilities were lengthy and difficult to write—due to the need to explain why some broad CHC abilities did not display strong relations (as expected based on theory and the prior Flanagan et al., 2006, review).  A resolution of these findings typically occurred when we examined the narrow ability results. It is our conclusion that the most important focus for CHC COG-ACH relations is at the narrow ability level. Broad CHC composites may demonstrate the best average predictive validity across a broad array of academic and non-academic criterion measures, but when attempting to understand and develop potential interventions for sub- areas of reading (e.g., word attack; sight vocabulary; reading comprehension) and math (e.g., learning math facts; solving applied math problems), narrow is better. Space does not allow for further explanation, but this finding is consistent with the classic “bandwidth-fidelity tradeoff” (Cronbach & Gleser, 1957) and the “attenuation paradox” (Boyle, 1991; Loevinger, 1954) issues in the validity and reliability measurement literature, respectively.  Broad best predicts and explains broad. Narrow best predicts and explains narrow. We believe that validated narrow cognitive ability indicators need to be the focus of assessment personnel working in the schools and should be featured in future cognitive battery test development. This conclusion was not apparent in the prior Flanagan et al. (2006) CHC COG-ACH research review.
  • œIntelligent” intelligence testing. The extant CHC COG-ACH literature, even the more conservative latent variable studies that include a general intelligence factor (g; full scale score proxy), confirms the conclusion that a number of broad and narrow CHC abilities are important above and beyond the influence of g when predicting school achievement. We believe this argues for more judicious, flexible, selective, “intelligent” (Kaufman, 1979) intelligence testing where practitioners select sets of tests most relevant to each academic referral. Unless there is a need for a full scale IQ g-score for diagnosis (e.g., MR; gifted), professionals need to break the habit of “one complete battery fits all” testing. The CHC COG-  ACH research summarized here should assist assessment professionals in making better decisions regarding which measures from an intelligence battery may provide the most diagnostic and instructionally relevant information for different academic domains at different ages or grades. Selective referral-focused assessments, with branching tree decision-rules for follow-up testing, need to be encouraged in practice and pre- and post-professional training (see McGrew, 2009 for examples). Before conducting assessments for reading and math problems, practitioners need to ask the following questions when designing their initial assessment: What is (are) the subdomain(s) of concern? What is the age of the student? What CHC abilities does research suggest are most related to this (these) domain(s) at this age level?  Our findings suggest that the design of assessments requires œintelligent” decisions that recognize CHC cognitive domain-by-achievement subdomain-by-age level interactions. Keith (1994) stated that œintelligence is important, intelligence is complex. Kaufman (1979) argued for “intelligent” intelligence testing. We agree and add that the intelligent design of individual assessments is critically important and complex (but not difficult or impossible) and must recognize the complexity of the domains of human cognitive abilities and achievements, and the nuanced differential interactions between different CHC abilities and achievement domains. The intelligent design of assessments does not come from a higher power—it comes from integrating the research synthesis presented here with professional and clinical experience.
  • There is a future for intelligence testing. We believe the current results are consistent with the call for an integration of RtI and intelligent intelligence testing (see Hale, Kaufman, Naglieri & Kavale, 2006). The cognitive markers mentioned by RtI early screening advocates correspond nicely with many CHC broad and narrow abilities identified in the current review. A variety of researchers have argued for early screening for “at risk” students based on cognitive “markers”(Berninger, 2006; Fuchs, Compton, Fuchs, Paulsen, Bryant, &Carol, 2005; Torgesen, 2002) that may be œprecursors to manifest disabilities” (Fletcher et al., 2002). In addition to the more academic variables (e.g., letter identification or knowledge of concepts of print), cognitive markers often mentioned as relevant to reading by proponents of some RtI models (Fletcher et al., 2002; Torgesen, 2002) have included (with our corresponding CHC broad or narrow ability classification) picture naming or receptive and expressive vocabulary (Gc-VL; lexical knowledge), sentence recall or verbal short-term memory (Gsm-MS; memory span), phonological awareness skills or processing (Ga-PC; phonetic coding), rapid automatic naming of objects, numbers or letters (Glr-NA; naming facility; Gs-P; perceptual speed), working memory (Gsm- MW; working memory), general oral language comprehension and development (Gc-LD: language development) and verbal knowledge (Gc- K0; general information). Although less researched, some (but not all) math cognitive markers mentioned by those supporting some RtI models (e.g, Fuchs et. al., 2005, 2006, 2008) have included efficiency of execution of cognitive tasks (Gs; processing speed), short-term memory (Gsm- MS; memory span), working memory (Gsm-MW; working memory), fluid intelligence (Gf; fluid intelligence), language ability (Gc; comprehension-knowledge), and vocabulary knowledge (Gc- VL; lexical knowledge). Interestingly, most of these reading and math markers were identified as significant COG-ACH relations in the current review. A number of today’s CHC-based intelligence batteries include reliable and valid tests that can serve as psychometrically sound markers as articulated by RtI advocates. We believe those who argue against the use of any cognitive ability tests in the new RtI environment either have: (a) failed to examine the abilities measured by many contemporary CHC intelligence batteries, (b) have not taken the time to do the RtI marker-to- CHC ability terminology “crosswalk” (as demonstrated above) or, (c) may have an agenda that is more sociopolitical than empirical.
The times and tests have changed during the past 20 years. Progress has been made in constructing more comprehensive (broader array of broad and narrow abilities sampled) CHC-based cognitive assessment batteries. Contemporary intelligence tests should be viewed as valuable tool boxes, with each tool carefully selected by intelligent craftsman to match the presenting problem. The current research synthesis provides empirical evidence to help guide assessment practices. We do not believe the current review is the end of the journey, but rather an important step toward a more complete understanding of the relationships between cognitive abilities and school achievement.