Mute those claims: No evidence (yet) for a causal link between arts study and academic achievement

Winner, E., & Cooper, M. (2000). Mute those claims: No evidence (yet) for a causal link between arts study and academic achievement. Journal of Aesthetic Education, 34(3/4), 11-75.

Abstract:

The purpose of this study was to quantify the results of existing research (from the period of 1950 to 1998) testing the claim that the study of the arts is associated with improved academic achievement. The researchers conducted five meta-analyses of studies assessing the effects of arts education on academic achievement. The studies they included in the analyses were studies that examined the effects of the arts in general rather than of specific art forms (e.g. music or dance). Three of the meta-analyses examined studies that had a correlational research design and two examined studies with an experimental design. The researchers find evidence for a positive relationship between arts education and academic achievement, but do not find evidence that arts education causes increases in verbal and mathematics achievement.

Key Findings:

The researchers’ three meta-analyses of correlational studies find a significant positive relationship between arts education and academic achievement. Due to the limitations of correlational research, however, these findings may not be interpreted to mean that arts education increases academic achievement. Correlational research does not enable causal inference and cannot rule out other explanations for an observed relationship between arts education and increased academic achievement. It remains just as likely, for example, that high academic achievers choose to study the arts as it does that studying the arts causes high academic achievement.

The researchers meta-analyses of studies with experimental designs (which allow for causal inference) do not provide evidence of an impact of arts learning on verbal and mathematics achievement.

Significance of the Findings:

These findings point to the need for further experimental research, employing random assignment to treatment and control groups, to examine transfer of arts education to other academic domains. Ideally, control group participants would be given a non-arts treatment. New research should also address the issue of assessing divergent thinking, rather than rely upon standardized tests and explicit teaching for transfer. Finally, as the authors contend, any evaluation of the educational outcomes of arts education should be based on learning in the arts.

Methodology:

The meta-analyses involved three basic steps: (1) a complete literature search for all possible studies, both published and unpublished, in the defined population which yielded 1,135 research records, of which 44 studies were potentially relevant and 31 selected for inclusion; (2) the relevant characteristics and results of the studies were identified and categorized; and (3) the data/outcomes were converted to 66 effect size measures. The researchers conducted three separate meta-analyses of the correlational studies they identified, which examined the impact of arts education on: (1) a combined measure of verbal and mathematics achievement, (2) verbal skills, and (3) mathematics skills. They also conducted two meta-analyses on the experimental studies they identified. These analyses examined the effects of arts education on: (1) math skill, and (2) verbal achievement.

Limitations of the Research:

The basic limitations of this research are those generally associated with the use of meta-analytic techniques—meta-analyses combine data from different studies conducted under varying circumstances; all studies may not provide sufficient data to calculate effect sizes; and meta-analyses perpetuate an inherent bias toward published articles because researchers may fail to uncover unpublished studies. In this research, the latter limitation was mitigated by a particularly exhaustive search for unpublished research. A limitation specific to this research is that the researchers were only able to identify two studies that employed a true experimental design using random assignment that were pertinent to the research questions set out for their meta-analyses.

Questions to Guide New Research:

Is there a threshold effect, a certain amount or dosage of arts education, that is necessary for transfer to occur? Is there a link between motivational outcomes (improved attitude toward school or improved academic self-concept), academic outcomes, and arts education? Do studies relying on measures of divergent thinking, rather than standardized tests, demonstrate a causal relationship between arts education and academic development? Is there a transfer effect if teachers explicitly teach for transfer to the new domain? Can the study of the arts promote a way of thinking, or a way of working, that can be applied to other academic areas?