Dbq775 | Education homework help (2024)

HANDBOOK OF EDUCATIONAL PSYCHOLOGY The third edition of the Handbook of Educational Psychology is sponsored by Division 15 of the AmericanPsychological Association. In this volume, 30 chapters address new developments in theory and researchmethods while honoring the legacy of the field’s past. A diverse group of recognized scholars within andoutside the United States provide integrative reviews and critical syntheses of developments in the substantiveareas of psychological inquiry in education, functional processes for learning, learner readiness anddevelopment, building knowledge and subject matter expertise, and the learning and task environment. Newchapters in this edition cover topics such as learning sciences research; latent variable models; data analytics;neuropsychology; relations between emotion, motivation, and volition (EMOVO); scientific literacy;sociocultural perspectives on learning; dialogic instruction; and networked learning. Expanded treatment hasbeen given to relevant individual differences, underlying processes, and new research on subject matteracquisition.

The Handbook of Educational Psychology, Third Edition, provides an indispensable reference volume forscholars in education and the learning sciences, broadly conceived, as well as for teacher educators, practicingteachers, policy makers, and the academic libraries serving these audiences. It is also appropriate for graduate-level courses in educational psychology, human learning and motivation, the learning sciences, andpsychological research methods in education and psychology.

Lyn Corno is former Professor of Education and Psychology (retired) at Teachers College, ColumbiaUniversity, USA, and co-Editor of Teachers College Record and the National Society for the Study ofEducation Yearbooks.

Eric M. Anderman is Professor of Educational Psychology and Chair of the Department of EducationalStudies at The Ohio State University, USA.

HANDBOOK OFEDUCATIONAL PSYCHOLOGY

Third Edition

Edited byLyn Corno and Eric M. Anderman

Third edition published 2016by Routledge711 Third Avenue, New York, NY 10017

and by Routledge2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN

Routledge is an imprint of the Taylor & Francis Group, an informa business

The right of the editors to be identified as the authors of the editorial material, and of the authors for their individual chapters, has beenasserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or othermeans, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, withoutpermission in writing from the publishers.

Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanationwithout intent to infringe.

First edition published by Routledge 1996Second edition published by Routledge 2006

Library of Congress Cataloging in Publication DataA catalog record has been requested

ISBN: 978-0-415-89481-4 (hbk)ISBN: 978-0-415-89482-1 (pbk)ISBN: 978-1-315-68824-4 (ebk)

Typeset in Timesby Swales & Willis Ltd, Exeter, Devon, UK

With gratitude for his gracious guidance over many years, we dedicate this volume to the memory of Robert C.Calfee.

10.

Contents

Foreword: Capturing the Landscape of Educational Psychology TodayLyn Corno and Eric M. Anderman

AcknowledgmentsAdvisory Board MembersList of Reviewers

Part I. Psychological Inquiry in Education

Philosophical Perspectives on Mind, Nature, and Educational PsychologyEric Bredo

Modes of Inquiry in Educational Psychology and Learning Sciences ResearchWilliam R. Penuel and Kenneth A. Frank

The Work of Educational Psychologists in a Digitally Networked WorldPunya Mishra, Matthew J. Koehler, and Christine Greenhow

The Prospects and Limitations of Latent Variable Models in Educational PsychologyBenjamin Nagengast and Ulrich Trautwein

Part II. Functional Processes for Learning

Learning as Coordination: Cognitive Psychology and EducationDaniel L. Schwartz and Robert Goldstone

Emotions and Emotion Regulation in Academic SettingsMonique Boekaerts and Reinhard Pekrun

MotivationLisa Linnenbrink-Garcia and Erika A. Patall

VolitionGabriele Oettingen, Jana Schrage, and Peter M. Gollwitzer

Part III. Learner Readiness and Development

Human Cognitive Abilities: Their Organization, Development, and UsePatrick C. Kyllonen

Cognition and Cognitive Disabilities

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

H. Lee Swanson

Personal Capability BeliefsEllen L. Usher

Motivation Interventions in Education: Bridging Theory, Research, and PracticeChris S. Hulleman and Kenn E. Barron

Beyond the Shadow: The Role of Personality and Temperament in LearningArthur E. Poropat

Cultural, Racial/Ethnic, and Linguistic Diversity and IdentityNa’ilah Suad Nasir, Stephanie J. Rowley, and William Perez

Language DevelopmentAlison L. Bailey, Anna Osipova, and Kimberly Reynolds Kelly

Character Education, Moral Education, and Moral-Character EducationCary J. Roseth

Part IV. Building Knowledge and Subject Matter Expertise

Literacy for Schooling: Two-Tiered Scaffolding for Learning and TeachingIan A. G. Wilkinson and Janet S. Gaffney

Warm Change about Hot Topics: The Role of Motivation and Emotion in Attitude and ConceptualChange about Controversial Science TopicsGale M. Sinatra and Viviane Seyranian

Toward an Educational Psychology of Mathematics EducationJon R. Star and Bethany Rittle-Johnson

Functional Scientific Literacy: Seeing the Science within the Words and Across the WebIris Tabak

Studying Historical UnderstandingChauncey Monte-Sano and Abby Reisman

Civic EducationMario Carretero, Helen Haste, and Angela Bermudez

Part V. The Learning and Task Environment

Sociocultural Perspectives on Literacy and LearningDavid O’Brien and Theresa Rogers

24.

25.

26.

27.

28.

29.

30.

II.

Learning Environments In and Out of SchoolBrigid Barron and Philip Bell

Networked LearningGary Natriello

Collaborative LearningCindy E. Hmelo-Silver and Clark A. Chinn

Black and Hispanic Students: Cultural Differences within the Context of EducationDonna Y. Ford

Dialogic Instruction: A New FrontierSherice N. Clarke, Lauren B. Resnick, and Carolyn Penstein Rosé

Assessment Illuminating Pathways to LearningEllen B. Mandinach and Andrea A. Lash

Being a Teacher: Efficacy, Emotions, and Interpersonal Relationships in the ClassroomLynley H. Anderman and Robert M. Klassen

Afterwords

Perspectives on the Past, Present, and Future of Educational PsychologyFrank Farley, Patricia A. Alexander, Eva L. Baker, David C. Berliner, Robert C. Calfee, Erik De Corte, JamesG. Greeno, Anita Woolfolk Hoy, and Richard E. Mayer

On Impact Beyond the FieldEric M. Anderman and Lyn Corno

Contributor BiosAuthor IndexSubject Index

ForewordCapturing the Landscape of Educational Psychology Today

LYN CORNO

Teachers College, Columbia University

ERIC M. ANDERMAN

The Ohio State University With this third edition of the Handbook of Educational Psychology, we hope to move the field forward at thesame time that we honor its history and legacy of years past. Today the field is not being redefined byindividual scholars studying general tendencies within groups or current movements in educational researchand policy, but by a growing understanding of what underlies learning and development through education.Over time, conducting meticulously planned studies of pointed research questions that build on one anotherhas provided the recipe for growth, changing even some historic markers in the field. For example, in someareas of educational psychology, there is now an established relationship between data and theory that isleading to a new practice of science marked by modeling and visualization rather than hypothesis testing. Inother areas, the data–theory relationship is informed by biography or narrative-experiential history. A thirdHandbook of Educational Psychology needs to support this changing landscape. The field no longer offers alimited menu; today it is more like a fully sustaining farm.

Structure of the Volume

To shape the volume, we conducted a content analysis of the two previous editions of this Handbook(Alexander & Winne, 2006; Berliner & Calfee, 1996), alongside the table of contents for the three-volumeEducational Psychology Handbook published by the American Psychological Association (Harris, Graham, &Urdan, 2012). This analysis led to a structure aligned with the previous editions of Division 15’s Handbook,and some similar sections. It also allowed us to assess potential problems with overlapping content across theseveral different volumes.

We formed an advisory board to help us to think about topic coverage for the third edition. The advisoryboard members included a range of scholars representing diversity in expertise. These advisors helped us toidentify salient topics, controversial issues, and new trends in the field. In addition, they assisted us inidentifying scholars who might provide innovative perspectives and voices.

Prior editions of this Handbook included chapters on the history of educational psychology along with itsunderlying theory, research foci, and traditional methods of investigation. The authors of those chaptersdiscussed central topics in learning and teaching within and across subject matter areas, as well as differenttypes of students and educational contexts. For the present volume, we needed to make sense of the fact that arevolution is occurring in learning science and technology at the same time that educational research is morestrongly than ever embracing the gold standards of longitudinal and experimental design, buttressed bysystematic reviews of such trials. Herein, we sought to feature cutting-edge research across a wide range ofconceptual approaches and topics.

We felt the need for fresh thinking in this Handbook, and so we decided to invite chapter authors who hadmade limited or no contributions to previous Division Handbooks. We also purposely invited some authorswhose expertise is in disciplines other than educational psychology, because they are doing work that isbecoming critical to our field. In addition, our publishing contract for this volume required relatively shortchapters, so we requested that authors keep the number of co-authors to a maximum of two, and asked thatthey select co-authors who were established scholars. We reasoned that fewer, more experienced authors couldproduce timely chapters of good density and breadth given the page length constraints. Thus, we choseauthors and topics, hoping to produce a volume that spoke to the factors making this era in our field such anexciting one.

Five Research Domains

What seems clear from the chapters that comprise this volume is that each reworking of a research domainserves a particular function. Accordingly, the five domains included in the third edition illustrate: (a) new waysthat we conduct our work and the assumptions that underlie them; (b) growing understandings of thefundamental processes that drive outcomes; (c) how best to conceptualize the critical qualities of learners andtheir development; (d) the combinations and sequences of instructional activities that lead to acquisition ofsubject matter knowledge and skill; and (e) the key role of contextual factors in education as a human culturalpractice.

Psychological inquiry in education. The first seeds, presented in our section on inquiry, generate an explication

of some philosophically deep issues at the heart of education as a cooperative endeavor of humans and theirenvironment. Neuroscience now shows that brain activity and circuits actually change with repeatedsituational transactions, providing justification for an evolutionary account of human reflective behavior drivenin part by education. The chapters in this section illustrate the sharp research questions and correspondingmethods of investigation that underlie dynamic, process-level theories and complex procedures for analyzingnested evidence. These chapters also address key measurement issues; they reflect the richness that is obtainedthrough qualitative description; and they illustrate how disaggregation can be enhanced by modern digitaltechnologies and data analytics. Finally, the value of multivariate and multimeasure longitudinal studies,which permit efficient assessment of change over a number of spaced occasions, is made clear; these methodsalso allow for the assessment of individual differences in change.

Functional processes for learning. The second section of the volume, on functional processes, provides amodern perspective on the ancient triumvirate of cognition, affection, and conation (Hilgard, 1980). Thesem*ntal processes all interplay with the tasks and events of learning and teaching. The chapter on cognitionemphasizes the role of coordination in student learning and decision making, including involuntary processesand functional changes being studied using neuroimaging techniques. It shows how the human attentionsystem helps to regulate and transform learner perceptions into short- and long-term cognitive-intellectualresources.

The three chapters on emotion, motivation, and volition together capture qualities that we might callEMOVO, for short. These are qualities that students bring into educational situations, but that also ariseduring completion of tasks and activities to either threaten or invigorate learning and productive follow-through. Studies of emotions, moods, attitudes, and physiological factors such as stress and flow states haveled to important self-regulation interventions across school and other life experiences (e.g., relationships,addictive behavior). In the long run, the study of EMOVO promises an integrated path to understanding avariety of psychological phenomena that have historically been examined in isolation from one another.EMOVO qualities are proving to be: (a) reliably measured with innovative indices; (b) influenced in somecases through relatively simple, priming-based interventions or curriculum-embedded instruction; and (c)demonstrably important to successful learning and engagement. Like some critical intellectual abilities,EMOVO sensibilities can also be fostered or wither, depending on social opportunities and student abilitiesto modulate them. Increasingly, researchers are also gaining insight into the neural basis of how habitualactions such as goal commitment or procrastination can alter an individual’s EMOVO for better or worse.

Learner readiness and development. The group of chapters under the section on readiness and developmentexamines general cognitive and language abilities and disabilities, as well as beliefs, personality, andbiographical characteristics of students, to inform how all of these variables influence motivational, linguistic,and moral development in schools. Person–environment interactions and modes of assessment contribute toindividual differences in cognitive functioning, as well as to moral and identity trajectories, but so dodifferences in brain memory processes and sensitivities to cues. Studies of developmental-relational systemsprovide strong evidence that many of the readiness qualities, including those initially conceived as trait-like,are best conceptualized as dynamically malleable within specific ecologies. Carefully designed interventionenvironments offer trusting affordances for skill or habit development as well as behavior change in even very

young children. Targeted and multicomponent programs can lead to demonstrable payoffs that grow overtime.

Building knowledge and subject matter expertise. The fourth part on knowledge and subject matter acquisitionpresents six chapters addressing the content areas of academic and social literacy, mathematics, science,history, and civics, respectively. Each of these chapters discusses new questions and methods for teachingpractice and subject matter instruction that are informed by cognitive and sociocultural theory. Literacyinterventions conducted in schools and classrooms can improve student knowledge in K-12 reading, writing,and use of oral language. Informal experiences in early childhood set the stage for thinking and learning inmathematics and science tasks that increasingly benefit from computer-guided inquiry. With controversialtopics, such as scientific debates, there are new techniques for influencing student attitudes and conceptualunderstandings. Likewise, an understanding of history can grow from reflection on students’ own experiencesas well as their interpretations of the past. Many subject matter gains reflect domain-dependent individualpractice, such as prompts for students to self-explain. These chapters show that deliberate practice is affordedby structured interventions designed to promote and accelerate complex learning.

The learning and task environment. Finally, Part V’s chapters consider the environment for learning in thelarger sense. These chapters discuss spaces where children actually live and work, both in and beyond school.Digital learning communities and after-school programs now supplement and complement projects in schoolsettings. The chapters bring us back to the critical importance of context by capturing the movement awayfrom the narrow focus on learning tasks in classrooms and labs that used to be routine in educationalpsychology. After the methodological controversies of recent decades, the landscape today reflects importantsociocultural principles. These authors are reworking the assumptions guiding earlier investigations, andsuggesting ways to address and embrace more appropriate theory for subsequent generations.

To take one example, this shifting paradigm is present in modern research on academic discourse inclassrooms where instruction is considered as dialogue, argumentation, and problem-based learning. Newtechnology also offers affordances for fixing problems (e.g., how teachers can reach all students in a class; howresearchers can collect real-time data to assess student learning). The internet can advantage learning ifstudents are using it for web-mediated knowledge synthesis. Students can profitably use the internet as well inways that make them think differently about educational communities, or for social purposes and fun that canincrease engagement with academic content. Counter-narrative, qualitative cultural studies add a critical layerinto this soil, calling into question untenable assumptions and methodological arrogance. Cultural values andpractices that surround learners during their development become enmeshed in brain activity that affectsbehavior. If poverty at home subverts school learning, then other factors in the system need to change.

In addition to heightening awareness of subtle stereotypes and other unintentional behavior hoveringbeneath the surface, cultural studies in education highlight the important role of social perspective taking inteacher–student relationships. They illustrate how cultivating an understanding of difference among thescholarly community further benefits the system. Only with deeper plantings like this does a research terrainfinally become loaded with the requisite resources needed for knowledge to grow.

An Ever-Changing Ground

As much as people may like to cling to established research principles and routines, we find it noteworthy thatour authors in this Handbook do not. Following on what their experiences produce, these scholars roll outentirely different prototypes for future work. If there is a problem with evidence in the field, then a newgeneration of research will investigate how to tighten or calibrate it, followed by how to extend and thencatalog it. Theory built from interpretive analyses of large quantities of catalogued databases now at thefingertips of research scientists is in fact among the current trends. The technological revolution makes itpossible for psychologists to capture some broader outcomes of education from the details of millions ofindividual students. Once again, the landscape is ready for new growth.

To edit of course is to collect and organize, as well as to gauge and guide the writing of others. It is also, wehave realized, to make material more readable and, one hopes, engaging. We have to offer what the scienceshows, even if this is not always material that translates easily to practical circ*mstances in education settings.Additionally, instead of including a conventional chapter on the history of educational psychology at thebeginning of our volume, we invited veteran educational psychologist Frank Farley to be innovative and craft aforward-looking chapter on its history that would appear at the end. This chapter, presented in theAfterword, provides a compilation of reflective commentaries from eight outstanding leaders of Division 15and our field more broadly. In these essays, the authors consider the harvest: what are the most importantaccomplishments and major failures of educational psychology during their own careers? These leaders alsocomment on the current status of educational psychology and where they feel we are and should be headed inthe future.

We are particularly pleased to include the essay by Robert C. Calfee, entitled “Back to the Future: Learningand Transfer Redux,” which is being published posthumously; Bob passed away on October 24, 2014. Wehave dedicated this Handbook to his memory—he was and will continue to be a revered mentor to many of uswriting in these pages. Knowing Bob, we feel he would concur with our final hope—that, overall, this volumeof the Handbook conveys a deep respect for our complex and varied, never self-satisfied field.

References

Alexander, P.A., & Winne, P. H. (Eds.) (2006). Handbook of Educational Psychology (2nd Ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Berliner, D.C., & Calfee, R. C. (Eds.) (1996). Handbook of Educational Psychology. New York, NY: Macmillan.Harris, K. R., Graham, S., & Urdan, T. (Eds.) (2012). Educational Psychology Handbook, Vols. 1-3. Washington, D.C.: American Psychological

Association.Hilgard, E. R. (1980). The trilogy of the mind: Cognition, affection, and conation. Journal of the History of Behavioral Sciences, 16, 107–117.

Acknowledgments To say thank you for the fine counsel and dedicated time provided by our board of advisors is insufficient butheartfelt.

Advisory Board Members

Philip Ackerman, Georgia Institute of TechnologyCarole Ames, Michigan State UniversityMimi Bong, Korea University, South KoreaEric Bredo, University of Toronto, CanadaMartha Carr, University of GeorgiaRobert Floden, Michigan State UniversityDonna Y. Ford, Vanderbilt UniversityJohn Hattie, University of Auckland, New ZealandRobert Klassen, University of Alberta, CanadaJonna M. Kulikowich, Pennsylvania State UniversitySusanne P. Lajoie, McGill University, CanadaMarcia C. Linn, University of California, BerkeleyRichard E. Mayer, University of California, Santa BarbaraMary M. McCaslin, University of ArizonaP. David Pearson, University of California, BerkeleyReinhard Pekrun, University of Munich, GermanyGale M. Sinatra, University of Southern CaliforniaSam Wineberg, Stanford University

We are also indebted to Trevor Gori, Rebecca Novak, Lane Akers, and the production editing team at Taylor& Francis/Routledge who worked closely with us to ensure quality in publication.

In the end, the authors and reviewers are what make a Handbook what it is and can be for posterity. Thosewriting in the pages that follow, as well as the reviewers who graciously read their work and gave feedback, arenot only among the finest scholars in the field, but also the kind of people who make editing satisfying as wellas worthwhile.

List of Reviewers

Philip Ackerman, Georgia Institute of TechnologyDonna E. Alvermann, The University of GeorgiaTerry Au, University of Hong Kong, Hong KongJürgen Baumert, Max Planck Institute for Human Development, GermanyDaphne Bevalier, University of RochesterBronwyn Bevan, ExploratoriumDerek Briggs, University of ColoradoBertram (Chip) Bruce, University of Illinois, Urbana-ChampaignMichelle M. Buehl, George Mason UniversityBrian Carolan, Montclair State UniversityClark A. Chinn, Rutgers UniversityJulie Coiro, University of Rhode IslandCarol Connor, Arizona State UniversityHeather A. Davis, North Carolina State College of EducationAngela L. Duckworth, University of PennsylvaniaRichard Duschl, Pennsylvania State UniversityJacquelynne Eccles, University of California IrvineChristina E. Erneling, Lund University, SwedenConstance Flanagan, University of Wisconsin MadisonKimberley Freeman, Howard UniversityRichard Gilman, Cincinnati Children’s HospitalChristine Greenhow, Michigan State UniversityJan-Eric Gustafsson, University of Gothenberg, SwedenGeneva Haertal, SRI InternationalRogers Hall, Vanderbilt UniversityKaren Harris, Arizona State UniversityJoan Herman, University of California, Los AngelesAvi Kaplan, Temple UniversityA. Eamonn Kelly, George Mason UniversityKenneth R. Koedinger, Carnegie Mellon UniversityRichard Lerner, Tufts UniversityTzu-Jung Lin, The Ohio State UniversityDavid Lubinski, Vanderbilt UniversityAndrew Martin, University of Sydney, AustraliaLucia Mason, University of Padova, ItalyAllison McCabe, University of Massachusetts LowellCarolyn MacCann, The University of Sydney, AustraliaDebra Meyer, Elmhurst College

James Moore, The Ohio State UniversityKaren Murphy, Pennsylvania State UniversityDarcia F. Narvaez, University of Notre DameHarry O’Neil, University of Southern CaliforniaDavid R. Olson, University of Toronto, CanadaAllison Ryan, The University of MichiganLeona Schauble, Vanderbilt UniversityDale H. Schunk, University of North Carolina at GreensboroSimone Schweber, University of Wisconsin MadisonPeter Sexias, University of British Columbia, CanadaFinbarr (Barry) Sloane, National Science FoundationJohn Sweller, University of New South Wales, AustraliaNicole Patton Terry, Georgia State UniversitySharon Tettegah, University of Illinois, Urbana-ChampaignTim Urdan, Santa Clara UniversityLieven Verschaffel, Katholieke Universiteit Leuven, BelgiumNoreen M. Webb, University of California, Los AngelesAllan Wigfield, University of MarylandLouise Wilkinson, Syracuse UniversityPhilip H. Winne, Simon Fraser UniversityFrank Worrell, University of California BerkeleyJonathan Zaff, Tufts University and Center for PromiseMoshe Zeidner, University of Haifa, Israel

Part IPsychological Inquiry in Education

1Philosophical Perspectives on Mind, Nature, and

Educational PsychologyERIC BREDO1

University of Toronto, Canada This chapter opens the Handbook, and provides a reflective, philosophical discussion intended to inform theeducational psychology emerging today. To gain perspective on current theory and research in the field it ishelpful to consider some of the basic ways that human nature and conduct are conceived. Since the most basicimages of human nature and conduct often have earlier historical origins as parts of larger visions of mind andnature that were first articulated by philosophers, perspective on present and emerging thought can often begained by considering it in the context of earlier ways of thinking.

The sections that follow begin with a discussion of what a philosophical perspective might mean topsychological research in education, and why an educational psychologist should care about a philosophicalconsideration of the field. Three different ways of thinking about mind and nature that address this issue arediscussed and related to vitalistic, mechanistic, and evolutionary ways of thinking represented, iconically, inthe work of Aristotle, Newton, and Darwin. The perspectives toward mind and nature that these three greatrevolutions in thought introduced bear on how contemporary educational psychologists conceptualize humannature and conduct. Some implications of these different conceptions for education are considered in theconclusions.

Philosophical Perspectives

The way an issue is conceptualized is usually the most important step in dealing with it. The first stroke onthe canvas, or the first cut of the marble, affects everything that comes later. The same is true of an initialchoice of conception, model, metaphor, or image in psychology. As William James wrote:

It is astonishing what havoc is wrought in psychology by admitting at the outset apparently innocent suppositions, that nevertheless containa flaw. The bad consequences develop . . . and are irremediable, being woven through the whole texture of the work. (James 1890/1952, p.146)

Despite the importance of the way an issue is first conceived, it is often difficult to correct poorconceptualizations because the basic distinctions involved in them are so familiar that they are not recognizedas choices that are open to revision (Wittgenstein, 1958). One of the aims of philosophy is to make these basicconcepts or distinctions more visible and open to conscious choice so that we do not become trapped inintellectual cages of our own construction.2 Isaiah Berlin articulated this task well:

The task of philosophy . . . is to extricate and bring to light the hidden categories and models in terms of which human beings think (thatis, their use of words, images and other symbols), to reveal what is obscure or contradictory in them, to discern the conflicts between themthat prevent the construction of more adequate ways of organizing and describing and explaining . . . and then, at a still “higher” level, toexamine the nature of this activity itself . . . and to bring to light the concealed models that operate in this second-order, philosophical,activity itself. (Berlin, 1939/1999, p. 10)

As this description suggests, there are both critical and constructive aspects of this effort, philosophyattempting to critically appraise different conceptualizations, and to suggest better, less problematic ones. Indoing so the aim is to develop “as unified, consistent, and complete an outlook upon experience as is possible”(Dewey, 1916b, p. 378). Let me begin with the critical aspect.

Prior Criticisms

In the first edition of this Handbook, written now 20 years ago, Denis Phillips noted three lines ofphilosophical criticism of then prominent thinking in educational psychology. These criticisms suggested thateducational psychologists had a tendency: (a) to derive recommendations for educational practice too directlyfrom psychological theories or generalizations; (b) for conceptual confusion and lack of conceptual clarity; and(c) to adopt psychological conceptions that fail to “do justice to human experience,” and depict “people asbeing, in some ways, sub-human” (Phillips, 1996, p. 1010). My own chapter in the second edition of theHandbook, written ten years later, elaborated on the second of these issues (Bredo, 2006a).

The first criticism was directed against the notion that educational practice can be directly deduced frompsychological laws or generalizations. The notion that finding “what works” will tell us how to educate, or that“education science” will do the same, are examples. The popular notion that knowledge of the brain will tellone how to teach is another. There are at least two problems with this notion. First, knowledge of what maybe effective does not speak to what is desirable, just as a street map does not determine where it is good to go.Second, theory is general but practical situations are unique. As a result the practical consequences of actingon a theory depend on many considerations that can never be fully stated in the theory. Theoreticalknowledge is not the same as practical wisdom.

The second line of criticism points to instances of conceptual confusion or lack of conceptual clarity. Oneform of conceptual confusion involves misplaced concreteness or reification (Ryle, 1949). The fact that theword “mind” is a noun often leads to thinking of it as a thing. Conceived this way, the inquirer naturally wantsto know where it is located, leading to the effort to locate the mind in the brain (for a critique, see Bakhurst,2008). Since it is impossible to find the mind as a whole in the brain as a whole, the effort then shifts tofinding specific mental functions in specific brain regions, as in the notion that there are different mental“modules” located in different brain regions (see Chapter 5, this volume). Such attempts at spatial localizationinevitably fail, however, because functions are relationships, and relationships have no location.3 To take a lessmystified example, the heart is vital to pumping blood and oxygen to other organs, but a heart beating withoutregard to the state of these other organs results in death, not proper functioning. Functioning is not in theheart since functioning involves coordination and coordination is not “in” anything. This issue is practicallyimportant because improperly locating a cause can lead to misguided treatment, such as focusing on an organor individual in isolation, while ignoring the relationships in which they participate (e.g., Watzlawick, Beavin,& Jackson, 1967).

Another form of conceptual error occurring in educational psychology and other fields is the tendency totreat a name for a form of behavior as its cause. Viewing a person’s angry behavior as caused by “anger,” orinattention as caused by “attention deficit hyperactivity disorder,” are examples. When terms for a pattern ofbehavior are treated as causal explanations of the same behavior, they clearly fail to explain anything. A closelyrelated problem occurs when psychological concepts useful for understanding or explaining behavior, such ascognitive rules, schemata, and structures, are imputed to the individuals themselves, as though the individualsused the same rules or structures as the psychologist does in accounting for their behavior (Phillips, 1987c).This tendency has been so common in psychology that James termed it the “psychologist’s fallacy” (James,1890/1952, p. 128). Yet another conceptual error involves confusing a representation with reality. Stated more

carefully, this involves confusing a representation with the representation (Goodman, 1972). Dewey termedthis the “philosophical fallacy” (Dewey, 1929/1958), but it is also common in other fields, includingeducational psychology. The notion that nature is basically or essentially mechanical is an example, of whichmore will be said later.

This brings us to Phillips’ third point: that psychological models are frequently dehumanizing.Simplification or idealization is necessary for explanation and empirical endeavor, which cannot attend toeverything, but every model biases attention in a certain way, highlighting some things and obscuring others(March, 1972). As Bandura noted:

What we believe man [sic] to be affects which aspects of human functioning we study most thoroughly and which we disregard. Premisesthus delimit research and are, in turn, shaped by it. As knowledge gained through study is put into practice, the images of man [sic] onwhich social practices are based have even vaster implications. (Bandura, 1974, p. 859)

Among the “vaster consequences” of the models adopted in educational psychology are effects on schools,whose structures and practices are often built around psychological concepts (McDermott & Hood, 1982;Ramirez & Boli-Bennett, 1982), ultimately affecting students. These considerations imply that a choice ofpsychological model involves ethical as well as descriptive concerns, since how we think of ourselves is notonly about who we are, but, when acted upon, also about who we want to become (see Chapter 12, thisvolume).

The Paradox of Self-Knowledge

These points begin to suggest a paradox implicit in the attempt to gain self-knowledge. Can we ever reallygrasp ourselves intellectually when every conception is necessarily partial, in both senses? This question applieswith some force to psychology, the science of the psyche. Since the Greek word “psyche” is commonly translatedas “soul,” the whole notion of “psychology,” a science of the psyche, appears to be a contradiction in terms (thesoul not being recognized as an object of scientific inquiry). As a result, psychology apparently faces a choicebetween being either scientific and soulless, or soulful and unscientific.

This point may seem facile since it merely plays on the word “psyche,” and psychology long ago got over theembarrassment of its name and chose science over the soul. Nevertheless, tensions remain between the aim ofunderstanding ourselves in a more definite scientific sense, and the aim of understanding ourselves moreholistically and sympathetically. This tension was evident in psychology’s formative years in Wilhelm Wundt’sthe “two psychologies,” one physiological and positivistic, and the other cultural and interpretive (Cole, 1996).Similar tensions continue today, reflected in the continuing methodology wars in education, among others(see Chapter 2, this volume). It would seem that the attempt to grasp ourselves involves a paradox, like thenotion of a hand grasping itself, in which only parts can be grasped firmly, leaving the whole only felt orvaguely known.

The philosopher Wilfred Sellars depicted this tension as arising from a conflict between two images ofourselves that have different historical origins (Sellars, 1963). The “manifest image” is the way we conceive ofourselves in everyday life, such as understanding our own behavior and that of others as resulting from motivesor desires associated with an inner self. Sellars suggested that this image, which is the default theory that weuse in understanding our own and others’ conduct, is largely Aristotelian in origin. 4 This first image conflictswith a second, “scientific image” that derives largely from seventeenth-century mechanics. When people, andparticularly scholars and researchers, seek to explain behavior “objectively” or “scientifically,” they are likely toappeal to this second image, explaining behavior in terms of external forces or material entities, such as thebrain.

These two images conflict in theory, one basing explanation on purposes or aims and the other on externalforces. They also conflict in practice, creating dilemmas about how to respond to conduct that can beinterpreted either as willed or as caused (Olson, 2011). The manifest image also appears to be relativelycomplete, but untrue, while the scientific image appears to be relatively true, but incomplete (Sellars, 1963).As a result, we are left with something like an M. C. Escher drawing in which one interpretation ends upcontradicting itself, suggesting another interpretation that also ends up contradicting itself. Sellars suggestedthat the philosophical task in such situations is to attempt to find an approach in which these “differingperspectives on a landscape are fused into one coherent experience” (Sellars, 1963, p. 4) (For furtherreflections on the need for conceptual double vision, see Bateson, 1988.)

Three Images of Mind and Nature

Given this tension, it may be helpful to consider three, rather than two, perspectives on mind, nature, andourselves. A first approach considered here is a vitalistic one, for which Aristotle is the iconic thinker. SinceAristotle might also be considered the first psychologist—the first to study the psyche naturalistically—thisview has historical priority and has been enormously influential in Western thought. The second approach is amechanistic one, for which Newton is the iconic scientist or natural philosopher, and Descartes a keyphilosopher/psychologist. Since Descartes is often considered the first modern psychologist, we might viewhim as representative of a “modern” view of human nature and conduct. Finally, a third image derives fromnineteenth-century evolutionary thought, for which Darwin is the iconic scientist, and Charles SandersPeirce, perhaps, the most prescient philosopher. Peirce may seem an odd choice here, and one could appeal toDewey instead; however, Peirce laid the groundwork for a statistical and non-linear approach to science,helped to found the science of semiotics, and developed a semiotic conception of mind and self, and as such issometimes considered the first “post-modern” philosopher. The fact that his work is gaining newconsideration today provides another reason for reconsidering it in the context of an emerging educationalpsychology.

While I have introduced these three approaches in historical terms, one can also consider them moreanalytically, and I will in fact do a bit of both. Perhaps the closest parallel to the set of distinctions I amdrawing is Dewey and Bentley’s comparison of explanations based on “self-action,” “interaction,” and“transaction” (Dewey & Bentley, 1949). In “self-action” the cause of behavior is viewed as internal to theobject, such as in its intrinsic character, quality, or potential. In “interaction” (as Dewey and Bentley used theterm) the cause of behavior is an interaction between two things that affect one another externally, likeinteracting billiard balls. Finally, in a “transaction” the relationship changes the interacting objects themselves.

An example closer to educational psychology occurs in Cronbach’s comparison of different paradigms ofscientific psychology. Cronbach (1957) first compared two paradigms: the psychology of human individualdifferences (characterized by its use of correlational methods), and behavioral psychology (characterized by theuse of experimental methods). As he noted, these paradigms are mirror opposites, the first focusing on theeffects of individual differences on behavior in the same environment, and the second on the effects ofenvironmental differences on the behavior of the same individuals. As such, each represented only half of amore general approach that could consider the interactive effects of individual and environmental differenceson behavior (task performance). In a later paper Cronbach argued for this more general approach, representedby aptitude × treatment interaction research (Cronbach, 1975). This discussion is important because itrepresents another way of highlighting a third form of relationship that is not reducible to simpler ones, atleast when interactions are actually present.

A number of other approaches to psychology and education adopt similar comparisons involving threekinds of relationships. Bandura’s (1985) comparison of internal theories (psychodynamic and trait theories),external theories (radical behaviorism), and interactional accounts (his own social-cognitive theory) is anexample. In the last approach three factors interact—personality factors, environmental factors, and behavior—while the first and second approaches are simpler.

In education, Kohlberg and Mayer’s (1972) comparison of “romantic,” “cultural transmission,” and

“progressive” approaches to educational aims is also similar, although here the focus is on educational aimsrather than on causes for changes in behavior. “Romantic” approaches to education base their aims on theinner desires or potentials of individuals, while “cultural transmission” approaches base aims on externaldemands. Kohlberg and Mayer’s preferred approach, a “progressive” one based primarily on Piagetian theory(which they claim, incorrectly in my view, also represents Deweyan theory), adapts educational aims touniversal stages of development that are viewed as integrating individual and cultural concerns. Finally, asimilar contrast between internalist, externalist, and interactional/evolutionary approaches informed my owndiscussion of the way the concept of “learning” has evolved in psychology (Bredo, 1997), and contrastingmethodological orientations in educational research (Bredo, 2006b). While these analytic comparisons may behelpful in clarifying the contrasts to be drawn here, I believe it is also helpful to consider educationalpsychology in the wider context of the historical development of ideas—beginning with Aristotle.

The Manifest Image—Vitalism

As noted previously, the “manifest image” is the everyday or common-sense way in which people tend toenvision themselves, inherited in fair part from Aristotle. Aristotle’s approach has been characterized as a formof “vitalism” (Royce, 1914/1951) or “natural teleology” (Bambrough, 1963/2003) because he considered all ofnature to be purposive or end-directed. As he put it, “Nature does nothing in vain. For all things that exist byNature are means to an end, or will be concomitants of means to an end” (McKeon, 1941, p. 600). Aristotle’sapproach was also typological, as many sciences are in their early phases. Adopting the Greek conception ofnature, physis, in which a thing’s nature is what it tends to develop into under ideal conditions, he sought tocategorize things in terms of the ends essential to their being that kind of thing. As one Aristotle scholar putit:

The character of a substance—what it is to be a substance of the kind to which it belongs—is comprehensible only in terms of thecondition in which the substance reaches its proper fulfillment . . . To know what something is is to know what it is for; to know whatsomething is for, we must learn what is its nature, its character, its form. (Bambrough, 1963/2003, p. xxxii)

Aristotle faced a version of the mind/body problem in his day since earlier materialists, like Democritus, hadargued that everything is made of minute atoms whose interactions result in an object’s properties, whileidealists, like Plato, believed that patterns or forms are primary, and that particular things are imperfectappearances of ideal types which constitute reality. Aristotle attempted to soften this matter/form dichotomyby introducing a series of steps or gradations between matter and mind. In this view even inert matter isdriven by ends, since it has inherent potentials that it tends to actualize, heavy things, like earth and water,tending to move down toward the center of the earth, if unconstrained, and light things, like air and fire, uptowards the heavens. Living things also have their characteristic ends, all acting to reproduce themselves and“partake of the eternal and divine” by making their forms eternal, so far as possible. This was the “goaltowards which all things strive” and “for the sake of which they do whatsoever their nature renders possible”(McKeon, 1941, p. 561).

What makes life possible, in this conception, is having a psyche, “the form of a natural body having lifepotentially within it” (McKeon, 1941, p. 555). The psyche was what enabled living things to move themselvesas they do. While all living things have a “nutritive” psyche enabling them to grow and reproduce, growthbeing considered a form of movement (see Darwin, 1881), animals have a “sensitive” psyche, in addition,enabling them to move away from threatening or toward beneficial conditions before they occur. Finally,human beings have a “rational” psyche (in addition to nutritive and sensitive psyches), a social or discursiveability allowing them to select responses on the basis of “fore-choice” (Randall, 1960). Considered in this way,the rational mental abilities of human beings are a subset of life functions possessed by all animals, which are asubset of those possessed by all living things, including plants. Mind was a subset of nature, rather thansomething opposed to it.

In a related analysis, Aristotle also distinguished between four types of cause—material, efficient, formal,and final. As with the analysis of the psyche, there are levels or types of cause that help relate crude materialbeginnings to final ideal endings. The material cause is the effect of the matter of which an object is formed,the efficient cause the effect of action on the object, the formal cause the effect of the form being sought, andthe final cause the ultimate end or function of the object. This analysis enabled Aristotle to account for a wide

range of phenomena, including human action, which is not well explained by efficient or mechanical causationalone (Juarrero, 1999).

Aristotle’s analysis of the human psyche suggested that it was hierarchically layered, the rational psyche ontop, regulating the sensitive and nutritive psyches. Since the essential or defining characteristic of humanbeings is to be rational, the end or telos of human life was understood to involve becoming a rationally self-regulated and self-realizing person (Aristotle, 1980). If we understand our own natures properly, he suggested,the goal of human life, eudaimonia (translated as “happiness” or “self-realization”) can be understood to be touse reason to form our own characters, shaping our habits and desires so that we tend to behave in ever morevirtuous ways.

This model of the psyche mirrored Aristotle’s hierarchical view of nature, in which inanimate things, livingthings, animals, and human beings are related in a ladder leading toward the divine, the unmoved mover, orGod, who accounted for motion and was viewed as pure mind or pure reflection. One can see this orderlymodel of the cosmos as also mirroring a hierarchical conception of social order in which the reasoning classesare on top, and other classes below. As Ferry comments,

The Greek world was fundamentally an aristocratic world, a universe organized as a hierarchy in which those most endowed by natureshould in principle be “at the top,” while the less endowed saw themselves occupying inferior ranks. And we should not forget that theGreek city-state was founded on slavery. (Ferry, 2002, p. 72)

Aristotle’s hierarchical conception of nature, social life, and the psyche served as a link between the Platonicconception of values and the “great chain of being” of the Middle with its feudal social order based on fixed,inherited classes and functions (Lovejoy, 1936).

This sketch may suggest some of what we have also inherited from Aristotelian thought (especially asmediated by Christian scholastics). Perhaps the most important is the tendency to explain change in terms ofinner drives or desires, representing latent potentials that are actualized under appropriate conditions.Conceiving of individuals (and biological species) as falling into fixed, essential types is another inheritance,the first study of character types being conducted by a Roman follower of Aristotle (Bambrough, 1963/2003).An emphasis on the use of reason to train one’s character toward virtue is a third inheritance, of which currentwork on student self-regulation can be seen as an outgrowth (see Chapter 12, this volume). Finally, the valuehierarchy implicit in Aristotelian (and Platonic) thought also remains important. We continue—in botheveryday and scholarly life—to value mind over body, theory over practice, reason over emotion or habit,“higher” humans over other “lower” organisms. We are more Aristotelian than we realize.

2.3.

The Scientific Image—Mechanism

The second image of mind, nature, and ourselves is based on mechanistic thought deriving from the scientificrevolution of the seventeenth century. Mechanistic ideas developed, in part, in reaction to Aristotelianthought, which had become associated with religious dogmatism under the scholastics (Alexander, 2014).With the development of the Copernican view of the heavens, supported by Galileo’s telescopic observationsand Newton’s equations, the earth became one bit of matter revolving around other bits, displacing humanbeings from the center of the cosmos. The development of an experimental approach to science also displacedpassive Aristotelian observation with a more directly manipulative approach, providing the basis forquantitative laws of motion that better accorded with the movement of matter than Aristotle’s categories andpotentials.

To be clear about what a classical “mechanistic” account means for contemporary theory and research it maybe helpful to summarize its basic assumptions and assertions, drawing on Newton (1686/1952).

Matter is composed of simple particles, without inner structure, whose properties do not change withchanges in their movement (Newton’s idealized point-masses).Changes in motion (acceleration) only occur as a result of external forces (Newton’s first law).Changes in motion are related to their cause (force) by a universal, deterministic law (Newton’s second law,F = ma).The force exerted by interacting particles on one another is equal and opposite (Newton’s third law).

These four statements help highlight the contrast between Newtonian and Aristotelian assumptions. First, ifmovement can be understood in terms of the interaction of idealized particles, then form plays no role innature. Bits of matter attract one another throughout the universe regardless of the ways in which they happento be clumped (although their distance from one another matters). Second, if changes in movement only occuras a result of external forces, then matter has no internal drives or tendencies. It isn’t “trying” to go anywhereand has no “potential” it seeks to actualize. If it has no inherent direction of change then time is also purelyextrinsic (Prigogine, 1980) and nature has no “consummations” (Dewey, 1929/1958). Third, Newton’squantitative, deterministic laws also differed from Aristotle’s qualitative categorizations and potentials.Finally, the fact that interacting masses affect one another means there can be no “unmoved mover,” unlikeAristotle’s conception of God and the rational psyche, which were viewed as causing movement but as notbeing moved themselves (Juarrero, 1999).

The puzzling feature of a mechanistic account is what to make of the observer. The observer has to careabout form, such as the size and shape of the earth or moon. The observer also has to act intentionally andcare about beginnings and endings, such as by performing an experiment and noting the results. In aNewtonian analysis the observer also has to be separable from the interactions being studied, which is notpossible if everything is affected in equal and opposite ways. Making nature a purposeless machine created agap between observer and observed, subject and object, mind and nature, ends and means that becamedifficult—if not impossible—to resolve.

Descartes at least faced this issue directly. Noting that some behaviors, like reflexes, are mechanical innature, he considered the body to be a complex machine or automaton, not unlike the reflex-arc conception of

the nineteenth century (Huxley, 1870/1893). Subtle “animal spirits” flowed “from the heart to the brain, andfrom there . . . through the nerves into the muscles . . . [giving] movement to all the members, without theneed for imagining any other reason” (Descartes, 1637/1969, p. 29). This flow was regulated by sensoryimpressions, internal passions, and memory, with the result that, “the members of this body move in as manydifferent ways . . . as our own bodies can move, without the intervention of our will” (Descartes, 1637/1969, p.30).

Other aspects of human behavior appeared to be decidedly non-mechanical, such as the ability to put wordsand signs together “to tell our thoughts to others” and arrange them “differently in order to answer to thesense of all that is said” (Descartes, 1637/1969, p. 30). The ability to solve novel problems also differentiatedhuman beings from machines, which “inevitably fail” because they “do not act through knowledge but onlythrough the disposition of their organs” (Descartes, 1637/1969, p. 30). Observing that if his consciousthought stopped he would have no idea if he existed, Descartes concluded that he must be, “a substance thewhole essence or nature of which was merely to think, and which, in order to exist, needed no place anddepended on no material thing” (Descartes, 1637/1969, p. 18).

In dividing human conduct in this way, Descartes split Aristotle’s more continuous levels of the psyche intwo. The rational psyche was placed in the soul, while the nutritive and sensitive psyches were allocated to thebody (Lowry, 1971). Aristotle’s types of causation were similarly divided, the material and efficient causesbeing represented in bodily behavior, and formal and final causes in the soul (Juarrero, 1999). In conceiving ofhuman behavior in this way, Descartes took thought back to the categorical division between the material andideal worlds that Christianity inherited from Plato, while rejecting Aristotle’s more continuous, naturalisticscheme.

Cartesian dualism was extraordinarily influential (Huxley, 1870/1893), perhaps because it allowed scienceand religion to go their separate ways without too much entanglement. Science could focus on means, whilereligion could focus on ends. However, conceiving of mind and matter as different substances, located incategorically different realms (the extensional realm of space and time and the intentional realm of ideas),created insuperable problems for understanding how the two could interact:

If thoughts and sensations belong to an immaterial or non-physical portion of reality . . . how can they have effects in the physical world?How, for example, can a decision or act of will cause a movement of a human body? How, for that matter, can changes in the physicalworld have effects in the non-physical part of reality? If one’s feeling pain is a non-physical event, how can a physical injury to one’s bodycause one to feel pain? (van Inwagen, 2007)

Since these questions appear to have no adequate answers, metaphysical dualism has often been rejected infavor of either its materialistic or idealistic half. Either everything is matter (conceived mechanistically), andmind, soul, and consciousness are illusions or, if everything is an idea, pattern, or form, then matter is anillusion. Contemporary “eliminative materialists” adopt the first position, arguing, “there is nothing more tothe mind than what occurs in the brain. The reason mental states are irreducible is not because they are non-physical” but because “they do not really exist” (Ramsey, 2013). But if mind is just the functioning of aparticularly complicated machine it becomes difficult to explain the non-mechanical aspects of behavior thatDescartes outlined, such as the ability to respond to hypothetical events that have not yet occurred or maynever occur. As Deacon argues, phenomena like life and mind involve responses to “incomplete” or“unfinished” events that are part of a process that has its own internal development, while matter,

mechanistically construed, has no intrinsic development and is moved only by actual events (Deacon, 2012).

The other alternative, idealism, conceives of material objects as really or essentially ideas, like Platonicforms. Today’s “radical constructivists” come close to this idea, suggesting that “objects” are mental constructs(Glasersfeld, 1995). The sense of this notion clearly depends on how one defines an “object,” but if matter isjust an idea, it becomes difficult to explain events like walking into a door accidentally, which has a decidedlynon-ideal quality. As a result all three of these choices—metaphysical dualism, materialism, and idealism—have tended to be problematic, raising suspicions that the whole line of thinking has gone wrong—undoubtedly at its conception.

Related disputes, important to educational psychology, derive from epistemological rather thanmetaphysical dualism. Here the issue is not a metaphysical question, such as whether the soul or mind is realor imaginary, but an epistemological question about whether knowledge is founded in mental or materialevents. Descartes’ followers started on the mental side, viewing knowledge as based on clear and distinct apriori ideas, while Newton’s followers, like Locke, started from the material side, viewing knowledge as basedon elementary sensations caused by external objects (Feingold, 2004). This tension has been represented indiffering approaches to learning theory as well as in methodological disputes between rationalists andempiricists. Debate between “configurationist” and “associationist” approaches to learning theory is oneexample, the former holding that learning is shaped by preexisting mental patterns, while the latter viewslearning as the association of elementary stimuli or sensory impressions (Bruner, 2004; Hilgard & Bower,1966). Each approach had its successes, but associationists found it difficult to account for one-trial andillusory learning, while configurationists had difficulty accounting for gradual and veridical learning (Hilgard& Bower, 1966). Later divisions between behavioral and cognitive learning theories repeated a similar pattern.Behaviorists (influenced by Hume’s skepticism about knowing other minds) focused on the association ofexternally observable stimuli and responses, while cognitivists focused on inferred internal rules orrepresentations, and symbol-manipulating operations (Newell & Simon, 1972). Each tended to oppose theother’s foundations—directly observed external changes versus inferred internal rules and procedures—but Ibelieve it is fair to say that each was, again, able to account for phenomena the other found difficult to explain(see, e.g. Catania, 1984).

The principal source of these difficulties appears to lie in the original notion that nature is inherentlymechanistic. Once nature is viewed as a machine, and this model confounded with nature itself (an instance ofthe philosophical fallacy), there is no place for purposes, goals, values, ends, functions, or meanings, sincethese all depend on intrinsic relations between beginnings and endings. As a result, it appears that we havewritten ourselves out of our own story of nature (Deacon, 2012). And, what is particularly relevant foreducational psychology, the resulting conception of mind as rationalistic and “out of it,” emotionally,practically, and socially, is dysfunctional if taken seriously (Damasio, 1994; Lave & Wenger, 1991; Suchman,1987). A strong division between mind and body may also align with the contemporary system of socialclasses and educational tracks, to the detriment of all.

The Evolutionary Image

This brings us to the third image of mind, nature, and ourselves—an evolutionary one. Evolutionary ideascame into vogue in the nineteenth century, affecting most scholarly fields (Prigogine & Stengers, 1984). Inthe United States the dominant influences were Darwinian ideas about the evolution of biological species andneo-Hegelian ideas about the evolution of culture (Miller, 1968). While stimulated by technological changes,as well as by social changes, such as the American and French revolutions, evolutionary ideas also grew out ofdevelopments within science, such as the adoption of statistical thinking (Gigerenzer & Swijtink, 1989;Maxwell, 1859/1965; Menand, 2001), and interest in self-governing machines, like the Watt steam engine(Maxwell, 1868). These developments began to lead to a third conception of science in which a statisticalapproach was adopted (Royce, 1914/1951), as well as a dynamically recursive, non-linear one.

Since there is continuing dispute about the merits of different ways of understanding evolution, it may behelpful to refer to Darwin’s original submission to the Linnean Society (Darwin & Wallace, 1858), which islucid and at least somewhat authoritative. As Darwin noted, the fact that different species of organisms canreproduce at geometric rates yet often have fairly stable populations in a region suggests that they continually“check” one another’s growth through competition. When a relatively stable relationship has developed and isthen disturbed by a novel event, like a change in climate or invasion by a new species:

Some of its inhabitants will be exterminated; and the remainder will be exposed to the mutual action of a different set of inhabitants, whichI believe to be far more important to the life of each being than mere climate . . . I cannot doubt that during millions of generationsindividuals . . . will be occasionally born with some slight variation, profitable to some part of their economy. Such individuals will have abetter chance of surviving, and of propagating their new and slightly different structure, and the modification may be slowly increased bythe accumulative action of natural selection to any profitable extent . . . An organic being . . . may thus come to be adapted to a score ofcontingencies—natural selection accumulating those slight variations in all parts of its structure, which are in any way useful to it duringany part of its life. (Darwin, in Darwin & Wallace, 1858, pp. 3–4)

This process, elaborated as specialized niches develop, was understood to be the principal cause of the“classification and affinities of organic beings,” that “seem to branch, and sub-branch like the limbs of a treefrom a common trunk” (Darwin & Wallace, 1858, p. 4).

Darwin’s model differs interestingly from the others we have been considering. It attends to differingspecies and forms, like Aristotle, but views them as changing, and rejects the notion that they have essences(since they are populations of unique, interbreeding individuals having common descent). It also involvesquantitative relationships, like those emphasized in Newtonian theory, but they are statistical and non-linear(growth of a population at one time increases its rate of growth in the next, while growth of its competitorsdecreases it), unlike the linear, deterministic relationships of Newtonian theory (e.g., F = ma). Third,explanation is neither in terms of fixed ends nor deterministic means, but in terms of cycles of implicitly risky,experimental action that may or may not work out to reproduce itself. Finally, Darwin’s suggestion that the“mutual action” of organisms is far more important than exogenous events, like a change in climate,introduces a social or co-evolutionary aspect to his account that is often overlooked.

While Darwin focused on the evolution of biological life, philosophers, other scholars, and scientistsgeneralized his approach to all of nature and the universe more generally (Dewey, 1910/1997; Mead, 1964).As Karl Popper put it more recently, “science suggests to us (tentatively of course) a picture of the universethat is inventive or even creative; of a universe in which new things emerge on new levels” (Popper, 1978, p.

341). Among these emergent phenomena is mind. To restore greater continuity between mind and nature,and do so scientifically, all three categories needed to be rethought.

The scholar who most presciently and carefully, rethought all three of these categories was Charles SandersPeirce. For those unfamiliar with him, Peirce was an amazing polymath who contributed to many fields. Heviewed himself primarily as a logician, and contributed to the development of modern relational logic,originated pragmatism as a school of philosophy, was one of the principal originators of semiotics, the scienceof signs, and made contributions to a number of other fields, such as mathematics, physics, and geography.Bertrand Russell described him as “beyond doubt . . . one of the most original minds of the later nineteenthcentury, and certainly the greatest American thinker ever,” and Popper as “one of the greatest philosophers ofall time” (Popper, 1972). The fact that he influenced important philosopher/psychologists, such as WilliamJames, Josiah Royce, John Dewey, and George Herbert Mead, and laid the groundwork for a new approach topsychology, makes his approach important for this discussion, even though he did not influence many othersuntil recently, since much of his work was unpublished and is only becoming available today.

The key to Peirce’s approach lies in an abstract distinction that he drew between three kinds ofrelationships and their objects, which he termed “firsts,” “seconds,” and “thirds” (Peirce, 1878/1992b). “Firsts”are characterized by their participation in unary relationships. They are essentially qualities before they havebeen described or categorized as such. The psychological equivalent would be a feeling before it has beenrecognized or become a stimulus to a response. “Seconds” are objects or events that are defined by theirparticipation in binary relationships, like stimulus and response, cause and effect, blow and pain, subject andobject. In this case each term is only meaningful in relation to the other. For example, a stimulus that makesno difference to a response is not a “stimulus,” just as a force that has no effect is not a “force.” Finally, “thirds”are defined by their participation in trinary relationships. An example would be the act of “giving,” whichrequires a giver, a recipient, and a gift. None of which makes sense if it is not related to the other two. Theconcept of “reinforcement” in psychology is an example, since it involves an initial stimulus, a response, and areinforcing stimulus that alters the relationship between the first two. Without all three terms the concept of“reinforcement” makes no sense.

While I have used psychological examples, this set of distinctions was applied far beyond psychology—toeverything. Since these distinctions were drawn initially from logic, and the contrast between logical terms,propositions, and arguments, Peirce argued that they are the basic kinds of relationships involved ineverything we can possibly know. In other words, they constitute a minimalistic metaphysics, a “guess at theriddle” of the universe (Peirce, 1878/1992b). The point of the analysis was to argue that all three of thesekinds of events or processes are needed to account for change, and that one cannot reduce the set by doingaway with “firsts,” which are essentially novel or random events, or by doing away with “thirds,” which aremediated events. The need for all three kinds of relationships in making sense of the universe, and theirirreducibility to a smaller set, was Peirce’s way of defending the reality of all three against the tendency toreduce or absorb them.

These points apply to evolution because it involves three similar elements—qualitatively unique events, suchas random variants (firsts), relatively stable ongoing relationships (seconds), and mediated processes in whichunique forms and established relationships are altered contingently, resulting in the new forms andrelationships. In some situations random changes may predominate, such as when a new organism appears in

an environment with few competitors. In others habitual or mechanical relationships dominate, such as whenorganisms are well adapted to one another in sufficient numbers so that variants are quickly overwhelmed.And in a third class of situations there is an interaction between random qualitative differences and “habitual”quantitative processes that results in the more gradual evolution of new forms. The parallel with Cronbach’sanalysis of three approaches to psychology, discussed earlier, may be evident, except now everything isinterpreted in cyclical, evolutionary terms.

Peirce applied this approach to reconceptualizing nature in evolutionary and statistical terms, rather thanmechanistic ones (Houser & Kloesel, 1992, pp. 99–199). Since Newtonian laws make such precise claims thatthey can never truly be tested, he argued that we are free to conjecture that they are approximations torelationships that have an element of randomness or looseness in them. On the other hand, recurrentprocesses, like those involved in canalization (e.g., rainfall making a gulley deeper so that future rainfall willlikely be channeled to the same place) also result in the formation of “habits,” or structures whose componentsinteract in highly predictable, tightly coupled ways (for processes of canalization in biology and psychology,see Waddington, 1975, and Scarr, 1983). As Popper noted, using a metaphorical contrast between looselyrelated, non-deterministic “clouds” and tightly related, deterministic “clocks,” “Peirce was the first post-Newtonian physicist and philosopher who . . . dared to adopt the view that to some degree all clocks are clouds;or in other words that only clouds exist, though . . . of . . . different degrees of cloudiness” (Popper, 1972).Given this view, the universe is neither chaotic nor tightly ordered, but partially ordered and evolving.

Peirce also applied an evolutionary approach to understanding the way beliefs change, especially beliefs inscience. In effect, his was one of the first evolutionary epistemologies (see, e.g., Campbell, 1974). Tounderstand Peirce’s approach one needs to understand his conceptions of “belief,” “doubt,” and “inquiry.”Peirce borrowed his conception of belief, “that on which a man is prepared to act,” from Alexander Bain.Believing in a proposition means that you will act as if the outcome it suggests is certain to occur, like bettingeverything on a horse without a moment’s hesitation. Doubt, on the other hand, involves conflict betweensuch beliefs or habits (generalized beliefs). If new information suggests that the horse is ill you might hesitateor freeze. Conceived in this way, belief and doubt are not subjective feelings or propositions in the head, butaction tendencies. Nonetheless, both may cause feelings. Doubt causes the “irritation of doubt,” a sense ofloss, puzzlement, or uncertainty (Peirce, 1877/1923) which stimulates a “struggle” to reduce the doubt. Peircetermed this struggle “inquiry.” While inquiry seems to mean “thinking,” it includes a wider set of actions thanare usually included in this concept, such as overt manipulation, observation, and experimentation. In otherwords, inquiry is also not internal or located in the head, but is a form of activity. Peirce argued that we haveno way of knowing if a belief is true, aside from having no doubts about it, so the aim of inquiry is toeliminate doubt: “With the doubt, therefore, the struggle begins, and with the cessation of doubt, it ends.Hence the sole object of inquiry is the settlement of opinion” (Peirce, 1877/1923, p. 16).

Since belief is a way of acting, it cannot be changed voluntarily or by willing it to be different, but only byexperience based on acting on the belief. Here Peirce outlined three social methods for resolving conflictsbetween entrenched beliefs—the methods of authority, a priori reasoning, and scientific experimentation(Peirce, 1877/1923). While beliefs have evolved, so have methods of resolving conflicts between beliefs, thelatter methods being more refined and general than the earlier ones since they can resolve conflicts the earliercannot. However, even with methods of scientific experimentation one can never determine that a

generalization is true, since the next case may always disprove it. “Truth” then becomes a kind of ideal,reached only at the end of an evolutionary process of inquiry when a belief is found that will never causefurther doubts, just as “reality” consists of the objects described by those ideal beliefs. As Peirce put it:

The opinion which is fated to be ultimately agreed to by all who investigate, is what we mean by the truth, and the object represented inthis opinion is the real. That is the way I would explain reality. (Peirce, 1877/1923, pp. 56–57)

As part of his effort to rethink science, Peirce also reconsidered logical inference. The point is important herebecause it has fairly direct implications for learning theory, analogous to those mentioned earlier in thediscussion of Cartesian dualism. As noted, traditional epistemological debate has tended to divide deductiverationalists, like the followers of Descartes, and inductive empiricists, like the followers of Newton or Locke.Learning theory has tended to divide similarly. In contrast to this division, Peirce reasoned that since there arethree statements in the simplest logical argument, a syllogism, there must also be three basic forms ofinference (Peirce, 1878/1992a). In deduction one reasons “forwards” from an antecedent and a rule to aconclusion; in induction one reasons “upwards” from antecedent and consequent events to a rule describing theuniverse from which they were sampled; and in retroduction (which Peirce also called “abduction” and“hypothesis”) one reasons “backwards” from a consequent and a rule to an antecedent. Given three statements,one can use a guess at any two to make an inference about the third. One can develop a hypothesis about aninitial cause, using retroduction, deduce some implications of that hypothesis and perform an experiment tosee if the results are as suggested, and then use the new facts generated by the experiment to inductively alterbeliefs about the rule (for Peirce’s use of this cycle, see Fisch, 1986). A cyclical, experimental approach inwhich each element can change gets one out of the conflict between rationalist and empiricist foundationalism(see Dewey, 1922/1984).

This analysis suggests that there is a third approach to “learning” that traditional approaches ignore(Wojcikiewicz, 2010). The common denominator in the traditional approaches is that the task remains fixed(Newman, Griffin, & Cole, 1989). For behavioristic learning theory this means that environmentalcontingencies have to remain stable, while for cognitive theories the inner symbolic problem representationmust also remain constant. The task cannot change in either case because it would then be unclear if one wereobserving “learning,” this being defined as a change in performance on the same task or class of tasks. Yetpeople and animals leave environments when they can, change environments so that they present differentcontingencies (such as by building houses), and reconceptualize the situations facing them. As a result itwould seem that one can also “learn” to change the task or problem. More generally, Peirce’s interactiveapproach to inquiry (like Dewey’s) suggests that inquiry proceeds by changing the situation, and not merelythe individual or the environment alone, doubt being resolved by changing the person/environment situationor relationship. To neglect this kind of “learning” seriously limits one’s conception of human abilities—a pointconsistent with current work on situated learning (Lave & Wenger, 1991), and Snow’s transactional approachto aptitude, which views it as a property of person-in-situation (Corno et al., 2002) and with analyses of theway teachers and students learn to modify common activities cooperatively (see Chapter 5, this volume).

Peirce’s later work on semiotics, the science of signs, generalized this approach, resulting in a broadevolutionary conception of mind applicable to a wide range of natural phenomena. Here again, conventionalscholarship has tended to be dualistic, distinguishing between “sign” and “object,” or “signifier” and

“signified.” This results in familiar problems of dualistic schemes, such as disputes over which determines the

other, or how such different things can relate to one another. In contrast, Peirce distinguished three phases ofa semiotic process: “object,” “sign,” and “interpretant.” As an example, an “object” might be a fire, smoke a“sign” of the fire, and someone’s running away after seeing the smoke an “interpretant” of the sign, as havingindicated a fire. Needless to say. given Peirce’s penchant for triples, this threesome is related to Peirce’sdistinction between “firsts,” “seconds,” and “thirds,” the object being a first, the relation between object andinterpretant a second, and the three-way relationship between object, sign, and interpretant a “third.”

The point of introducing three (rather than two) functions is, again, to make the analysis more interactiveand evolutionary. With three differences in play, meanings may change and evolve, yet may also be correctedwhen what is signaled does not arrive. In effect, one gets the benefits of both subjectivism (novelty) andobjectivism (correction). Introducing three phases to the process of “semiosis” also makes it possible tounderstand how social meanings develop and are used in interaction, which is the basis for human reflectivethought. To see this point it may help to turn to George Herbert Mead’s social interactional account of theevolution of mind and self, which was based on Peirce’s semiotic distinctions (Mead, 1934/1967).

Using the concept of a “gesture,” borrowed from Darwin (1889/1904), Mead suggested that organismsfrequently learn from the beginnings of each other’s acts (a gesture), responding to them as though theymeant, or were signs of, a completed act to come. If one boxer begins to pull an arm back, that beginning maybe responded to by a second by beginning to duck, as though the original arm movement was the beginning ofa “blow.” In this case the initial gesture functions as a “sign” of a completed act (the “object”) as interpreted bya response (the “interpretant”). Each boxer may respond to the other’s gestures, in a continuing series or set ofcycles, implicating giving each a meaning, although the process may be largely unconscious, at least until afterthe fact.

Mead viewed this interactive behavior as the basis for social meaning, since each gesture’s meaning isworked out in the interaction itself. Meaning is then neither in anyone’s head nor determined outside of theinteraction. What a gesture comes to mean—what it signals is coming—depends on how it is interpreted by aresponse, and whether that interpretation is itself confirmed or disconfirmed by subsequent responses.

The point of this analysis is that human reflective thought can be considered to be a kind of “conversationof gestures” that one carries on with oneself. Insofar as one participates in joint action with others in whichsignaling is important to coordination, and can learn to “take the role of the other” by responding to one’s ownsigns or gestures as another would, one can respond to the beginning of one’s own acts in terms of theirmeaning. Responding to the meaning of a latent or emerging act is what is involved in reflective thought, asopposed to the non-reflective thought of animals, or at least of animals incapable of this kind of behavior.Similarly, responding to oneself as a meaningful object, as others would, is what it means to have a “self”(Mead, 1934/1967). Considered in this way, reflective thought is a social process in which one works out themeaning of one’s own latent or potential acts over time, first taking one perspective then another in an effortto find a sequence of acts that resolves an issue. This may involve manipulating propositions in the form ofaural statements, or written inscriptions, or other signs, to which one responds, and then responding to theresponse. Thinking evolves, like the other processes we have considered. And, just as cognitive psychologistshave argued that there are no images in the head, one can also argue similarly that there are also nopropositions in the head, mind being a social/interactional process, and not a thing.

This brief account neglects many very important things, such as the difference between gestures andlinguistic signs, whose meaning is at least partly determined by convention. The present point has only beento suggest how Peircean semiotics can form the basis for a theory of meaning and reflective thought. One canalso extend a similar analysis of semiosis “downwards” to consider meaningful communication between plantsor other organisms, where there is no possibility of reflective thought, and no sensing of the meaning ofgestures at a distance, only direct sensing, over time, of gradients or differences. As this suggests, Peirce’ssemiotics can be used to lay the groundwork for a very general approach to “mind” applicable across a widerange of natural phenomena. The field of biosemiotics has adopted Peircean ideas to analyze meaningfulsignaling in biological systems (Kull, Deacon, Emmeche, Hoffmeyer, & Stjernfelt, 2009), just asanthropologists have also adopted them in considering human symbolic communication in all of its variedforms (Danesi, 2004).

The idea of an evolutionary approach to mind and nature that involves an interplay of randomness,regularity, and corrective adjustment appears to be extremely valuable, yet still underappreciated, althoughmany of its wider ramifications were worked out by Dewey in his evolutionary conceptions of education,democracy, and science. In each case there is an evolving, self-corrective process that builds on its ownprevious results (except when they, too, are corrected). There is no certainty, either internal or external, only acontinued effort to build experimentally, based on what has worked in the past and what appears likely towork in the future. The price is loss of certainty and centeredness balanced by a gain in life.

Conclusions

This chapter represents an attempt to gain philosophical perspective on educational psychology by consideringthe origins of psychological conceptions of human nature and conduct in wider and longer-run movements ofthought. Since the dominant way of thinking in educational research arguably continues to be mechanistic orexternalist, the news is that there is more to an Aristotelian approach than one might imagine, and that wemay still not have caught up with the implications of a Darwinian evolutionary approach.

In fleshing out some further implications for educational psychology and educational practice I would liketo return to three streams of educational psychology that were strongly influenced by evolutionary ideas(Boring, 1963) but failed to fully incorporate the wider meaning of a Darwinian approach, at least asfrequently interpreted. Two of these were considered earlier, differential or trait psychology and behavioristicpsychology. The third consists of hierarchical stage theories of development. Differential or trait psychologyemerged from the work of Darwin’s cousin, Francis Galton, on hereditary genius in English families (Galton,1869/1892). Behaviorism emerged from experimental work comparing the mental functioning of differentanimals. Stage theories of development emerged, in turn, from Ernst Haeckel’s notion that “ontogenyrecapitulates phylogeny,” as represented in the work of G. Stanley Hall, among others.

These three traditions of educational psychology have been enormously influential in educational theoryand practice. They point to phenomena that are extremely important in schooling—what students bring tothe classroom, the kinds of knowledge and skills they are expected to learn, and their long-run patterns ofdevelopment. From both teacher and student perspectives these represent the beginnings, middles, and endsof an educational experience that should be related to one another. However, divisions between thesetraditions tended to fragment educational psychology into concerns for beginnings, middles, and endings thatwere unrelated, as suggested by Cronbach’s analysis of the first two. The third, hierarchical-stage theories ofdevelopment, can also be seen as unrelated since the ends of development, such as formal or post-conventionalreasoning, are considered to be already known, independent of knowledge of student aptitudes and schooltasks.

One of the reasons that these approaches violated an evolutionary attitude, despite claiming to be consistentwith one, is that they tended to base research on external norms that were presumed to be universal. That is,research was based on situations where it was clear what the “right” or “good” answer was. In IQ testing, forexample, “intelligence” was assumed to be universal among human beings—and could be judged by gettingmore right answers than other individuals on school-like tasks (normed, of course). In learning theory fixedexternal tasks created a norm defining a “correct” (reinforced) answer, the laws of learning to get there beingpresumed to be universal. And in hierarchical theories of development, the highest stage of developmentdefined the end of development, also presumed to be universal across all cultures.

All three of these approaches have been criticized extensively. Criticism of trait psychology (of which IQtesting is a branch) is long-standing, as indicated by Allport’s defense of it, originally written in 1929 (Allport,1968). The notion that “latent traits” express themselves in different forms of behavior, or performance, likeAristotelian potentials, fails to recognize that people tend to be more diverse in their ways of responding thanthis suggests, and that responses may be interpreted or “taken” in different ways, resulting potentially in thedevelopment of different patterns of interaction. An analogy would be the notion that oxygen has the “latent

trait” to become water, when it clearly has the potential to become many different things depending on theother elements with which it interacts.

Shifting to learning theory, the notion that one learns from external task contingencies depends on how thetask is conceived, as cognitive theorists pointed out (e.g., Chomsky, 1959). Students approaching a school taskin order to get a good grade or to please the teacher have made those activities the “task” to which theyattempt to learn answers (Dewey, 1916a). Beyond the notion that tasks may be conceived differently lies thepoint that they may also be modified and negotiated in interaction. Newman, Griffin, and Cole’s (1989)analysis of the way teachers and students may appropriate one another’s conceptions or approaches to a taskinto their own conceptions or approaches is an example.

Similar points apply to hierarchical-stage theories of development. The notion that cognitive developmentproceeds through a linear sequence of stages leading toward a predefined “highest” stage, such as formaloperations, has been criticized extensively. Intellectual problems or puzzles do not necessarily have only onesolution or equilibrium, and while steps may build on one another, there need not be a single set ofhierarchical stages (Phillips, 1987a, 1987b). As in other evolutionary or developmental processes, the end isnot predetermined at the beginning. Rather, it seems that developmental trajectories get worked out ininteraction with others, certain paths becoming relatively irreversible or closed off, canalizing likely futures incertain directions while making others highly unlikely (Werthman, 1970). At the populational level there alsoappear to be interactions between differences in early trauma and vulnerability and the degree to which thesocial environment is structured to create unequal opportunities and contingencies (Keating & Hertzman,1999).

In each of these cases the suggestion is that an interactional/evolutionary approach—in which similarstudent beginnings can evolve in different ways, present task environments can be renegotiated, and long-runfutures may develop in diverse ways in accord with different values—may be helpful. This may be so inparticular when considering the simplifications of an externalist approach (like the mechanistic attitudesuggested earlier) or an internalist one (like the vitalistic approach also discussed). Aninteractional/evolutionary approach may also make it possible to unite the three streams of traditionaleducational psychology by suggesting how what a student brings to school, how tasks are understood andmutually negotiated, and how development occurs in long-run directions are related over time, in a semioticprocess, rather than remaining unrelated phases of life. Adopting an interactional/evolutionary approachwould then relate educational psychology to Dewey’s (1938) criteria of an educative experience as involving“interaction” and “continuity,” rather than building educational psychology in a way that interrupts these veryprocesses.

Another implication of this discussion is that the models of human nature and conduct adopted ineducational psychology need to be considered in the light of their wider social and ethical consequences. WhatBruner suggested of psychological models of the learner applies to the other areas of educational psychology aswell:

The best approach to models of the learner is a reflective one that permits you to “go meta,” in enquire whether the script being imposedon the learner is there for the reason that was intended or for some other reason . . . You cannot improve education without a model of thelearner. Yet the model of the learner is not fixed, but various. A choice of one reflects many political, practical, and cultural issues. Perhapsthe best choice is not a choice of one, but an appreciation of the variety that is possible. (Bruner, 1985, p. 8)

I would only add that such models may not only be chosen in the light of wider aims but can also be evaluatedin terms of their observed consequences. These points apply to the present discussion as well, which also needsto be assessed in the light of its consequences, intended and unintended. At the very least I hope it serves tointroduce some of the concepts, issues, and debates in the educational psychology emerging today.

3.4.

Notes

I would like to acknowledge helpful comments and suggestions from Ramsey Affifi. I am grateful for many helpful comments andsuggestions from Christina Erneling, Walter Feinberg, Sarah Cashmore, Lyn Corno, Ray McDermott, David Olson, and Denis Phillips.With colleagues like these the source of remaining difficulties is clear.When I use the pronouns “we” or “us” or “ourselves,” it is meant to refer to the reflexivity practiced by all human beings and not to anyparticular person or group.As an example, try to identify the location of the difference or similarity between two things.As Newman wrote, “We cannot help, to a great extent, being Aristotelians, for the great Master does but analyze the thoughts, feelings,views, and opinions of human kind. He has told us the meaning of our own words and ideas, before we were born. In many subject matters,to think correctly, is to think like Aristotle, and we are his disciples whether we will or no, though we may not know it” (Newman,1854/1976, pp. 165–166). I am grateful to Trystan Goetze for this quotation.

References

Alexander, A. (2014). Infinitesimal: How a dangerous mathematical theory shaped the modern world. New York: Scientific American.Allport, G. W. (1968). Traits revisited. In G. W. Allport (Ed.), The person in psychology: Selected essays. Boston, MA: Beacon Press.Aristotle. (1980). Nicomachean ethics. Oxford: Oxford University Press.Bakhurst, D. (2008). Minds, brains, and education. Journal of Philosophy of Education, 42 (3–4), 415–432.Bambrough, R. (1963/2003). Introduction. In R. Bambrough (Ed.), The philosophy of Aristotle (pp. xi–xxxv). New York: Penguin.Bandura, A. (1974). Behavior theory and the models of man. American Psychologist December, 859–869.Bandura, A. (1985). Models of human nature and causality. In Social foundations of thought and action: A social cognitive theory (pp. 1–46). New

York: Prentice Hall.Bateson, G. (1988). Mind and nature: A necessary unity. New York: Bantam.Berlin, I. (1939/1999). The purpose of philosophy. In Concepts and categories: Philosophical essays (pp. 1–11). London: Pimlico.Boring, E. G. (1963). The influence of evolutionary theory upon American psychological thought. In R. I. Watson & D. T. Campbell (Eds.),

History, psychology, and science: Selected papers (pp. 159–184). New York: John Wiley.Bredo, E. (1997). The social construction of learning. In G. Phye (Ed.), Handbook of academic learning: The construction of knowledge (pp. 3–43).

New York: Academic Press.Bredo, E. (2006a). Conceptual confusion and educational psychology. In P. Winne, P. Alexander, & G. Phye (Eds.), Handbook of educational

psychology (pp. 43–57). Fairfax, VA: Techbooks.Bredo, E. (2006b). Philosophies of educational research. In J. Green, G. Camilli, & P. Elmore (Eds.), Complementary methods in educational

research (pp. 3–31). Washington, DC: American Educational Research Association.Bruner, J. (1985). Models of the learner. Educational Researcher, 14(6), 5–8.Bruner, J. (2004). A short history of psychological theories of learning. Daedalus, Winter, 13–20.Campbell, D. T. (1974). Evolutionary epistemology. In P. Schlipp (Ed.), The philosophy of Karl Popper. La Salle, IL: Open Court.Catania, C. A. (1984). Learning. Englewood Cliffs, NJ: Prentice-Hall.Chomsky, N. (1959). A review of B. F. Skinner’s Verbal Behavior. Language 35, 1–58.Cole, M. (1996). Cultural psychology: A once and future discipline. Cambridge, MA: Harvard University Press.Corno, L., Cronbach, L. J., Kupermintz, H., Lohman, D. F., Mandinach, E. B., Porteus, A. W. . . . et al. (2002). Remaking the concept of

aptitude: Extending the legacy of Richard E. Snow. Mahwah, NJ: Lawrence, Erlbaum Associates.Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist, 12, 671–684.Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116–127.Damasio, A. R. (1994). Descartes’ error: Emotion, reason, and the human brain. New York: G. P. Putnam’s Sons.Danesi, M. (2004). Messages, signs, and meanings: A basic textbook in semiotics and communication. Toronto: Canadian Scholars Press.Darwin, C. (1881). The power of movement in plants. New York: D. Appleton.Darwin, C. (1889/1904). The expression of the emotions in man and animals. London: John Murray.Darwin, C., & Wallace, A. R. (1858). On the tendency of species to form varieties; and on the perpetuation of varieties and species by natural

means of selection. In P. H. Barrett (Ed.), The collected papers of Charles Darwin (Vol. 2, pp. 3–19). Chicago, IL: University of Chicago Press.Deacon, T. W. (2012). Incomplete nature: How mind emerged from matter. New York: W. W. Norton.Descartes, R. (1637/1969). Discourse on the method of rightly conducting one’s reason and seeking truth in the sciences. In M. D. Wilson

(Ed.), The essential Descartes (pp. 1–42). New York: Mentor.Dewey, J. (1910/1997). The influence of Darwinism on philosophy. In The influence of Darwin on philosophy and other essays (pp. 1–19).

Amherst, NY: Prometheus.Dewey, J. (1916a). Democracy and education. New York: Macmillan.Dewey, J. (1916b). Philosophy of education. In Democracy and education (pp. 375–387). New York: Macmillan.Dewey, J. (1922/1984). Experimentalism, answer to the conflict between empiricism and rationalism. In Types of thinking (pp. 111–119). New

York: Philosophical Library.Dewey, J. (1929/1958). Experience and nature. New York: Dover Publications.Dewey, J. (1938). Experience and education. New York: Collier-Macmillan.Dewey, J., & Bentley, A. (1949). Knowing and the known. Boston, MA: Beacon Press.Feingold, M. (2004). The Newtonian moment. Oxford: Oxford University Press.Ferry, L. (2003). A brief history of thought. New York: Harper.Fisch, M. H. (1986). Peirce, semiotic, and pragmatism. Bloomington, IN: Indiana University Press.Galton, F. (1869/1892). Hereditary genius: An inquiry into its laws and consequences. London: Macmillan.

Gigerenzer, G., & Swijtink, Z. (1989). The empire of chance: How probability changed science. Cambridge, UK: Cambridge University Press.Glasersfeld, E. v. (1995). Radical constructivism: A way of knowing and learning. Washington, DC: Falmer Press.Goodman, N. (1972). The way the world is. In Problems and projects (pp. 24–32). Indianapolis, IN: Hackett Publishing.Hilgard, E. R., & Bower, G. (1966). Theories of learning. New York: Appleton-Century-Crofts.Houser, N., & Kloesel, C. (Eds.) (1992). The essential Peirce (Vol. 1). Bloomington, IN: Indiana University Press.Huxley, T. H. (1870/1893). Descartes’ discourse on method. In T. H. Huxley (Ed.), Collected essays (pp. 166–198). London: Macmillan.James, W. (1890/1952). The principles of psychology. In R. M. Hutchins (Ed.), Great books of the western world (Vol. 53). Chicago, IL:

Encyclopedia Britannica.Juarrero, A. (1999). Dynamics in action: Intentional behavior as a complex system. Cambridge, MA: MIT Press.Keating, D., & Hertzman, P. (1999). Developmental health and the wealth of nations: Social, biological, and educational dynamics. New York:

Guilford Press.Kohlberg, L., & Mayer, R. (1972). Development as the aim of education. Harvard Educational Review, 42(4), 449–496.Kull, K., Deacon, T., Emmeche, C., Hoffmeyer, J., & Stjernfelt, F. (2009). Theses on biosemiotics: Prolegomena to a theoretical biology.

Biological Theory, 4(2), 167–173.Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.Lovejoy, A. O. (1936). The great chain of being. Cambridge, MA: Harvard University Press.Lowry, R. (1971). The evolution of psychological theory: 1650 to the present. Chicago, IL: Aldine.March, J. G. (1972). Model bias in social action. Review of Educational Research, 42(4), 413–429.Maxwell, J. C. (1859/1965). Illustrations of the dynamical theory of gases. In W. D. Niven (Ed.), The scientific papers of James Clerk Maxwell

(Vol. 1, pp. 377–409). New York: Dover Press.Maxwell, J. C. (1868). On governors. Proceedings of the Royal Society of London, 16, 270–283.McDermott, R., & Hood, L. (1982). Institutionalized psychology and the ethnography of schooling. In P. Gilmore & A. Gladthorn (Eds.),

Children in and out of school. Washington, DC: Center for Applied Linguistics.McKeon, R. (1941). The basic works of Aristotle. New York: Random House.Mead, G. H. (1934/1967). Mind, self, and society: From the standpoint of a social behaviorist. Chicago, IL: University of Chicago Press.Mead, G. H. (1964). Evolution becomes a general idea. In A. Strauss (Ed.), George Herbert Mead: On social psychology (pp. 3–18). Chicago, IL:

University of Chicago Press.Menand, L. (2001). The metaphysical club. New York: Farrar, Straus & Giroux.Miller, P. (1968). American thought: Civil War to World War I. New York: Holt, Rinehart and Winston.Newell, A., & Simon, H. (1972). Human problem solving. Englewood Cliffs, NY: Prentice-Hall.Newman, J. H. (1854/1976). The idea of a university, defined and illustrated. Oxford, UK: Clarendon Press.Newman, D., Griffin, P., & Cole, M. (1989). The construction zone: Working for cognitive change in schools. Cambridge, UK: Cambridge

University Press.Olson, D. R. (2011). Agency and intentionality in pedagogy: Where the accountability train left the tracks. Paper presented at the workshop

Building Learning Cultures through Genre Practice, New York, USA. Retrieved from http://www.learningcultures.net/journal/archive/agency-an‐d-intentionality-in-pedagogy-where-the-accountability-train-left-the-tracks.

Peirce, C. S. (1877/1923). The fixation of belief. In M. R. Cohen (Ed.), Love, chance, and logic (pp. 7–31). New York: Harcourt, Brace and Co.Peirce, C. S. (1878/1992a). Deduction, induction, and hypothesis. In N. Hauser & C. Kloesel (Eds.), The essential Peirce: Selected philosophical

writings (Vol. 1, pp. 186–199). Bloomington, IN: Indiana University Press.Peirce, C. S. (1878/1992b). A guess at the riddle. In N. Hauser & C. Kloesel (Eds.), The essential Peirce: Selected philosophical writings (Vol. 1,

pp. 247–279).Phillips, D. C. (1987a). Change and development of cognitive structures: Piaget as theorist. Philosophy, science, and social inquiry (pp. 158–168).

Elmsford, NY: Pergamon Press.Phillips, D. C. (1987b). Kohlberg’s stages of moral development: A Lakatosian critique. In Philosophy, science, and social inquiry. New York:

Pergamon.Phillips, D. C. (1987c). Describing a student’s cognitive structure. Philosophy, science, and social inquiry. New York: Pergamon.Phillips, D. C. (1996). Philosophical perspectives. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 1005–1019).

New York: Macmillan.Popper, K. (1972). Of clouds and clocks. Objective knowledge: An evolutionary approach (pp. 206–255). Oxford: Clarendon Press.Popper, K. (1978). Natural selection and the emergence of mind. Dialectica, 34(3–4), 339–355.Prigogine, I. (1980). From being to becoming: Time and complexity in the physical sciences. San Francisco: W. H. Freeman.Prigogine, I., & Stengers, I. (1984). Order out of chaos: Man’s new dialogue with nature. New York: Bantam.

http://www.learningcultures.net/journal/archive/agency-and-intentionality-in-pedagogy-where-the-accountability-train-left-the-tracks

Ramirez, F., & Boli-Bennett, J. (1982). Global patterns of educational institutionalization. In P. G. Altbach, R. F. Arnov, & G. P. Kelley(Eds.). Comparative Education. New York: Macmillan.

Ramsey, W. (2013). Eliminative materialism. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (summer ed.). Retrieved from htt‐p://plato.stanford.edu/archives/sum2013/entries/materialism-eliminative/ (accessed 13 March 2015).

Randall, J. H. (1960). Aristotle. New York: Columbia University Press.Royce, J. (1914/1951). The mechanical, the historical and the statistical. In D. S. Robinson (Ed.), Royce’s logical essays: Collected logical essays of

Josiah Royce (pp. 35–62). Dubuque, IA: Wm. C. Brown.Russell, B. (1959). Wisdom of the west (p. 276). Garden City, NY: Doubleday.Ryle, G. (1949). The concept of mind. Chicago, IL: University of Chicago Press.Scarr, S. (1983). An evolutionary perspective on infant intelligence. In M. Lewis (Ed.), Origins of intelligence: Infancy and early childhood (pp.

191–223). New York: Plenum Press.Sellars, W. (1963). Philosophy and the scientific image of man. In R. Colodny (Ed.), Frontiers of science and philosophy (pp. 35–78). Pittsburgh:

University of Pittsburgh Press.Suchman, L. A. (1987). Plans and situated actions: The problem of human–machine communication. Cambridge, U.K.: Cambridge University Press.van Inwagen, P. (2007). Metaphysics. Stanford Encyclopedia of Philosophy. Retrieved from http://plato.stanford.edu/archives/win2014/entries/‐

metaphysics/ (accessed 13 March 2015).Waddington, C. H. (1975). The evolution of an evolutionist. Ithaca, NY: Cornell University Press.Watzlawick, P., Beavin, J. H., & Jackson, D. D. (1967). Pragmatics of human communication. New York: W. W. Norton.Werthman, C. (1970). The function of social definitions in the development of delinquent careers. In P. Garabedian & D. Gibbons (Eds.),

Becoming delinquent: Young offenders and the correctional process. Chicago, IL: Aldine.Wittgenstein, L. (1958). Philosophical investigations. New York: Macmillan.Wojcikiewicz, S. K. (2010). Dewey, Peirce, and the categories of learning. Education and Culture, 26(2), 65–82.

http://www.plato.stanford.edu/archives/sum2013/entries/materialism-eliminative/

http://wwwplato.stanford.edu/archives/win2014/entries/metaphysics/

1.2.3.4.

2Modes of Inquiry in Educational Psychology and Learning

Sciences ResearchWILLIAM R. PENUEL

University of Colorado Boulder

KENNETH A. FRANK

Michigan State University Educational psychologists and learning scientists rely on multiple modes of inquiry to answer different kinds ofresearch questions. Modes of inquiry refer to more than just methods and sources of data. They refer todistinct approaches to the study of learning and development in educational settings that draw from differentdisciplines. Moreover, modes of inquiry employ different standards of evidence for making judgments aboutthe validity and reliability of claims, and make different kinds of value commitments (Eisenhart & Towne,2003). In this chapter, we take up four questions for six different contemporary and emerging modes ofinquiry within the field:

What kinds of claims can the mode of inquiry support?Who are the audiences for these claims?What is the form and substance of arguments in this mode of inquiry?What are the scope and some limitations of this mode of inquiry?

Each of these questions frames educational psychology or learning sciences research as a form of humanisticinquiry grounded in argument from evidence. This perspective on educational psychology is grounded inToulmin’s (1958) model of practical arguments, particularly as elaborated within House’s (1977, 1979) notionof evaluative arguments in educational research. In both Toulmin’s and House’s frameworks, investigatorsbegin with a claim or conclusion to be elaborated, refined, or tested through empirical research. Such claimsare framed always in terms of specific goals for research, and they are informed by values about desired endsfor education (Kelly & Yin, 2007). An example of a claim investigated by educational psychologists is,“Schools can play an active role in the provision of opportunities for social mobility or in the exacerbation ofsocial inequality, depending on how they are structured” (Muller, Riegle-Crumb, Schiller, Wilkinson, &Frank, 2010, p. 1039). This claim is framed relative to a specific purpose—to explore whether racially diversehigh schools provide equality of opportunity to students of different racial backgrounds. The claim is alsoinformed by presumed shared values of the audience, especially equality of opportunity and social mobility.Implicit in the claim are some implications for policy and practice, particularly with respect to how resources

should be allocated, in this case to schools and organizations that support schools to structure more equitable

opportunities for student learning. Each of these aspects of claims highlights the significance andconsequentiality of evaluative arguments.

Different modes of inquiry are well suited for some kinds of claims but not others. Modes of inquiry are central toargumentation, in that the credibility of the warrant for linking claims and data is based on the particularmethods used, which must be backed by a “shared and appropriate methodology” (Kelly & Yin, 2007, p. 134).As such, different modes of inquiry are well suited for making certain kinds of claims, but not others. Anethnographic study of how a young person encounters mathematics at home and school may be useful forinvestigating claims about how young people recruit family members to assist with homework (Jackson, 2011).Further following young people as they move across contexts of learning may help to account for why someschool-based interventions succeed or fail, but such a study cannot address a claim about the comparativebenefits of different interventions, such as the claim that “providing students with extra homework help atschool is better than providing parents with guidance about how to help their children.” Such a claim mightbetter be addressed with a series of comparison studies, explored across a range of contexts that are informedby findings from ethnographic studies.

Arguments are made to persuade particular audiences. The task of research, from the perspective we develophere, is persuasion and not proof (House, 1977). As such, adjectives such as “credible” and “plausible” aremore appropriate for describing and evaluating arguments than are terms like “definitive” or “unassailable.”What is credible or plausible, moreover, is always at least partly relative to a particular audience, and theaudiences of educational research are varied. They include other researchers, teachers, educational leaders, andpolicy makers, each of whom bring different concerns, intentions for using research, as well as differentapproaches to deliberation about how research findings should be used (Asen, Gurke, Solomon, Conners, &Gumm, 2011). Even among researchers, there is considerable contestation about what modes of inquiry areappropriate for what purposes (e.g., Eisenhart & Towne, 2003), and the uses of even a single research findingor study are often varied. These differences are not to be overcome, but they are a reminder that researchyields knowledge that is both uncertain and contestable.

Different modes of inquiry can be distinguished by the form and substance of arguments. Kelly (2004) calls theform that arguments take for a particular mode of inquiry that mode’s argumentative grammar. He defines anargumentative grammar as “the logic that guides the use of a method and that supports reasoning about itsdata” (Kelly, 2004, p. 118). An argumentative grammar is not peculiar to a particular study, and it isanalytically separable from the substance of a particular study’s focus, claims, and conclusions. It shouldprovide researchers who regularly employ a different mode of inquiry—in the case of this chapter, educationalpsychologists and learning scientists—a framework for evaluating the strength of the argument advanced in aparticular study within a different mode of inquiry.

The substance of arguments is also an important consideration for judging the quality of individual studies.A study may faithfully adhere to the argumentative grammar of a particular mode of inquiry, but its argumentmay not be persuasive to others. The substance of the claims may be judged to be neither important norrelevant to the audiences for the research, or audiences may challenge the credibility of the components of the

argument or links among those components (Cronbach, 1988). Audiences may also judge arguments to beeither incomplete or biased toward a particular point of view (House, 1977).

Scope and limitations of forms of inquiry. Judging the degree to which a particular mode of inquiry provides anadequate backing for a warrant linking claim and evidence depends upon the reproducibility andgeneralizability of findings (Cronbach, 1982). Judgments about reproducibility answer the question of whetheranother investigation would hypothetically have generated the same conclusions. Sources of variation thatcould condition claims and reduce reproducibility include changes to the sample, a change in investigators, orchoice of different sources of evidence (Cronbach, 1982). Even though such thought experiments aboutreproducibility inevitably involve qualitative judgments about what kinds of differences might leadinvestigators to draw different conclusions from a particular study, it is possible to quantify certain aspects ofthe situation, such as how large an unmeasured variable’s relation to an outcome would have to be toinvalidate an inference (e.g., Frank, 2000). Insofar as claims developed in research are intended to apply toother contexts, populations, and actions, research aims at some form of generalizability. Particular modes ofinquiry seek to generalize to other kinds of objects: for example, case study (Yin, 2003) and ethnographicresearch (Goetz & LeCompte, 1984) both aim principally to generalize to theory rather than to populations.Just as claims to reproducibility are contested on qualitative grounds, so, too, are claims about generalizability.Here as well, efforts to quantify generalizability can inform such arguments (Hedges, 2013).

Table 2.1

Organization of this Review

In this chapter, we review several different modes of inquiry and describe their use in studies published ineducational psychology journals between 2006 and 2013. To help us organize the chapter, we initiallyreviewed abstracts of articles published in five major English-language journals in educational psychology:Educational Psychologist, Journal of Educational Psychology, Educational Psychology, British Journal of EducationalPsychology, and Contemporary Educational Psychology. To broaden our reach, we also reviewed abstracts fromthe Journal of the Learning Sciences, the American Journal of Evaluation, and the Teaching, Learning, andHuman Development section of the American Educational Research Journal. We illustrate the typology ofdifferent modes of inquiry reviewed below with selected articles from this initial search that we believe areparadigmatic—meaning useful studies for explicating the mode of inquiry to someone less familiar with it.The typology includes both methods that are widely used within educational psychology today, as well assome that are still emerging and developing. Ours is certainly a selective approach with respect to emergingareas of inquiry, but our selection was guided by considerations of areas of increased interest among bothresearchers and policy makers in education who make funding decisions about educational research.

For each mode of inquiry, we present answers to the four key questions posed in the introduction. Inaddition, to illustrate the substance of argumentation within each mode of inquiry, we describe a researchstudy that is an example of the approach. Each example was selected carefully to be paradigmatic of theapproach, and because other investigators have—in some way or another—taken the interventions, theories,or design principles from the study, and applied them to other contexts. Table 2.1 identifies the modes ofinquiry reviewed in this chapter and the kinds of claims each type seeks to support with evidence fromresearch.

Typology of Modes of Inquiry in Educational Psychology and Learning Sciences

Type of Research Description

Intervention Research

Randomized controlled trials A type of intervention research in which the principal aim is to investigate the impact ofprograms and policies. Researchers use random assignment to treatment and comparisongroups to reduce threats to internal validity of findings

Design-based research A type of intervention research in which the principal aims are to develop theories,principles for the design of learning environments, or new interventions that can be refinedand tested in subsequent studies. Researchers both engineer and study the learningenvironments in iterative cycles

Studies of Development over Longer Periods of Time and Across Settings

Longitudinal observation studies A type of research that analyzes relationships among psychological processes and outcomesover time. Researchers use these studies both to describe and develop causal explanationsfor growth or decline on focal measures of outcomes

Learning trajectories or learning progressionsresearch

A type of developmental research that analyzes viable routes to learning in disciplines overtime. Researchers generate and empirically test these routes using a range of cross-sectionaland longitudinal designs

Emerging Forms of Learning Sciences and Educational Psychology Research

Research on learning as a cross-settingphenomenon

A type of research that focuses on learning that takes place as people move across variedsociocultural contexts and practices. Researchers use ethnographic methods to documentwhat individuals bring from one setting to another and the social supports they rely on in

and across settings

Educational data mining and learning analytics A type of research that uses large datasets from digital learning environments to discoveremergent patterns of learning and to develop insight into theoretically informed learningprocesses. Researchers use techniques for investigating online interactions with peers andcontent to draw inferences about learning

Intervention Research in Educational Psychology

Intervention research has long been an important mode of inquiry within educational psychology. It is a formof applied psychological research that overlaps closely with concerns and methods of both policy andevaluation researchers. Intervention research in educational psychology has often sought to investigate claimsrelated to the efficacy and effectiveness of particular interventions in group or cluster randomized controlledtrials. Another kind of intervention research, design-based research, is aimed not at testing the efficacy ofinterventions, but rather at developing new hypotheses about how to support learning. Below, we review bothof these modes of inquiry into interventions.

Cluster Randomized Controlled Trials of Educational Interventions

Randomized controlled trials can be used to obtain unbiased estimates of the impacts of interventions.Because participants are randomly assigned to treatments, estimates are not expected to be biased bydifferences between students or teachers opting into programs or other factors that are unrelated to theintervention but that could shape outcomes (Shadish, Cook, & Campbell, 2002). Such trials may be focusedon establishing the efficacy of an intervention, under conditions in which researchers try to optimize conditionsfor successful implementation, or for testing the effectiveness of an intervention under more typical conditionsof implementation (Flay et al., 2005). Regardless, a defining feature of such studies is random assignment totreatment and comparison conditions or to alternate treatment conditions.

Today, most randomized controlled trials in education are field trials because they occur in real educationalsettings, rather than in laboratories. In addition, while some involve random assignment of individuals totreatment and comparison conditions, many involve random assignment of clusters of students withinclassrooms and schools, whichever is appropriate given the purpose and scope of the intervention. In the pastdecade, software programs like Optimal Design (Spybrook, Raudenbush, Liu, Congdon, & Martinez, 2009)have been developed to help researchers estimate the necessary sample size for intervention research, givenassumptions about the level of treatment assignment (i.e., whether assignment is at the level of an individualstudent or teacher, or at the level of classroom or school), the likely magnitude of effects, the size of groups orclusters such as classrooms, and the estimated variance in outcomes associated with groups. In addition, avariety of statistical software packages exist today that permit researchers to analyze the results of clusterrandomized trials using multi-level modeling techniques.

Claims cluster randomized controlled trials can support. The results of randomized controlled trials areintended to support causal inferences about the impacts of programs, that is, claims about whether a programcauses an increase in an outcome of interest (Shadish et al., 2002). In educational psychology in the pastdecade, randomized controlled trials have focused, for example, on the efficacy of interventions to improveacademic performance (Vadasy, Sanders, & Peyton, 2006) and citizenship outcomes (Schultema, Veugelers,Rijlaarsdam, & ten Dam, 2009). In addition, randomized controlled trials have been conducted ofprofessional development interventions that aim to improve teaching and learning outcomes (Powell,Diamond, Burchinal, & Koehler, 2010).

When trials focus on programs, it is possible to investigate whether and how the implementation of certain

program features correlate with outcomes (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002). Someresearchers question whether supporting causal inferences about the contribution of specific features tooutcomes is possible when there is no random assignment to different versions of a treatment (Green, Ha, &Bullock, 2010). Occasionally, cluster randomized controlled trials assign schools or classrooms to alternateversions of a treatment, which can support more robust causal inferences about the efficacy of specific designfeatures of the treatment (e.g., Penuel, Gallagher, & Moorthy, 2011).

Audiences for cluster randomized controlled trials. Key audiences for results of studies of the efficacy andeffectiveness of interventions are policy makers and educational leaders. Both of these sets of actors makedecisions about programs, curriculum materials, and interventions to support struggling students. They alwaysface constrained resources, and under such circ*mstances, policy makers and educational leaders benefit fromguidance that allows them to choose more effective programs over less effective ones (Dynarski, 2008). Thereare now databases of summaries of research findings from randomized controlled trials in education, such asthe What Works Clearinghouse (http://www.whatworks.ed.gov/), that are intended to help policy makers andeducational leaders select programs with the best and most credible evidence of effectiveness.

Form and substance of arguments in cluster randomized controlled trials. Many intervention researchers embracerandom assignment as the best approach to eliminate bias from estimates of program impact, because it is thebest procedure for eliminating or significantly reducing threats to internal validity of claims. Common threatsto internal validity include selection processes, in which certain kinds of people are more likely to choose to bepart of an intervention; maturation, in which natural growth accounts for changes in outcomes instead of thetreatment; and attrition. True experiments that employ random assignment are still subject to some threats tointernal validity and potentially biased results, such as attrition. However, if participants have an equalprobability of being assigned to each of the conditions and one participant’s outcomes are not affected by anyother participant’s assignment, then researchers can use randomized experiments to obtain unbiased estimatesof any differences in outcomes between those in different treatment conditions (Rubin, 1974).

An evidentiary argument that supports claims about program impact requires more than appeals to the logicof causal inference. The persuasiveness of claims depends in part on the strength of the argument about themechanism by which the treatment is expected to produce the desired outcome (Cordray & Pion, 2006). Inaddition, the outcome must measure student behavior that the intervention is intended to develop and mustbe sensitive to a range of interventions focused on changing those behaviors (Ruiz-Primo, Shavelson,Hamilton, & Klein, 2002). The argument’s persuasiveness also depends on the nature and appropriateness ofthe control condition to the potential users of evaluation results (Morgan & Winship, 2007) and the quality ofimplementation, or the achieved relative strength of the intervention (Cordray & Pion, 2006). Finally, acompelling argument considers possible counterarguments that would lead to different claims about programimpact. For example, perhaps students in a treatment performed better on an outcome measure than studentsin a comparison group, but treatment students learned more simply because they had been more time on taskthan comparison group students had and not because the treatment engaged students in a more effective,efficient manner.

Intervention research on a program called SimCalc Mathworlds illustrates how one team of researchersdesigned a series of studies that built toward a well-designed cluster randomized field trial. SimCalc is a

http://www.whatworks.ed.gov/

technology-supported intervention intended to democratize access to the mathematics of change and thatemploys graphical visualizations of mathematical phenomena to facilitate student understanding (Roschelle,Kaput, & Stroup, 2000). The team that developed SimCalc wanted to test the claim that the interventioncould be implemented in a wide variety of classrooms and effective with a range of student subpopulations.Prior to conducting a large cluster randomized controlled trial of SimCalc, the team pursued three preliminarystudies to reduce uncertainty and aid in the design of the larger study. First, the team developed and tested aset of assessments that measured the kinds of complex mathematics problem-solving skills taught in SimCalcbut that are rarely included as part of standardized tests used by states for accountability purposes. Second, theteam developed what they called a “curricular activity system,” that is, an integrated set of curriculum materialsfor teachers, student activities, and professional development designs that could support implementation(Roschelle et al., 2010b). Third, the team undertook a set of pilot studies, so that they could use the results toestimate potential effect sizes, as well as variance in outcomes associated with classrooms and students, inorder to estimate the sample size they would need for a larger study (Tatar et al., 2008).

The cluster randomized controlled trial that the SimCalc team conducted provided supporting evidence forthe claim that SimCalc could have a positive impact on student learning in a wide variety of contexts(Roschelle et al., 2010b). In that trial, 88 eighth-grade teachers who volunteered for the study were assignedto one of two conditions, a treatment condition or a comparison condition. Teachers in both conditionsreceived professional development in the mathematics content that was the focus of the SimCalc curricularactivity system; teachers in the treatment condition also received the SimCalc materials and professionaldevelopment in the use of those materials. The researchers used logs or diaries of instruction to assessimplementation fidelity. Pre- and post-test differences on the SimCalc-designed assessments were analyzed tosupport inferences about differences between the two conditions with respect to an overall average treatmenteffect, and results were also disaggregated by ethnicity (Roschelle et al., 2010a). A subsequent analysis of thesame data provided evidence that these impacts might be generalizable to most school districts in the statewhere the study was conducted, districts that include significant racial, ethnic, and socioeconomic diversity(Hedges, 2013).

Scope and limitation of cluster randomized controlled trials. Field-based randomized controlled trials are adifficult undertaking, and they are more easily conducted when interventions are well defined, brief, and nottoo difficult to implement well (Cook, 2002). It is possible to test the efficacy of interventions in earlier stagesof development using randomized controlled trials, but, because of smaller samples, they may not haveadequate statistical power to detect statistically significant effects (Osborne, 2008). In addition, randomizedcontrolled trials often include supportive conditions for implementation that may not be continued whenstudies end, such as more intensive professional development or technology support. As a consequence, theprogram may be discontinued following the trial, especially when implementers perceive that the programdoes not meet the needs of participants or support the goals of their organization (Fishman, Penuel, Hegedus,& Roschelle, 2011). Finally, although cluster randomized controlled trials of programs may be useful indetermining whether programs work, few are designed to investigate how to make programs work for a widevariety of participants, or to explain why they do not (Bryk, Gomez, & Grunow, 2011).

Design-Based Research

Educational psychologists who identify as learning scientists, an interdisciplinary group of researchers focusedon the study of learning processes, employ methods that are more akin to those used by engineers than bypsychologists. Design-based research aims to create or engineer new supports for learning in order to study howchildren learn (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Sandoval, 2004; Sandoval & Bell, 2004).Design-based research is perhaps best described as a “family of approaches” to research—rather than amethodology—because its proponents do not yet share a common argumentative grammar or logic forargumentation (Kelly, 2004; Reimann, 2011). Indeed, proponents employ a variety of terms to describe thekind of design-based research they do. These terms, which share common characteristics but have differentroots, include design experiments (Brown, 1992), formative experiments (Reinking & Bradley, 2008), anddevelopment research (van den Akker, 1999).

Claims design-based research can support. A key aim of design experiments is to specify how people learn toengage in practices of disciplines (e.g., argumentation in science) and to gather research evidence to identifyways to support their learning successfully (Cobb et al., 2003). The particular goals for disciplinary learningdraw from multiple sources, including studies of expert practice as well as tradition (Gravemeijer & Cobb,2006). The claims design-based research generates pertain to conjectures about how people learn, principlesfor designing learning environments, and educational design practice (Edelson, 2002).

The particular conjectures for which design-based research develops claims are sometimes called “localinstruction theories” (Gravemeijer & Cobb, 2006), because they pertain to learning goals that pertain to a fewdisciplinary core ideas and practices, and because they specify ways that teaching can support learning theseideas and practices. Here, teaching encompasses the skillful sequencing and orchestration of tasks andclassroom discourse to promote learning, as well as the use of tools (including technology) that enable studentsto construct and assess their own understanding of subject matter. Guiding the teacher’s decision making indesign research is a hypothetical learning trajectory that specifies one or more pathways from an instructionalbeginning point to the learning goal (Simon, 1995).

In other design-based research in which the goal is to develop new software or learning environments toexplore new possibilities for learning, claims are developed with the support of a conjecture map (Sandoval,2014). A conjecture map is a graphical depiction—accompanied by text with supporting theory and evidence—of key features of a learning environment hypothesized to support learning in a particular domain. Design-based research that uses conjecture mapping has as its main aim the refinement and revision of conjecturesrelated to how particular tools and materials, task structures, participant structures, and discourse practices cansupport learning when used in concert in a particular learning environment.

Audiences and uses for design-based research. The proponents of design-based research argue that research oninterventions developed, refined, and tested in the crucible of classrooms can play an important role indeveloping theories of learning and producing practical, usable interventions for teachers and students(Edelson, 2002). In fact, few products of design-based research have had a broad impact on practice, in partbecause the supports needed to organize learning environments on a small scale by learning scientists requirecapacities and resources many schools do not have (Fishman & Krajcik, 2003). Moreover, design-based

researchers do not typically engage directly in policy debate, and so the impacts on policy have been morelimited than with some of the other modes of inquiry (Penuel & Spillane, in press).

Design-based research has had its greatest influence on other researchers and on curriculum developers,through the articulation of design principles. Design principles are statements intended to inform thedevelopment of new learning environments and innovative curriculum materials warranted by evidence fromdesign research studies (Bell, Hoadley, & Linn, 2004). Design principles may be focused on how to supportlearning along specific learning trajectories, or on how to design technology-based learning environments.

Form and substance of arguments in design-based research. Unlike experimental research that develops claimsabout the efficacy of particular interventions, the goal of design-based research is to develop process-orientedexplanations of learning (Reimann, 2011). Design-based research puts local instruction theories andconjectures “in harm’s way” by implementing designs in real classrooms, gathering evidence of studentlearning processes and outcomes, and then iteratively refining those theories and conjectures. The logic ofdesign research is distinct from formal hypothesis testing, and is more akin to the goal of grounded theory(Charmaz, 2000), in which a central aim of research is the generation of theory. When specific conjectures areembodied in designs, implementation can lead either to a refinement of the conjecture or to an iterativeimprovement of some aspect of the design itself (Sandoval, 2014).

In design-based research, iteration unfolds both as the research unfolds and retrospectively, after a cycle ofclassroom-based research is complete (Gravemeijer & Cobb, 2006). In the course of a single designexperiment, researchers typically debrief with classroom teachers and members of the research team after eachday’s activities. In these debriefings, the team relies on a combination of informal observations, notes aboutstudent interactions, and artifacts from the classroom to make decisions about how the learning trajectorymight need to be altered the next day, or to identify some specific ways that the conjectured means of supportdid or did not help students learn that day. In a retrospective analysis, the research team reviews videorecordings of classroom activities, artifacts of student work, and formal and informal assessments of studentthinking (often in the form of pre- and post-tests, administered as interviews with students). The aim inretrospective analysis is to refine, revise, or even challenge particular aspects of a local instructional theory,using evidence from these different sources. The retrospective analysis may lead to a new local instructionaltheory, a new set of conjectures or design, or a new cycle of design-based research. It can also lead to thespecification of an intervention whose efficacy may be tested using a different mode of inquiry (such as anefficacy trial).

An example of design-based research from mathematics is reported by Jurow, Hall, and Ma (2008), whodeveloped and studied an adaptation to the materials of the middle school Mathematics through ApplicationsProject (MMAP). MMAP was a project-based, technology-supported curriculum in mathematics aligned tostandards developed in the 1990s (Greeno & Middle School Mathematics through Applications ProjectGroup, 1998). The adaptation that Jurow and colleagues made was to incorporate professional design reviewsof student work at the end of individual units. As part of these design reviews for one such unit on populationecology, graduate students in biology served as expert reviewers of students’ models of predator–prey systemdynamics. The idea for design reviews came from an earlier set of studies of the mathematical practices ofprofessional architects, who worked in a field where design reviews were an integral part of their practice

(Stevens & Hall, 1998).The organization of the research project was typical of many design-based research projects in most

respects. The team worked in close partnership with a single teacher in a school near the researchers’ homeinstitution over the course of a school year. The team jointly discussed and planned the adaptations to theunit, and conducted weekly research meetings that included the teacher. At these meetings, the teamdiscussed the classroom activities and analyzed classroom interactions captured on videotape. For most of theunit, the teacher taught the activities, but for the new activities—the design reviews—the researchers assistedand were more involved in supporting implementation.

Also, as is typical of design-based research, the research team had as a primary aim to generate designprinciples that could inform future research on learning. To that end, their analyses focused on variation in theunfolding and effects of what they termed “recontextualization exchanges” between the graduate studentbiologists and students in design reviews. By this term, the researchers called attention to contingentinteractions in which students were positioned much as professional biologists might be in situations wherethey are asked to defend, justify, or revise their systems models. Their research drew on a rich tradition ofstudy of recontextualization from interactional sociolinguistics, particularly with respect to the development ofprofessional vision (Goodwin & Goodwin, 1996).

Scope and limitations of design-based research. The products of design-based research—local instructiontheories, design principles, learning environments—may be most useful to other researchers exploring newand emerging areas of learning, in that design-based research can guide what to look for, suggest importantkinds of evidence to gather, and provide interpretive frameworks for analysis (Kelly, 2004). From theperspective of its proponents, being of use to other researchers is not enough, because design-based researchaims to yield practical solutions that can be implemented in real classrooms. To that end, a number of scholarsare engaged in efforts to extend design-based research to problems of implementation (Bryk et al., 2011;Donovan, 2013; Penuel, Fishman, Cheng, & Sabelli, 2011). The aim of these forms of design-based researchis to increase the reproducibility of research in practice settings by developing and testing conjectures abouthow best to support the implementation of innovations.

Critics of design-based research sometimes question the generalizability of findings from smaller-scaledesign-based research. For one, they question whether or not design-based research is the best mode ofinquiry for testing whether observed learning is causally related to specific learning designs (Shavelson,Phillips, Towne, & Feuer, 2003). Design-based researchers might counter that the aim of their research is notto estimate causal impacts, but rather to identify causes of learning evident from analysis of interactions ofstudents with materials, peers, and their teacher in classrooms (e.g., Gravemeijer & Cobb, 2006). Even so,there are not yet agreed-upon standards for selecting classroom episodes for analysis or deciding how totriangulate among different sources of evidence that may point to competing explanations for learning (Kelly,2004).

Studies of Development Over Longer Periods of Time and Across Settings

A second mode of inquiry in educational psychology aims to document and analyze changes to learning anddevelopment over time and conditions that account for those changes. Its methods are principallyobservational, though in some cases, researchers may intervene to promote and observe certain psychologicalphenomena, forms of problem solving and reasoning, or subject matter understandings. In addition,researchers may, as part of their work, attempt to develop validity evidence for the use of particular measuresof learning over time.

Longitudinal Observation Studies Using Large-scale Databases

Many observational studies today make use of the large-scale databases that have been developed in recentdecades to address research questions of fundamental interest to educational psychologists. The power of thesedatabases for inquiry is multifold. Most draw random samples of subjects that represent well-definedpopulations, reducing concerns about external validity (Shadish et al., 2002). In addition, most includemultiple waves of data, which allow researchers to estimate the effects of educational or other experiences onchange over time. Third, many are interoperable, meaning they can be linked to other datasets. This propertyis especially useful for conducting analyses that cross societal sectors, such as education and social services(McLaughlin & O’Brien-Strain, 2008).

Claims longitudinal observation studies can support. Observational studies carried out longitudinally usinglarge-scale databases can support descriptive or causal claims. For example, studies using the WisconsinLongitudinal Study have documented the educational and occupational aspirations and trajectories of thosegraduating from high school in Wisconsin since the 1950s (e.g., Sewell, Hauser, Springer, & Hauser, 2003).Some longitudinal studies develop causal inferences about the relation of psychological processes to outcomes.For example, Chan and Moore (2006) developed evidence for claims about the causal influence of students’beliefs about reasons for school success or failure and use of learning strategies on achievement. Otherlongitudinal studies aim for accurate prediction of future outcomes. For example, Balfanz and colleagues(Balfanz, Herzog, & Mac Iver, 2007) sought to develop “early warning indicators” for high-school dropoutfrom middle-school administrative datasets that could accurately target young people who are off track forgraduation.

Audiences for longitudinal observation studies. The results of observational studies can be useful to policymakers at a range of levels in educational systems. For example, analyses of longitudinal databases can helpidentify new issues and concerns and investigate the plausibility of theories of change associated withparticular interventions (Penuel & Means, 2011). Longitudinal observation studies (e.g., Burstein, 1993) havealso been influential in developing and supporting the adoption of new standards for core subject areas inAmerican schools (McDonnell & Weatherford, 2013). In addition, a number of districts have developed “ontrack” and “early warning systems” on the basis of longitudinal analyses showing the promise of earlyidentification of students for dropout (Kemple, Segeritz, & Stephenson, 2013).

Form and substance of arguments in longitudinal observation studies. The basic form of argument in a

longitudinal observation study is that the changes people exhibit between two time periods are a function ofdifferent measured experiences between those two time points. Inferences from longitudinal observationstudies depend on the capacity of the study to account for alternative explanations of the inferredphenomenon. In the absence of random assignment of subjects to treatments, others may challenge inferenceson the basis of confounding variables not accounted for in the analysis. To inform debate about suchchallenges, analysts of longitudinal observation datasets can use techniques that allow one to estimate howlarge of an effect a confounding variable would have to be to invalidate an inference about a relationshipbetween two variables.

As an example, consider the study by Hong and Raudenbush (2005) that examined effects of graderetention in kindergarten on 11,843 children’s achievement. They found that children who were retained inkindergarten learned less in the following year than those who had been promoted. On the basis ofobservational data, Hong and Raudenbush (2005) inferred that “children who were retained would havelearned more had they been promoted” (p. 200). They accounted for alternative explanations by matchingretained and promoted students for their propensity to be retained based on 207 covariates, including familybackground, emotional disposition, and multiple pretests. Such statistical controls, especially for pretests, havebeen shown to account for 84–94% of bias in an observational study compared to an analogous randomizedexperiment (e.g., Shadish, Clark, & Steiner, 2008).

Although Hong and Raudenbush employed extensive statistical controls, in the absence of randomassignment of subjects to treatments, their inference may still be challenged based on alternative explanationsnot accounted for. Frank and colleagues (Frank, Maroulis, Duong, & Kelcey, 2013) respond to such concernsby quantifying how much an estimated effect would have to be due to bias to invalidate a causal inference. Inparticular, 85% of Hong and Raudenbush’s estimate would have to have been due to bias to invalidate theirinference. As a basis of comparison, controlling for family background, including mother’s education, alteredthe estimated effect by less than 1% (once controlling for pre-tests). Therefore any omitted variable(s) wouldhave to be 85 times more important than family background to invalidate Hong and Raudenbush’s (2005)inference that kindergarten retention reduces achievement. While Frank et al.’s analyses do not change theinference, they contribute to scientific discourse in a research or policy community by quantifying theconditions that would invalidate the inference.

Scope and limitations of observational studies. In recent years, educational psychologists have used a range oflongitudinal databases to study children’s learning and development. For example, McCoach, O’Connell,Reis, and Levitt (2006) employed the Early Childhood Longitudinal Study Kindergarten cohort (ECLS-K)data to study children’s reading growth over the course of kindergarten and first grade. Their analyses focusedon between-school differences in instruction, resource allocation, and student composition; their findingsunderscored the significance of initial differences in reading ability and summer learning loss in accounting forachievement gaps between low-income and more advantaged children. Other databases used by educationalpsychologists include the High School and Beyond Database (e.g., Marsh, 1992), the National EducationalLongitudinal Study (e.g., Jordan & Nettles, 2000; Lan & Lanthier, 2003), the Educational LongitudinalStudy of 2002 (e.g., Fan & Williams, 2010; Fan, Williams, & Corkin, 2011), and the High SchoolLongitudinal Study of 2009 (Willoughby, Adachi, & Good, 2012).

While large-scale longitudinal databases can contribute to policy formation and choices, they haveconsiderable limitations. For one, estimates of effects from observational studies are not theoretically unbiasedas they are from randomized experiments. Second, observational studies rarely provide the close, iterativecontact with practitioners leveraged in design-based research. One must carefully consider the political andscientific contexts in which potentially alternative explanations are conceptualized and measured inobservational studies (e.g., Altonji, Elder, & Taber, 2005). Finally, it may be important to know trajectories ofbehavior or learning prior to receiving a treatment, as well as participants’ state at baseline. For example,Hong and Raudenbush use multiple lagged measurements to control for growth rates prior to the end ofkindergarten. Interrupted time series analyses (Shadish et al., 2002) take this to its logical conclusion,estimating effects based on trajectories leading up to and then after experimental conditions.

Research on Learning Trajectories or Learning Progressions

Another mode of inquiry within educational psychology that analyzes learning over time focuses ondeveloping evidence for learning trajectories (the term used by those studying mathematics) or learningprogressions (a synonymous term used by those studying science) research. A learning trajectory or progressionis a set of testable, empirically supported hypotheses about how students’ understanding develops towardspecific disciplinary goals for learning (Smith, Wiser, Anderson, & Krajcik, 2006). Some researchers trace thisline of research to Piaget’s constructivism (e.g., Simon, 1995), but learning trajectories researchers argue thatone way that their hypotheses differ from traditional stage theories of development is that a trajectoryrepresents one of many possible viable pathways of learning.

Claims learning trajectories and progressions research can support. The claims made in learning trajectories orprogressions are hypotheses pertaining to how student understandings of particular core disciplinary ideas andpractices develop over time (Duncan & Hmelo-Silver, 2009). Each level in a trajectory or progression isdescribed in qualitative terms, relative to some ideal understanding of an idea or mastery of a practice. Levelson a trajectory are ordered developmentally, in two senses of that term. First, higher levels of a trajectoryrepresent more differentiated and hierarchically integrated (e.g., Werner, 1957) understandings and skills.Second, higher levels are imagined as building upon the foundation of levels below, suggesting a possible waythat understanding and skills might emerge over the course of a few weeks, months, or years. For mostlearning trajectories researchers, movement along the trajectory requires encounters with instructionintentionally designed to support learning (Lehrer & Schauble, 2011). At the same time, many also emphasizethe importance of basic developmental capacities for enabling movement along a hypothetical trajectory,particularly at the entry points of that trajectory (Clements & Sarama, 2004).

Audiences and uses for learning trajectories and progressions research. Learning trajectories or progressionsresearch aims to be of use to educational policy makers, curriculum developers, and teachers. For educationalpolicy makers, a key aim is to inform and guide the development of standards and assessments (NationalResearch Council, 2006). Hypothetical learning trajectories are intended to guide the design of standardsaround a small number of core ideas and practices in the disciplines that can be developed across grade levelsand assessed in greater depth. Learning trajectories are also intended to guide curriculum developers inorganizing materials so that they support the development of understanding of these core ideas and skills in

practice over time (Wiser, Smith, & Doubler, 2012). Some researchers intend hypothetical learningtrajectories to be of use to teachers to interpret what students say and do when they engage with tasksintended to support their learning along the trajectory (Lehrer & Petrosino, 2013).

Form and substance of arguments in learning progressions research. Developing evidence to support ahypothetical learning trajectory typically requires many different kinds of studies conducted over many years(Clements, 2007; Lehrer & Schauble, 2011). The process begins with the specification of an initialhypothetical trajectory that draws from past research on learning of the focal core ideas and practices. Somelearning trajectories researchers then proceed to develop and test their hypothetical trajectories through aseries of teaching or design experiments, in which they develop instructional sequences that are designed tosupport student development along the trajectory (e.g., Ni & Zhou, 2005). In this approach, researchersconduct studies to test whether exposure to designed supports does in fact enable students to display the kindsof thinking at the highest levels of the trajectory (e.g., Songer, Kelcey, & Gotwals, 2009). Other researchersdevelop evidence related to an initial trajectory by devising and implementing measures of the trajectory, andthen analyzing the fit of student responses to assessments to the hypothetical trajectory (e.g., Mohan, Chen,& Anderson, 2009). In this approach, when student responses do not fit the pattern predicted by thetrajectories, researchers may revise their hypothetical trajectory and test the revised trajectory in a subsequentstudy (Shea & Duncan, 2013).

A good example of a well-developed learning progression in science focuses on children’s developingunderstanding of how to model phenomena in biological and ecological systems. It focuses on how children developproficiency in the practice of developing and using models to develop and test theories that address particularscientific questions (Lehrer & Schauble, 2006). The researchers developed their hypothetical learningtrajectories over the course of several years and in collaboration with classroom teachers through a series ofteaching experiments, in which the researchers and teachers sought to promote the development of students’scientific modeling practice. The researchers were testing the validity of their approach to cultivating students’scientific modeling practice in a large-scale randomized controlled trial. They compared performance ofstudents in a treatment group to that of students in a coherent, alternative approach to instruction. Outcomeswere measures specially developed to assess learning along the hypothetical progression.

As other progressions do, this progression articulates entry points, intermediate levels of understanding, andtarget understandings related to the practice of modeling. One dimension of the progression is focused onhow children develop and use displays of data to model phenomena. In that dimension, the team has positedthat an entry point into displaying data is to construct displays of data without reference to the question beingposed. Children might group values they obtain from observing or measuring phenomena according toschemes of their own invention (e.g., “We grouped odd and even numbers together”). Over time and withguidance from instruction, students begin to notice or group cases of observation based on similar values andin relation to specific questions they pose. Students whose modeling practice has become sophisticated areable to quantify properties of displays (e.g., through ratios, percentages), recognize that displays provideinformation about a collective or sample of cases, and discuss how general patterns or trends are eitherexemplified by or missing from subsets of cases.

Scope and limitations of research on learning trajectories. Research within this mode of inquiry currently focuses

on a few areas of mathematics and science learning. Standards developers have had to posit many trajectoriesfor areas that do not yet have empirical support. To date, though some assessments of trajectories have strongvalidity evidence (e.g., Gunckel, Covitt, Salinas, & Anderson, 2012), no large-scale assessments used in statesyet employ these trajectories. There are many potential limitations to using learning progressions to guide thedevelopment of large-scale assessments, because they require significant departure from the way traditionaltest items are designed and scored (Alonzo, Neidorf, & Anderson, 2012). Moreover, the design oftrajectories-based assessments often requires the development of interventions to help students engage informs of reasoning and problem-solving characteristic of the highest hypothesized levels of a trajectory(Penuel, Confrey, Maloney, & Rupp, 2014). In addition, when confronting disconfirming evidence,researchers must grapple with uncertainty as to whether or not their hypothetical trajectories are incorrectlyspecified or their assessment tasks are not effective in eliciting student thinking and reasoning (Shea &Duncan, 2013).

Emerging Forms of Educational Psychology and Learning Sciences Research

In this section, we briefly note two emerging areas of inquiry being led by educational psychologists andlearning scientists: in both of these cases, evaluative arguments are still under development. These emergingareas point to expanding conceptions of how and where learning happens. They also underscore the potentialfor new technological tools that can advance understandings of learning and development.

Research on Learning as a Cross-setting Phenomenon

One emerging form of research is the study of how learning takes place across different settings (see Chapter24, this volume). This perspective draws on sociocultural and social practice theories that frame learning anddevelopment as transformations in the ways that people participate in and appropriate the resources ofculturally valued practices (Gutiérrez & Rogoff, 2003; Holland & Lave, 2009; Lave & Wenger, 1991; Rogoff,2003). Educational psychologists and learning scientists who pursue this line of work principally useethnographic methods for “following the person” across different contexts (Jackson, 2011). They focus bothon the agency that people exercise within particular settings of practice and on the social supports learners relyupon for guidance and for brokering access to other (Barron, 2006, 2010; Cooper, Cooper, Azmitia, Chavira,& Gullatt, 2002; Cooper, Denner, & Lopez, 1999; Stevens, O’Connor, Garrison, Jocuns, & Amos, 2008).These researchers also document how different institutions restrict access to learning opportunities by limitingparticipation in particular practices and by valuing some forms of learning over others (Baines, 2012; Bang &Medin, 2010). Other research investigates how groups of people are remaking places in ways that draw on orleverage familiar cultural repertoires for participating in social practices. The hope is that these repertoires canextend access to disciplinary practices taught in schools (Gutiérrez, 2008; Hand, Penuel, & Gutiérrez, 2012).

Educational Data Mining and Learning Analytics

Many new digital learning environments provide researchers with opportunities to make use of student“clicks” or keystrokes both to study basic learning processes and to test interventions. A new field callededucational data mining is emerging that is elaborating and testing methodologies for using large datasets todiscover patterns in learning and find associations between forms of learner engagement in particularinterventions and learning outcomes (Baker & Yacef, 2009; Bienkowski, Feng, & Means, 2012). Researchershave used data-mining techniques to study learning processes when students learn mathematics throughintelligent tutoring systems (Baker, Goldstein, & Heffernan, 2011) and when students learn science as part ofvirtual learning environments, including game-like environments (Gobert, Sao Pedro, Baker, Toto, &Montalvo, 2012).

A mode of inquiry closely related to educational data mining that is emerging is learning analytics (seeChapter 29, this volume). In contrast to educational data mining, a central focus of learning analytics isdeveloping insight into learning from computer-mediated peer interaction and student–teacher interactionduring learning activities (Buckingham Shum & Ferguson, 2012; Wise & Chiu, 2011). Scholars in this areaare also developing methods for studying the emergence of new connections students make among ideas andways of thinking they encounter in digitally mediated learning activities. For example, Shaffer and colleagues(2009) have been using a technique they call Epistemic Network Analysis (ENA) to study how students come

to adopt epistemic frames characteristic of social participation in disciplinary practices of urban planners in thecontext of a game in which learners develop a plan for addressing issues facing a city.

Looking to the Future of Educational Psychology Research

In coming decades, educational psychology and learning sciences research is likely to continue to embrace awide range of modes of inquiry, as it has in the past. At the same time, there will continue to be contest andcontroversy over methods, goals, and strategies of interventions. In this chapter, we have sought to framedifferences in modes of inquiry in terms of different evaluative arguments they can and cannot support. In theprevious section, we introduced new or emerging methodologies, ones that will require and undoubtedly seedevelopment over the coming decade.

In future debates over modes of inquiry, the dialogue must move beyond aphorism and simple dichotomies.The methods researchers use should always “match the question at hand,” but such an aphorism diminishesthe importance of a fully articulated evaluative argument in designing and judging the quality of researchstudies. Moreover, we must go beyond House (1977) and others who dichotomized “qualitative” and“qualitative” forms of reasoning. There are multiple modes of inquiry to consider, including some in whichargumentation relies on an integration of qualitative and quantitative evidence (Johnson & Onwuegbuzie,2004). Third, we must pay closer attention to different conceptions of cause employed by researcherscommitted to different modes of inquiry. Some modes of inquiry reviewed in this chapter emphasize the needfor causal inferences about interventions’ impacts (e.g., Shadish et al., 2002), while others focus onidentification of causes of learning evident from analysis of microprocesses in classrooms (e.g., Gravemeijer &Cobb, 2006). These different notions of cause imply the need for a nuanced evaluation of what kinds of claimsare suitable for what modes of inquiry.

Finally, we anticipate that, in the coming decade, dialogue will continue over how best to reconcile goals forrigor and goals for relevance. Although these two goals need not be opposed, in actuality, rigorous researchcan yield empirical results that are not useful for improving practice, and relevant research can producefindings that are not trustworthy guides to decision making. A potential way forward is to promote morelong-term partnerships between researchers and practitioners that can address persistent problems of practice(Coburn, Penuel, & Geil, 2013; Donovan, 2013). In our view, research policies are needed that embraceevidence standards that call for both rigor and relevance to practice (Gutiérrez & Penuel, 2014). Such policieshave the best potential for supporting the design and conduct of educational psychology research that mightbroadly impact society.

References

Alonzo, A. C., Neidorf, T., & Anderson, C. W. (2012). Using learning progressions to inform large-scale assessment. In A. C. Alonzo & A.W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 211–240). Rotterdam, the Netherlands: SensePublishers.

Altonji, J. G., Elder, T. E., & Taber, C. R. (2005). An evaluation of instrumental variable strategies for estimating the effects of Catholicschooling. Journal of Human Resources, 40(4), 791–821.

Asen, R., Gurke, D., Solomon, R., Conners, P., & Gumm, E. (2011). “The research says”: Definitions and uses of a key policy term in federallaw and local school board deliberations. Argumentation and Advocacy, 47, 195–213.

Baines, A. D. (2012). Positioning, strategizing, and charming: How students with autism construct identities in relation to disability. Disabilityand Society, 27, 547–561.

Baker, R. S. J. d., Goldstein, A. B., & Heffernan, N. T. (2011). Detecting learning moment-by-moment. International Journal of ArtificialIntelligence in Education, 21(1–2), 5–25.

Baker, R. S. J. d., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational DataMining, 1(1), 3–17.

Balfanz, R., Herzog, L., & Mac Iver, D. J. (2007). Preventing student disengagement and keeping students on the graduation path in urbanmiddle-grades schools: Early identification and effective interventions. Educational Psychologist, 42(4), 223–235.

Bang, M., & Medin, D. (2010). Cultural processes in science education: Supporting the navigation of multiple epistemologies. ScienceEducation, 94(6), 1008–1026.

Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecology perspective. Human Development, 49(4),193–224.

Barron, B. (2010). Conceptualizing and tracing learning pathways over time and setting. In W. R. Penuel & K. O’Connor (Eds). Learningresearch as a human science. National Society for the Study of Education Yearbook, 109(1), 113–127.

Bell, P., Hoadley, C., & Linn, M. C. (2004). Design-based research in education. In M. C. Linn, E. A. Davis, & P. Bell (Eds.), Internetenvironments for science education (pp. 73–88). Mahwah, NJ: Erlbaum.

Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics.Draft for public comment. Menlo Park, CA: SRI International.

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings.Journal of the Learning Sciences, 2(2), 141–178.

Bryk, A. S., Gomez, L. M., & Grunow, A. (2011). Getting ideas into action: Building networked improvement communities in education. InM. Hallinan (Ed.), Frontiers in sociology of education (pp. 127–162). Dordrecht, the Netherlands: Verlag.

Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3–26.Burstein, L. (1993). The IEA study of mathematics III: Student growth and classroom processes (Vol. 3). New York, NY: Pergamon.Chan, L. K., & Moore, P. J. (2006). Development of attributional beliefs and strategic knowledge in years 5–9: A longitudinal analysis.

Educational Psychology, 26(2), 161–185.Charmaz, K. (2000). Grounded theory: Objectivist and constructivist methods. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative

research (pp. 509–535). Thousand Oaks, CA: Sage.Clements, D. H. (2007). Curriculum research: Toward a framework for “research-based curricula”. Journal for Research in Mathematics

Education, 38(1), 35–70.Clements, D. H., & Sarama, J. (2004). Learning trajectories in mathematics education. Mathematical Thinking and Learning, 6(2), 81–90.Cobb, P. A., Confrey, J., diSessa, A. A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher,

32(1), 9–13.Coburn, C. E., Penuel, W. R., & Geil, K. (2013). Research–practice partnerships at the district level: A new strategy for leveraging research for

educational improvement. Berkeley, CA and Boulder, CO: University of California and University of Colorado.Cook, T. D. (2002). Randomized experiments in education: Why are they so rare? Educational Evaluation and Policy Analysis, 24(3), 175–199.Cooper, C. R., Cooper, R., Azmitia, M., Chavira, G., & Gullatt, Y. (2002). Bridging multiple worlds: How African American and Latino

youth in academic outreach programs navigate math pathways to college. Applied Developmental Science, 6(2), 73–87.Cooper, C. R., Denner, J., & Lopez, E. M. (1999). Cultural brokers: Helping Latino children on pathways toward success. The Future of

Children, 9, 51–57.Cordray, D. S., & Pion, G. M. (2006). Treatment strength and integrity: Models and methods. In R. R. Bootzin & P. E. McKnight (Eds.),

Strengthening research methodology: Psychological measurement and evaluation (pp. 103–124). Washington, DC: American PsychologicalAssociation.

Cronbach, L. J. (1982). Designing evaluations of educational and social programs. San Francisco, CA: Jossey-Bass.Cronbach, L. J. (1988). Five perspectives on validation argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Hillsdale, NJ:

Erlbaum.Donovan, M. S. (2013). Generating improvement through research and development in educational systems. Science, 340, 317–319.Duncan, R. G., & Hmelo-Silver, C. E. (2009). Learning progressions: Aligning curriculum, instruction, and assessment. Journal of Research in

Science Teaching, 46(6), 606–609.Dynarski, M. (2008). Bringing answers to educators: Guiding principles for research syntheses. Educational Researcher, 37(1), 27–29.Edelson, D. C. (2002). Design research: What we learn when we engage in design. Journal of the Learning Sciences, 11(1), 105–121.Eisenhart, M., & Towne, L. (2003). Contestation and change in national policy on “scientifically based” education research. Educational

Researcher, 32(7), 31–38.Fan, W., & Williams, C. M. (2010). The effects of parental involvement on students’ academic self-efficacy, engagement and intrinsic

motivation. Educational Psychology, 30(1), 54–74.Fan, W., Williams, C. M., & Corkin, D. M. (2011). A multilevel analysis of student perceptions of school climate: The effect of social and

academic risk factors. Psychology in the Schools, 48(6), 632–647.Fishman, B. J., & Krajcik, J. (2003). What does it mean to create sustainable science curriculum innovations? A commentary. Science Education,

87(4), 564–573.Fishman, B. J., Penuel, W. R., Hegedus, S., & Roschelle, J. (2011). What happens when the research ends? Factors related to the sustainability

of a technology-infused mathematics curriculum. Journal of Computers in Mathematics and Science Teaching, 30(4), 329–353.Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S.,... Ji, P. (2005). Standards of evidence: Criteria for efficacy,

effectiveness, and dissemination. Prevention Science, 6(3), 151–175.Frank, K. A. (2000). Impact of a confounding variable on a regression coefficient. Sociological Methods and Research, 29, 147–194.Frank, K. A., Maroulis, S. J., Duong, M. Q., & Kelcey, B. M. (2013). What would it take to change an inference? Using Rubin’s causal model

to interpret the robustness of causal inferences. Educational Evaluation and Policy Analysis, 35(4), 437–460.Gobert, J. D., Sao Pedro, M. A., Baker, R. S. J. d., Toto, E., & Montalvo, O. (2012). Leveraging educational data mining for real-time

performance assessment of scientific inquiry skills within microworlds. Journal of Educational Data Mining, 4(1), 111–143.Goetz, J. P., & LeCompte, M. D. (1984). Ethnography and qualitative design in educational research. Orlando, FL: Academic Press.Goodwin, C., & Goodwin, M. H. (1996). Seeing as situated activity: Formulating planes. In Y. Engestrom & D. Middleton (Eds.), Cognition

and communication at work (pp. 61–95). New York: Cambridge University Press.Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In J. van den Akker, K. Gravemeijer, S. E. McKenney

& N. Nieveen (Eds.), Educational design research (pp. 17–51). New York, NY: Routledge.Green, D. P., Ha, S. E., & Bullock, J. G. (2010). Enough already about “black box” experiments: Studying mediation is more difficult than

most scholars suppose. Annals of the American Academy of Political and Social Science, 628, 200–208.Greeno, J. G., & Middle School Mathematics through Applications Project Group. (1998). The situativity of knowing, learning, and research.

American Psychologist, 53(1), 5–26.Gunckel, K. L., Covitt, B. A., Salinas, I., & Anderson, C. W. (2012). A learning progression for water in socio-ecological systems. Journal of

Research in Science Teaching, 49(7), 843–868.Gutiérrez, K. D. (2008). Developing sociocritical literacy in the third space. Reading Research Quarterly, 43(2), 148–164.Gutiérrez, K. D., & Penuel, W. R. (2014). Relevance to practice as a criterion for rigor. Educational Researcher, 43(1), 19–23.Gutiérrez, K. D., & Rogoff, B. (2003). Cultural ways of learning: Individual traits or repertoires of practice. Educational Researcher, 32(5), 19–

25.Hand, V., Penuel, W. R., & Gutiérrez, K. D. (2012). (Re) framing educational possibility: Attending to power and equity in shaping access to

and within learning opportunities. Human Development, 55, 250–268.Hedges, L. V. (2013). Recommendations for practice: Justifying claims of generalizability. Educational Psychology Review, 25(3), 331–337.Holland, D., & Lave, J. (2009). Social practice theory and the historical production of persons. Actio: An International Journal of Human Activity

Theory, 2, 1–15.Hong, G., & Raudenbush, S. W. (2005). Effects of kindergarten retention policy on children’s cognitive growth in reading and mathematics.

Educational Evaluation and Policy Analysis, 27(3), 205–224.House, E. R. (1977). The logic of evaluative argument. Center for Study of Evaluation Monograph Series in Evaluation. Los Angeles, CA: Center

for the Study of Evaluation, University of California.House, E. R. (1979). Coherence and credibility: The aesthetics of evaluation. Educational Evaluation and Policy Analysis, 1(5), 5–17.Jackson, K. (2011). Approaching participation in school-based mathematics as a cross-setting phenomenon. Journal of the Learning Sciences,

20(1), 111–150.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher,33(7), 14–26.

Jordan, W. J., & Nettles, S. M. (2000). How students invest their time outside of school: Effects on school-related outcomes. Social Psychology ofEducation, 3, 217–243.

Jurow, A. S., Hall, R., & Ma, J. Y. (2008). Expanding the disciplinary expertise of a middle school mathematics classroom: Re-contextualizingstudent models in conversations with visiting specialists. Journal of the Learning Sciences, 17(3), 338–380.

Kelly, A. E. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences, 13(1), 113–128.Kelly, A. E., & Yin, R. K. (2007). Strengthening structured abstracts for education research: The need for claim-based structured abstracts.

Educational Researcher, 36(3), 133–138.Kemple, J. J., Segeritz, M. D., & Stephenson, N. (2013). Building on-track indicators for high school graduation and college readiness:

Evidence from New York City. Journal of Education for Students Placed at Risk (JESPAR), 18(1), 7–28.Lan, W., & Lanthier, R. (2003). Changes in students’ academic performance and perceptions of school and self before dropping out of school.

Journal of Education for Students Placed at Risk, 8(3), 309–332.Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, MA: Harvard University Press.Lehrer, R., & Petrosino, A. (2013). A learning progression in data modeling emerges in a trading zone of professional community and identity. Paper

presented at the Waterbury Summit, Pennsylvania State University, State College, PA.Lehrer, R., & Schauble, L. (2006). Cultivating model-based reasoning in science education. In R. K. Sawyer (Ed.), Cambridge handbook of the

learning sciences (pp. 371–388). Cambridge, MA: Cambridge University Press.Lehrer, R., & Schauble, L. (2011). Designing to support long-term growth and development. In T. Koschmann (Ed.), Theories of learning and

studies of instructional practice (pp. 19–38). New York: Springer.MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and

other intervening variable effects. Psychological Methods, 7(1), 83–104.Marsh, H. W. (1992). Extracurricular activities: Beneficial extension of the traditional curriculum or subversion of academic goals. Journal of

Educational Psychology, 84(4), 553–562.McCoach, D. B., O’Connell, A. A., Reis, S. M., & Levitt, H. A. (2006). Growing readers: A hierarchical linear model of children’s reading

growth during the first 2 years of school. Journal of Educational Psychology, 98(1), 14–28.McDonnell, L. M., & Weatherford, M. S. (2013). Evidence use and the Common Core State Standards Movement: From problem definition

to policy adoption. American Journal of Education, 120(1), 1–25.McLaughlin, M. W., & O’Brien-Strain, M. (2008). The Youth Data Archive: Integrating data to assess social settings in a societal sector

framework. In M. Shinn & H. Yoshikawa (Eds.), Toward positive youth development: Transforming schools and community programs (pp. 313–332). New York: Oxford University Press.

Mohan, L., Chen, J., & Anderson, C. W. (2009). Developing a multi-year learning progression for carbon cycling. Journal of Research in ScienceTeaching, 46(6), 675–698.

Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference. London, UK: Cambridge University Press.Muller, C., Riegle-Crumb, C., Schiller, K. S., Wilkinson, L., & Frank, K. A. (2010). Race and academic achievement in racially diverse high

schools: Opportunity and stratification. Teachers College Record, 112(4), 1038–1063.National Research Council. (2006). Systems for state science assessment. Washington, DC: National Academies Press.Ni, Y., & Zhou, Y.-D. (2005). Teaching and learning fraction and rational numbers: The origins and implications of whole number bias.

Educational Psychologist, 40(1), 27–52.Osborne, J. W. (Ed.). (2008). Best practices in quantitative methods. Thousand Oaks, CA: Sage.Penuel, W. R., Confrey, J., Maloney, A., & Rupp, A. A. (2014). Design decisions in developing assessments of learning trajectories: A case

study. Journal of the Learning Sciences, 23(1), 47–95.Penuel, W. R., Fishman, B. J., Cheng, B., & Sabelli, N. (2011). Organizing research and development at the intersection of learning,

implementation, and design. Educational Researcher, 40(7), 331–337.Penuel, W. R., Gallagher, L. P., & Moorthy, S. (2011). Preparing teachers to design sequences of instruction in Earth science: A comparison of

three professional development programs. American Educational Research Journal, 48(4), 996–1025.Penuel, W. R., & Means, B. (2011). Using large-scale databases in evaluation: Advances, opportunities, and challenges. American Journal of

Evaluation, 32(1), 118–133.Penuel, W. R., & Spillane, J. P. (in press). Learning sciences and policy design and implementation: Key concepts and tools for collaborative

engagement. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (2nd ed.). Cambridge, UK: Cambridge University Press.Powell, D. R., Diamond, K. E., Burchinal, M. R., & Koehler, M. J. (2010). Effects of early literacy professional development intervention on

Head Start teachers and children. Journal of Educational Psychology, 102(2), 299–312.

Reimann, P. (2011). Design-based research. Methodological Choice and Design, 9(2), 37–50.Reinking, D., & Bradley, B. A. (2008). Formative and design experiments: Approaches to language and literacy research. New York, NY: Teachers

College Press.Rogoff, B. (2003). The cultural nature of human development. New York: Oxford University Press.Roschelle, J., Kaput, J., & Stroup, W. (2000). SimCalc: Accelerating students’ engagement with the mathematics of change. In M. Jacobson &

R. Kozma (Eds.), Innovations in science and mathematics education: Advanced designs for technologies of learning (pp. 47–75). Hillsdale, NJ:Earlbaum.

Roschelle, J., Pierson, J., Empson, S., Shechtman, N., Dunn, M., & Tatar, D. (2010a). Equity in scaling up SimCalc: Investigating differencesin student learning and classroom implementation. In K. Gomez, L. Lyons & J. Radinsky (Eds.), Learning in the disciplines: Proceedings ofthe 9th International Conference of the Learning Sciences (Vol. 1, pp. 333–340). Chicago, IL: International Society of the LearningSciences.

Roschelle, J., Shechtman, N., Tatar, D., Hegedus, S., Hopkins, B., Empson, S.,... Gallagher, L. P. (2010b). Integration of technology,curriculum, and professional development for advancing middle school mathematics: Three large-scale studies. American Educational ResearchJournal, 47(4), 833–878.

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5),688–701.

Ruiz-Primo, M. A., Shavelson, R. J., Hamilton, L. S., & Klein, S. (2002). On the evaluation of systemic science education reform: Searchingfor instructional sensitivity. Journal of Research in Science Teaching, 39(5), 369–393.

Sandoval, W. A. (2004). Developing learning theory by refining conjectures embodied in educational designs. Educational Psychologist, 39(4),213–223.

Sandoval, W. A. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23(1), 18–36.

Sandoval, W. A., & Bell, P. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist,39(4), 199–201.

Schultema, J., Veugelers, W., Rijlaarsdam, G., & ten Dam, G. (2009). Two instructional designs for dialogic citizenship education: An effectstudy. British Journal of Educational Psychology, 79(3), 439–461.

Sewell, W. H., Hauser, R. M., Springer, K. W., & Hauser, T. S. (2003). As we age: A review of the Wisconsin Longitudinal Study, 1957–2001. Research in Social Stratification and Mobility, 20, 3–111.

Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experimentcomparing random and nonrandom assignments. Journal of the American Statistical Association, 103(484), 1334–1344.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston,MA: Houghton-Mifflin.

Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E.,... Mislevy, R. J. (2009). Epistemic network analysis: Aprototype for 21st century assessment of learning. International Journal of Learning and Media, 1(2), 33–53.

Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1),25–28.

Shea, N. A., & Duncan, R. G. (2013). From theory to data: The process of refining learning progressions. Journal of the Learning Sciences, 22(1),7–32.

Simon, M. A. (1995). Reconstructing mathematics pedagogy from a constructivist perspective. Journal for Research in Mathematics Education, 26,114–145.

Smith, C. L., Wiser, M., Anderson, C. W., & Krajcik, J. (2006). Implications of research on children’s learning for standards and assessment:A proposed learning progression for matter and the atomic-molecular theory. Measurement: Interdisciplinary Research & Perspective, 4(1&2),1–98.

Songer, N. B., Kelcey, B., & Gotwals, A. W. (2009). How and when does complex reasoning occur? Empirically driven development of alearning progression focused on complex reasoning about biodiversity. Journal of Research in Science Teaching, 46(6), 610–631.

Spybrook, J. K., Raudenbush, S. W., Liu, X., Congdon, R., & Martinez, A. (2009). Optimal design for longitudinal and multilevel research:Documentation for the “Optimal Design” software. New York: William T. Grant Foundation.

Stevens, R., & Hall, R. (1998). Disciplined perception: Learning to see in technoscience. In M. Lampert & M. L. Blunk (Eds.), Talkingmathematics in school: Studies of teaching and learning (pp. 107–149). Cambridge: Cambridge University Press.

Stevens, R., O’Connor, K., Garrison, L., Jocuns, A., & Amos, D. M. (2008). Becoming an engineer: Toward a three dimensional view ofengineering learning. Journal of Engineering Education, 97(3), 355–368.

Tatar, D., Roschelle, J., Knudsen, J., Schectman, N., Kaput, K., & Hopkins, B. (2008). Scaling up innovative technology-based mathematics.

Journal of the Learning Sciences, 17(2), 248–286.Toulmin, S. (1958). The uses of argument. New York: Cambridge University Press.Vadasy, P. F., Sanders, E. A., & Peyton, J. A. (2006). Code-oriented instruction for kindergarten students at risk for reading difficulties: A

randomized field trial with paraeducator implementers. Journal of Educational Psychology, 98(3), 508–528.van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, K. Branch, K. Gustafson, N. Nieveen & T.

Plomp (Eds.), Design approaches and tools in education and training (pp. 1–14). Dordrecht: Kluwer.Werner, H. (1957). The concept of development from a comparative and organismic point of view. Minneapolis, MN: University of Minnesota Press.Willoughby, T., Adachi, P. J., & Good, M. (2012). A longitudinal study of the association between violent video game play and aggression

among adolescents. Developmental Psychology, 48(4), 1044–1057.Wise, A. F., & Chiu, M. M. (2011). Analyzing temporal patterns of knowledge construction in a role-based online discussion. International

Journal of Computer-Supported Collaborative Learning, 6(3), 445–470.Wiser, M., Smith, C. L., & Doubler, S. (2012). Learnng progressions as tools for curriculum development: Lessons from the Inquiry Project.

In A. C. Alonzo & A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 359–404). Rotterdam,the Netherlands: Sense Publishers.

Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage.

3The Work of Educational Psychologists in a Digitally

Networked WorldPUNYA MISHRA1

MATTHEW J. KOEHLER

CHRISTINE GREENHOW

Michigan State University New digital technologies have had a dramatic effect on all arenas of human work, and the work of educationalpsychologists is no exception. There are many ways in which we can think about the role that technologiesplay in what we do as scholars and researchers of educational psychology. Technology, for instance, haschanged how we think about teaching and learning in two key ways. First, it has influenced the kinds ofmodels and theories we have of the mind (from clay tablets to digital computers). Second, technology haschanged the ecology or contexts within which learning occurs to include several intersecting spaces (temporal,spatial, home, community, online, etc.).

A close examination of the relationship between technology and the work of educational psychologistsreveals changes in nearly every aspect of the work that educational psychologists do. Thus, we have organizedthis chapter according to the role that technology has played in the everyday activities of educationalpsychologists, grouped into eight general categories, briefly described below.

Study phenomena. Educational psychologists study phenomena and contexts where teaching and learningoccur. New technologies provide new phenomena and contexts for teaching and learning through theadvent of social media, games, and virtual learning environments. These new learning environments alsoprovide new kinds of data and new techniques for data analysis.Design studies. Technology affords new forms of research designs allowing, for example, researchers totrack individual behavior through online environments, provide tailor-made inputs to individual students(or groups of students), and develop new models of simulation and modeling of virtual learners.Collect data. New technologies have afforded new types of data to researchers (including data fromeducational neuroscience, simulated data, eye-tracking data, video data, and social network data), leading tochanges in data collection.Assess learning. Digital technologies offer new possibilities and opportunities for assessment of learningthrough the design of new assessment tasks and the power of large-scale assessment through automatedscoring, immediate reporting, and improved feedback.Analyze data. New technologies lead to new forms of data analysis—offering tools that provide greaterpower and efficiency in how quantitative and qualitative analyses are conducted.

Develop theories. The development of sound, predictable, data-driven theories is paramount to the conductof research in educational psychology. Some of the consequences of the inclusion of digital technologies aredesign-based research, testing boundary conditions for the application of theories, and questioning thevalue of theory itself.Read, write, publish, and disseminate ideas. Today’s educational psychologists must consider how newtechnologies have contributed to changes in publishing, accessibility, and scholarship.Confront ethical issues. As in all research, new technologies have brought about a new range of ethicalissues that educational psychologists have to contend with (such as those related to data security and newregulations concerning institutional review).

We explore the changes occurring at the intersection of educational psychology and technology in thesections below, which correspond with the eight general categories of activity of educational psychologists.Some of these categories are more specific to the work of educational psychology scholars than others. Forinstance, under “Study phenomena,” we explore phenomena now available for investigation by educationalpsychologists due to technological change, but, under “Read, write, publish, and disseminate ideas,” thechanges we discuss apply more generally to scholars in many, if not all, disciplines. That said, we focusattention on issues specifically impacting the work of educational psychology researchers, paying less attentionto issues that have broader implications across all fields of study. We limit our discussion of the collaborativework educational psychologists do facilitated by technological tools, and refrain from discussing newtechnology-based sources of data such as electroencephalograms (EEGs), functional magnetic resonanceimaging (fMRI), and positron emission tomography (PET) scans in the burgeoning subfield of educationalneuroscience. Although these are valuable for educational psychology research, we have not addressed theseissues in the present chapter for reasons of space as well as the fact that these topics are addressed in otherchapters in this handbook (see Chapters 5, 25, and 26 in this volume).

Study Phenomena

New technologies have significantly impacted the phenomena we study as researchers in two primary ways.First, technology has introduced a host of new phenomena worthy of research through the advent of socialmedia, games, and virtual learning environments. Second, it has shifted traditional dichotomies, such asinformal versus formal, and created new ones, such as virtual versus physical and online versus offline. Byintroducing new phenomena, technology has often shifted the landscape of these “boundaries,” thuscomplicating what on the surface may appear to be somewhat simplistic dichotomies.

Social Networks

Technological advancements have contributed increasingly to people’s adoption of social media, a term oftenused to refer to online technologies and applications which promote people, their interconnections, and user-generated content (Cormode & Krishnamurthy, 2008). Among the many different kinds of social media, ofparticular interest to educational psychologists are social network sites, including Facebook, LinkedIn, GooglePlus, and Twitter, which are dominant in the early decades of the twenty-first century. Such social networksites typically feature the ability to consume, produce, or interact with streams of user-generated contentprovided by one’s connections (Ellison & boyd, 2013).

Social networks offer educational psychologists the opportunity to study a wide range of empirical questionssuch as how these networks factor into, shape, and are shaped by the learning ecology of their participants(Barron, 2006). Social networks are increasingly being used in virtually all areas of pedagogy (Manca &Ranieri, 2013; Ranieri, Manca, & Fini, 2012). For instance, scholars have studied how online socialnetworking can facilitate new forms of collaboration not feasible with traditional communication technologies(Greenhow & Li, 2013) and the use of social media for teachers’ professional development (Ranieri, Manca,& Fini, 2012). This work suggests possibilities for educational designs powered by social media within avariety of learning and teaching contexts as well as a revisiting of conventional learning theories as they playout in such contexts. For instance, in studying social networks, scholars have found that social links indicatedin automatically generated and dynamically updated network graphs (e.g., Facebook visualizations) are notvalid indicators of real user connection as previous research using social graphs from physical observations ofin-person interactions would suggest (Wilson, Sala, Puttaswamy, & Zhao, 2012). Other scholars haveexamined how aspects of computer-supported collaborative learning theory, generated in other collaborativespaces, are contradicted in social network sites (Judele, Tsovaltzi, Puhl, & Weinberger, 2014). Such studiessuggest how educational psychology research may shift, requiring more accurate modeling to evaluate socialnetwork phenomena in light of new technologies (see Chapter 25 in this volume).

Games

Although the educational possibilities of learning from games have been conjectured and studied throughouthistory, the advent of digital games is a relatively recent phenomenon with tremendous economic, cultural,and social implications (Squire, 2006). Educational psychologists have studied the cognitive, social, andemotional impacts (both positive and negative) of game playing under various conditions. On the positiveside, research has shown playing computer games can enhance cognitive processes such as perception,

attention, and cognition (Anderson & Bavelier, 2011); reaction time (Karle, Watter, & Shedden, 2010); andmental rotation (Sims & Mayer, 2002). Games have also been shown to have some success in transferringlearners’ skills to “real-world” situations, including flight training, the training of surgeons, the care ofdiabetes, and the development of prosocial behavior (Tobias, Fletcher, & Wind, 2014). On the other end ofthe spectrum are concerns that game play is often associated with lower school achievement (Gentile, 2011)and negative behaviors such as aggression (Tobias et al., 2014).

The important issue for educational psychologists is that these game interactions can have significantpsychological consequences because they occur in environments characterized by pretense, virtuality, distance,and mediation. Learning in these networked, digital spaces often occurs through active participation in thegame’s virtual social structures (Salen & Zimmerman, 2004) and is evaluated through actual performance—adifferent manner of engaging in learning than in a traditional learning environment such as the classroom.

Virtual Environments

Virtual environments are systems where individuals interact with simulated objects, people, or environments.Virtual worlds represent one type of virtual environment. In virtual worlds users are often identified by two- orthree-dimensional representations called avatars and communicate with each other using text, visual gestures,and sound. Educational psychologists can explore how such environments integrate with, intensify, orcontradict learning and teaching in physical environments and explore learners’ negotiation of identitieswithin and between these spaces (Tettegah & Calongne, 2009). Moreover, virtual environments form anintegral component of the growing contemporary use of online education.

Online Education

Online education is fast becoming an alternative mode of teaching and learning and a supplement totraditional face-to-face education (Picciano & Seaman, 2009). Online education may consist of wholly onlinecourses or hybrid or blended courses that combine online components with traditional face-to-facecomponents. Most recently, online education has seen the rise of massive open online courses (MOOC), aterm referring to online courses targeting large-scale interactive participation and open access via the internet.Regardless of format (wholly online, blended, or MOOC), online courses may consist of traditional courseresources such as readings, videos, tools to facilitate synchronous and asynchronous participation, and coursemanagement systems.

The rise of online education offers new phenomena for educational researchers to examine. Researchershave examined issues such as the effectiveness of online instruction compared to face-to-face instruction,practices associated with effective online learning, and factors that influence the effectiveness of onlinelearning (Means, Toyama, Murphy, Bakia, & Jones, 2010). Additionally, approaches in online education(particularly MOOCs) have the potential to generate large datasets—through both the content people uploadand the behavioral traces (such as log files) they leave behind—which can be mined for patterns and used totest learning and teaching theories at a scale not previously seen.

Design Studies

In many ways, how we design studies is at the heart of what research is and of what we do as educationalpsychologists, academics and scholars; this issue therefore drives the central issues in each of the eightcategories of work that educational psychologists do. In this section, however, we focus on three new contextsthat digital and networking technologies have created for designing new studies and two important researchdesign strategies that digital environments provide.

Studies in Virtual Worlds

One new context that networking technologies have provided is the online virtual world—digitalenvironments where people can work and interact in a somewhat realistic manner. Research contexts includeexisting recreational, multi-user, virtual worlds that have been adopted for educational purposes (e.g., ActiveWorlds or Second Life) or worlds designed specifically for educational purposes, such as River City (Clarke,Dede, Ketelhut, & Nelson, 2006). Virtual worlds make attractive research environments because they can bedesigned to automatically generate data as users interact with the world (e.g., activities most performed, timeon task, content generated by users). Designed studies of virtual worlds can examine how well pedagogicalapproaches used in other settings function in these worlds. They can also test prominent learning theories,such as theories of self-directed learning and motivation; compare learning and teaching processes andoutcomes in-world and out; and explore the co-evolution (or contradiction) of learning and design (De Lucia,Francese, Passero, & Tortora, 2009). Virtual worlds designed for education can be studied in terms of howwell they help learners understand disciplinary concepts (e.g., scientific reasoning: see Chapter 24 in thisvolume), to test theories of how people learn and teach (Bransford, Brown, & co*cking, 2000), and to explorehow learning, pedagogical, and design theories co-evolve and shape one another over successive iterations ofvirtual-world participation and design revisions.

Simulations and Modeling as Experimentation

A second related context that can provide expanded sites for research are simulations and other forms ofcomputer-generated modeling. Simulations are constructed worlds that are a close representation of thephysical world governed by the same rules. Simulations and simulated labs (e.g., virtual frog dissections inscience education) may be useful where repeated practice is required or where the actual physical experimentwould be too costly, time consuming, or otherwise impractical to enact in real life. Simulations have been usedto illustrate key principles in disciplines such as biology, chemistry, physics, and earth and space science.Studies can be designed to examine whether and how simulations help learners understand disciplinaryconcepts. For example, studies of the simulation environment NetLogo have investigated middle- and high-school students’ derivation of the ideal gas law from microlevel interactions among gas particles in a box(Wilensky, 2003); creating and testing models of predator–prey interactions (Wilensky & Reisman, 2006);and exploring the rates and directions of chemical reactions for individual molecules (Stieff & Wilensky,2003). Studies can also be designed to compare learners’ outcomes following simulations versus hands-on labexperiences (Ma & Nickerson, 2006). As technology improves, so does the fidelity of the simulations,providing ever-greater opportunities for future research (see Chapter 20 in this volume).

Online Education and Massive Open Online Courses

The rise of online education and MOOCs targeting large-scale interactive student participation, open accessvia the internet, interorganizational collaboration, and the generation of big datasets provides opportunitiesfor interdisciplinary, intercultural research on a scale not previously seen.

Though relatively new, the potential for research on MOOCs is immense. MOOCs allow for thedevelopment of learning analytics that can be used for adaptation and personalization of curriculum throughpredictive modeling and forecasting of learner behavior and/or achievement or for the application of socialnetwork analysis techniques to optimize learner interactions. Insights generated from such studies maycontribute to new theoretical models, such as models of self- and peer-assessment, as well as to the design ofautomated mechanisms to support and augment students’ learning goals and processes.

Designing Studies with Big Data

What is common to all of the technological contexts described above is that users leave complex traces of theirinteraction with the environment, the content, and with each other and thus generate large and complexdatasets. By employing a combination of modern artificial intelligence, machine learning, and statisticaltechniques, these datasets can be examined in a variety of ways to reveal relationships, patterns, and insightsnot easily discoverable through standard database management tools or data-processing applications.Coinciding with the rise of big data, learning analytics is a recent area of scholarship that seeks to collect,analyze, and report data “about learners and their contexts, for purposes of understanding and optimizinglearning and the environments in which it occurs” (Siemens & Long, 2011, p. 4).

However, designing studies involving big datasets can also be problematic. Designed studies canoversimplify complex human actions and motivations, magnify data errors when multiple datasets arecombined, and create divides between those who have access to big data and those who do not (boyd &Crawford, 2011). Additional challenges include establishing norms for collaborating across big data projects,creating ways to measure and reward individual contributions, and defining the most pressing problems; thatis, distinguishing the needle from the big data haystack.

Studies in which Every Participant gets a Tailor-made Condition

Newer digital technologies also enable educational psychologists to design studies in which each subject isassigned a custom experimental condition. For instance, diagnostic educational gaming environments thatunlock levels of game play based on how and how well individuals progress through the game can provideeach subject with a tailor-made condition. Similarly, different versions of an online course that are randomlyassigned to learners can allow for true experiments to test different interventions or theoretical frameworks.Such technologies suggest the promise of tailoring research conditions for individual participants.

Collect Data

With new technologies come new types of data. Changes in the available types of data also bring aboutchanges in the focus of researchers’ attention, the methods they use to study these phenomena, and the typesof questions they ask. Key possibilities for new types of data afforded to educational psychologists bytechnological advances are highlighted in the sections that follow.

Data from the World Wide Web

Since the advent of the first widely available web browser (Mosaic) in 1993, the internet had grown to over2.3 billion users by the year 2014. Along with the explosion in the number of users, websites, domain names,and sophisticated designs, the types of data available as tools have also dramatically increased for businesses,marketers, and, more recently, researchers.

Web analytics are the data collected automatically by web servers to track visitors’ interactions and behaviorswith websites. When combined with data from other websites and browser tracking (via cookies and sessiondata), information can be generated about the visitor to a website, including the visitor’s prior browsinghistory, likes and dislikes, sex, age, income, ethnicity, and purchasing history. Another source of data from theinternet comes from a technique called web content mining or web scraping (Bharanipriya & Prasad, 2011). Inthis approach, data are gathered or extracted from websites via automated processes. These techniques ofgenerating big data have the same strengths and weaknesses described above.

Simulated Data

Not all data come from direct observation of phenomena or derived measures. The rise of computingtechnology has driven increased use of simulated data, data generated by computing processes that simulatedata that might be otherwise difficult to obtain. This technique has been commonly used in statistics to testthe properties of many statistical procedures. For example, in simple statistical analyses such as the t-test, thestatistical power (type II error) can be computed exactly from formulas if all statistical assumptions are met.Other procedures, like non-parametric statistical analyses, do not have easily computed type II values, becausethe procedure depends so heavily on the type of data to be analyzed. In these cases, Monte Carlo techniques(Kalos & Whitlock, 2009), a type of simulated approach, can be used to generate many samples of the kind ofdata expected. The statistical procedure is then run on these simulated data, repeatedly, in order to establishthe rate at which the null hypothesis is rejected. This rate is an estimate of the type II error rate (for examplesof this, see Mumby, 2002; Muthén & Muthén, 2002).

Given recent advances in computing power, Monte Carlo techniques will soon become more commonplacein other arenas of social science. Bakker, van Dijk, and Wicherts (2012) explored how researchers have foundstatistically significant results 96% of the time, when on the surface there is insufficient statistical power tosupport rejecting the null hypothesis at such a high rate. Generating data for multiple studies under varyingeffect sizes, sample sizes, and research practices (analyzing more than one variable, sequential testing, splittingstudies, and removing outliers), these researchers found that true type I error rate may be as high as 0.40 usingsuch practices and may explain why 96% of studies report significant results.

User Data Capture

Traditional methods of research in the educational sciences, such as think-aloud protocols, interviews, surveys,and observations, rely on second-hand or indirect data. Technology is increasingly providing opportunities forresearchers to collect first-hand data from participants. For example, a tool like Morae allows researchers tosimultaneously record screen captures, audio and video data (typically of users interacting with the system),and mouse and keyboard clicks. Morae also has built-in analyses that detect patterns in the massive amountsof data that such recordings can generate.

The challenge with data generated from seamlessly recording participants’ interactions with systems is that,even when built-in analyses detect patterns in the massive amount of data captured, researchers must decipherwhat such patterns mean. It is one thing to know that at a particular moment in time a user clicked on aparticular portion of the screen; it is another matter completely to figure out why the user did so and what thatinteraction means. More research is needed to help uncover patterns in the data and ascribe meaning to data.

Eye-Tracking Data

One type of user data capture, eye-tracking data, is particularly noteworthy. The previously mentioned formsof user data capture all record what participants do—where they click, what they say, and what they are doing.By recording the movements of the human eye, researchers can gain insight into what participants payattention to, whether or not that attention is brief or extended, and what may be interesting to participants(Duchowski, 2007). For example, Galesic, Tourangeau, Couper, and Conrad (2008) used eye-tracking data toinvestigate possible causes for response order effects (i.e., that survey responses tend to be skewed in favor ofresponses presented earlier in the list of choices). They found that participants often take cognitive shortcuts.The most salient of these is that participants tend to devote more attention to earlier choices and less(sometimes none) to later choices.

The cost of procedures for eye tracking is consistently declining, increasing access of these data toresearchers. This newfound access to eye-tracking data has fueled the development of new analysis tools and,in many cases, add-ons to existing data analysis tools. For example, Morae offers several plugins andextensions to seamlessly integrate eye-tracking data with key-logging data, allowing for researchers tosynchronize participants’ actions with their perceptions (Alves, Pagano, & da Silva, 2010).

Video Data

Dramatic changes in affordability, availability, storage, and quality of video have led researchers to routinelyuse it as research data. Tools such as Transanna, Morae, DIVER, and ATLAS have been developed to helpresearchers organize, code, analyze, and connect video data to other data (e.g., transcripts, interviews,qualitative analyses).

The new affordances that video data offer to researchers bring new challenges. Derry et al. (2010) identifiedfour challenges to researchers using video data: (a) how to select specific elements to focus study on withincomplex settings or large corpuses of video; (b) choosing what analytic frameworks to guide the analysis ofvideo; (c) choosing the appropriate technology to organize, store, and analyze the video; and (d) adhering toappropriate ethics involving consent and use of video data while at the same time promoting sharing andcollaboration.

Data from Social Networks

Online social networks present a number of novel types of data available to researchers. These include the“network” or community itself as a representation of connections between individuals, which can be depictedas a network graph summarizing the “degrees of separation” between people in a community. Types of dataavailable from these networks include the content of the interaction (their messages or photos) and socialtagging (short descriptions or signifiers of content) that occurs (Aggarwal, 2011). Other types of data thatemerge from social networks include reputation systems, badge systems, and influence scores. These measurescapture and reward useful user behavior—such as completing a task, helping others, and so on. Clearly these“reward” structures are important to researchers in that they provide meaningful information about userbehavior and interactions (see Chapter 25 in this volume).

Assess Learning

Digital technologies also offer new possibilities and opportunities for the assessment of learning, primarilythrough the design of new assessment tasks as well as the power of large-scale assessment through automatedscoring, immediate reporting, and improved feedback. One fundamental challenge, however, faced by allassessment techniques (irrespective of the use of technology) is making the assessment tasks valid and reliable,even while making them amenable to computational analysis. For instance, computers are good at evaluatingresponses to tightly constrained questions, such as multiple-choice questions, and less effective whenevaluating open-ended, constructed responses, such as the traditional essay. Though the nature of multiple-choice questions does not preclude the measurement of higher-order thinking skills, there is a general beliefthat such constrained questions typically focus on measuring lower-order skills. The demand for alternativeassessments comes from both a skepticism toward multiple-choice assessments as well as the push towardsmore authentic, performance-based assessments (see Chapter 29, this volume).

Scalise and Gifford (2006) offered a taxonomy that may be useful in computer-based assessment consistingof 28 item types “based on 7 categories of ordering involving successively decreasing response constraints fromfully selected to fully constructed,” (p. 3). At the most constrained end of the spectrum are multiple-choicequestions, while at the other end are assessment types that seek to measure student performance undersimulated or real conditions. The five intermediate categories fall along this dimension and are classified as:(a) selection/identification; (b) reordering/rearrangement; (c) substitution/correction; (d) completion; and (e)construction types. They also suggest that the 28 types of assessment they describe within these seven broadcategories are not necessarily comprehensive in that a variety of other item formats can be designed bycombining some of the types listed or through including new media formats such as video, audio, andinteractive graphics (e.g. animations or simulations). Two areas (from opposite ends of the constraintspectrum) that have received significant attention are computer-adaptive testing (CAT) and automated textanalysis.

Computer-adaptive Testing

CAT is the computer-based extension of the adaptive testing started with Binet in 1905 (Linacre, 2000). Theterm encompasses a wide range of assessment approaches administered on a computer, where the testdifficulty is adaptively targeted to match the proficiency of the test taker in order to provide the best and mostefficient assessment of abilities (Luecht, 2005). Behind the scenes, item response theory (IRT) is typicallyused to judge the relative difficulty of items, select the next items for test takers to receive, and equate itemsacross test takers.

As CAT approaches become more commonplace, especially in the context of high-stakes testing, there areimplications for educational psychologists. CAT approaches are generally considered to be more accurateassessments of skill (Thissen & Mislevy, 2000) but at the same time do not produce tests that can be strictlyequated across test takers. CAT approaches offer possibilities for fast or immediate test results for test takersand can easily scale up to large participant pools. Developing the test, however, can be a time-consuming andcostly endeavor as CAT approaches require the development of many more items that require large amountsof pilot data to be properly equated using IRT models.

Automated Text Analysis

Automated essay evaluation, which is derived from automated essay scoring, is the “process of evaluating andscoring written prose via computer programs” (Shermis & Burstein, 2003, p. 7). The approach uses advancesin natural language processing, applied mathematics, machine learning, and computational linguistics toanalyze syntax, word usage, discourse structure, and higher-level meaning such as thematic analysis. Forexample, latent semantic analysis (LSA) is an early approach that performed statistical computations on thesimilarity of all the meanings in a large text, which was then used to approximate writing coherence and thequantity and quality of the writer’s knowledge (Landauer, Foltz, & Laham, 1998). More recent andsophisticated approaches include the E-Rater system employed by the Educational Testing Service in many ofits high-stakes tests; Coh-Metrix (Graesser, McNamara, & Kulikowich, 2011), which provides multiple-levelindices of text coherence; and LightSIDE (Mayfield & Rosé, 2013), which provides open-source machinelearning software customizable to many different evaluation purposes. These more advanced approachesmerge combinations of LSA, feature extraction (word occurrences, word dyads and triads, parts of speech),machine learning to train underlying models, and multi-level evaluation. For example, Coh-Metrix cangenerate over 100 indices (features) from a given text, which are in turn used in formulae to compute variousmetrics of text coherence, which some researchers have used to make direct judgments about the quality ofwritten texts.

The growing popularity of such approaches has important implications for educational psychologists. Onone hand, there are clear-cut advantages in terms of efficient data analyses that are increasingly becoming asreliable as (and less opaque than) human raters (Shermis & Burstein, 2013). On the other hand, there arelegitimate concerns about an undue emphasis on product over process, a focus on the wrong qualities ofwriting (e.g., its function as expression), and a philosophical concern about the equivalence between howhuman raters and machine raters make judgments.

Analyze Data

Data in educational psychology are often used in two ways that need to be carefully delineated (Behrens &Smith, 1996). Behrens and Smith call the first level the “data of phenomena”—the recordings of senseexperiences that are then transformed into a second representational level, the “data of the analysis.” The dataof analysis consist of records of experience, which may include field notes, survey responses, video recordings,software usage characteristics, and tally marks that count particular user behaviors. In quantitative analysis, therecording of experience emphasizes measurement and precision while qualitative analysis emphasizesinterpretation. The goal in both cases is to reduce large amounts of data to representations that arecomprehensible, allowing researchers to develop a deeper understanding of the original phenomena understudy. Both approaches require navigating and managing a series of tradeoffs between the precision andrichness of description and the validity of the inferences we can make from the data and subsequent analysis.

Quantitative Analysis

A powerful impetus for new approaches to quantitative data analysis and representation has come from theworld of business and commerce, which is focused on using large datasets and user-generated data to improvedecision making, managerial practices, and quality control processes. Examples include the recommendationengines of Amazon, Netflix, iTunes, Google, and Facebook, which provide users with targeted advertisem*ntsbased upon past behavior.

Accordingly, it is not surprising that the use of student data for educational improvement has also seenincreased prominence in education. Learners are increasingly leaving behind sophisticated and detailed tracesof their actions as they work in technologically mediated environments, a form of big data then available toeducational psychologists. Moreover, educational policies, such as Race to the Top and No Child LeftBehind, have added pressure to the need to collect and analyze large amounts of student data.

Technology has influenced how quantitative data analyses are conducted. At a basic level, statistical analysispackages that offer comprehensive tools for computing descriptive statistics, hypothesis testing, and drawinginferential conclusions have made statistical analyses increasingly assessable, user friendly, commonplace, andpowerful. These include standard statistical analysis packages (such as SPSS, SAS, and R) as well as somemore specialized packages, such as LISREL, which is used for confirmatory factor analysis and structuralequation modeling.

One of the most important areas where computational power has changed educational research is in thearea of data mining and visualization. Data mining is the process of examining large sets of data with multiplevariables to uncover trends and patterns. These data-mining techniques can be combined with the capabilitiesof digital technologies to represent and present data in rich, visual, and intuitively recognizable formats.Standard statistical packages, such as Excel and SPSS, have increasingly powerful tools for datarepresentation. Beyond this, there are other software programs, such as the interactive environment for dataanalysis and visualization MATLAB, the computational knowledge engine Wolfram Alpha, and the algebraicand symbolic mathematics package Mathematica, that specialize in the construction and display of complexand sophisticated graphical displays. As Knezek and Christensen (2014) wrote, “the distinction betweenanalysis, modeling, and display tools is beginning to blur as ‘math packages’ are being routinely employed to

produce elegant summaries and visual displays of findings from traditional research” (p. 219). Free web-basedsoftware applications, such as Google Fusion Tables and Many Eyes from IBM, allow researchers to uploadlarge datasets and display the data in multiple formats, such as graphs, maps, intensity maps, timelines, andstory lines.

Qualitative Data Analysis

Qualitative research has generally been defined as “any kind of research that produces findings not arrived atby means of statistical procedures or other means of quantification” (Strauss & Corbin, 1990, p. 17). Thus,qualitative researchers require technologies that assist in gathering and coding data to uncover phenomena andmake meaning through analyzing patterns in stories, common ideas, and emergent themes. Organization andinterpretation are important fundamentals of this work, and new technologies can assist with this (Anfara,Brown, & Mangione, 2002; Creswell, 1998). While the foundational elements of qualitative research—theguiding principles, determinants of reliability, validity, and so forth—remain in place, new technologies haveshifted aspects of methodology, and in some ways have changed the way we “see” or interpret qualitative data(Brown, 2002).

Digitalized qualitative processes allow researchers to store and access a variety of types of qualitative data,including text, audio, video, and graphics files. One of the most basic and critical newer uses of technologyinvolves the use of digital audio or video recording for field studies or interview sessions. At a surface level,such digital recordings are a way to preserve a clearer record of events and conversations, but, at another level,digital recordings afford new ways of thinking about how analysis develops out of the data and how datasupport it (Gibbs, Friese, & Mangabeira, 2002).

Educational researchers can now attend to small-scale and detail-oriented content in teaching and learningscenarios such as characteristics of speech, movements, or body language (see Chapter 28 in this volume).Examinations of such focused minutiae can be undertaken quickly, putting increased analytical power to workon observed data. While digital media has allowed researchers to home in on visual and audio data at a smallerscale, it has also opened up possibilities for much larger-scale studies because multiple researchers and analystscan connect and collaborate via qualitative coding software.

Computer-assisted qualitative data analysis software (CAQDAS), such as NVivo, Atlas.ti, orHyperRESEARCH, makes the core processes of organizing and coding data from observations, interviews,field research, or ethnography easier and more efficient (Lewins & Silver, 2009). By facilitating organizationand categorization of data, such programs facilitate the process of meaning making (Fielding & Lee, 2002).One of the common experiences of qualitative research has always been the challenge of careful and complexmanagement of large amounts of texts, codes, memos, field notes, and observations (Moustakas, 1994).CAQDAS options allow for greater efficiency and consistency in systematic data management.

Such software programs typically provide flexible code trees (or code books), which allow for a moresophisticated categorization and increased ease of complex data searches. A range of group codes, individualcodes, and subcodes can allow new and unique visualizations of the themes within a study for a specific look atthe building blocks of the study. This allows the coding process—a foundational process in qualitative work—to be not only more systematic in approaching data but also more dynamic and responsive to emergentinterpretations.

As noted, many CAQDAS programs today offer coding and organizational techniques for working withvideo and more traditional text and/or audio transcription. Digital video has unique properties that allowresearchers to capture, observe, and reobserve complex phenomena visually and then code or notate behaviors,themes, comments, or anything else of interest (Spiers, 2004). Such affordances can bring the traditionalthematic organization of qualitative work to video data and allow researchers to incorporate video vignettes—another powerful addition to the story-telling tradition of qualitative research (Creswell, 1998; Patton, 2002).

Develop Theories

The development of sound, predictable, data-driven theories is paramount to the conduct of research ineducational psychology. Theories provide us with concepts, terminologies, and classification schemes todescribe phenomena accurately, highlighting relevant issues and ignoring irrelevant ones. Theories also allowus to make inferences and predict the consequences of an intervention or change. Finally, theories have apragmatic function, informing how we can apply ideas to the real world by helping us design better learningcontexts and systems and by bridging the gap between description and design.

Digital technologies have changed the phenomena being studied, the kinds of data that can be collected(which ground the theory-making process), and the data analyses that are possible. Altogether, these changesin phenomena, data, and analyses have resulted in strong tests of theories not possible before. Theorygeneration, however, remains outside the scope of even the most intelligent computer programs. That said,technology and theory building have interacted in three significant ways. First, educational design-basedresearch (EDBR) methodologies have allowed researchers to study the effects of technological interventions ineducational settings iteratively. Second, technological contexts have provided testing grounds for the boundaryconditions of psychological theories and ideas, which have typically been based on studies conducted in face-to-face conditions. Third, the rise of “big data” has potential impacts for the role of theory in educationalpsychology.

Educational Design-Based Research

EDBR is a type of research methodology in which educational interventions are conceptualized and thenimplemented iteratively in natural settings to both test the validity of existing theories and generate newtheories for conceptualizing learning, instruction, design processes, and educational reform. A more detaileddescription of EDBR (and its variations) can be found in Chapter 2 in this volume. Our emphasis here is ontwo key aspects of EDBR. The first is an emphasis on the development of theory and the second is that manyEDBR studies have focused on innovation driven by technology.

One of the main goals of EDBR is the development of theory—to not only use theory to provide arationale for the intervention or to interpret findings but also help “develop a class of theories about both theprocess of learning and the means that are designed to support learning” (Cobb, Confrey, diSessa, Lehrer, &Schauble, 2003, p. 9). Also, though EDBR does not necessarily require the use of technology, it is frequentlydriven by the urge to integrate new psychological conceptions with technological possibilities. An example ofEDBR and the twin emphasis on theory generation and technology-related contexts can be seen in thedevelopment of the Technological Pedagogical Content Knowledge (TPACK) framework. This frameworkexplicates the knowledge teachers need to know in order to teach effectively with technology by extendingShulman’s (1986) idea of pedagogical content knowledge to include technological knowledge (Mishra &Koehler, 2006). This framework emerged from over seven years of multiple studies aimed at understandingthe development of teachers’ knowledge for effective technology integration while simultaneously helpingteachers (through courses, workshops, and other interventions) to develop their teaching with technology.Overall, this work led to a number of smaller studies (or EDBR “iterations”) and publications that stood ontheir own as well as a larger framework (Mishra & Koehler, 2006) that emerged through synthesizing across

the iterations.

Technological Contexts as Providing Boundary Conditions

Most long-standing psychological theories—such as theories of transfer, motivation, and mindfulness—weredeveloped based on research conducted in traditional face-to-face situations. New technologies provide newcontexts for studying human interaction and can serve as important tests of the boundary conditions underwhich such theories can succeed or fail. As Walther (2009) argues, “Boundaries are being foisted upon us bytechnological developments that may limit (or maybe revise) the scope of our extant theoretical frameworks.There are implicit boundaries that have always been there but which we have ignored, misapprehended, orfailed to investigate” (p. 750). At the heart of the issue is the question of fidelity of representation or thecorrespondence between the virtual and the physical world and our psychological responses to thesedifferences.

For example, consider how studies in human computer interaction show that people often treat computerrespondents just as they treat humans. The computers as social actors (CASA) paradigm argues that peoplemay unconsciously perceive interactive media as being “intentional social agents” and read personality, beliefs,and attitudes into them; more importantly, the CASA paradigm argues that people often act on theseperceptions. There is a strong body of empirical evidence to support this position: People are polite tomachines (Nass, Moon, & Carney, 1999), read gender and personalities into machines (Nass, Moon, &Green, 1997), are flattered by machines (Fogg & Nass, 1997), treat machines as team mates (Nass, Fogg, &Moon, 1996), and get angry and punish them (Ferdig & Mishra, 2004). Technology, however, also illustratesthe boundary conditions under which such attributions fail. For instance, Mishra (2006) found thatparticipants respond differently to praise and blame feedback from computer evaluators than they do fromhuman evaluators, suggesting the need for a more complicated theory of interaction.

Do We Need Theories in the Age of Big Data?

The rise of “big data” has caused some to argue that theories are becoming obsolete (e.g., Anderson, 2008)and will be replaced by large amounts of data, powerful analyses, and pattern recognition. For example,Google Translate works not by “understanding” any of the texts it translates but rather by tracking patternsacross a large corpus of texts in multiple languages and associating inputs with outputs. This has led somecomputer scientists and other researchers using big data to argue that there will be no need for theory ormodels of phenomena when we have enough data and patterns to process. Although this discourse has notentered the realm of education, it may soon do so. Whether or not this “data deluge” brings about the strongversion of “the end of theory,” educational psychologists cannot ignore the future impacts of big data ontheory building.

Read, Write, Publish, and Disseminate Ideas

The processes of reading, writing, publishing, and dissemination have seen radical changes brought about bythe advent of new digital and networking technologies. First, as in other academic disciplines, educationalpsychologists read and survey the field to conceptualize broader frames or perspectives in which to situateexisting and new research. As has been explained in the scholarship on academic work life (Fry & Talja,2007), and as touched on here, technology-driven changes in reading have an overall impact on the world ofacademia (Palmer & Cragin, 2008).

Reading, for example, has become increasingly on the screen (National Endowment for the Arts, 2007).This move towards more online and on-screen reading places “large demands on individuals’ literacy skills”(RAND Reading Study Group, 2002, p. 4) and requires new literacies, skills, strategies, dispositions, and socialpractices (Coiro, Knobel, Lankshear, & Leu, 2008). Surveying the field, too, has been transformed by newdigital and networking technologies, as new databases and citation indexes (Kousha & Thelwall, 2007; Meho& Yang, 2007) offer both qualitative and quantitative changes to how scholars access prior research andscholarship. Such tools can make it easier to gather resources from a wider range of sources and speed up therate at which new findings can be presented and shared. This can lead to too much cognitive load but also tothe creation of fresh connections to related information or to citations that would not otherwise have beenpossible.

Several important themes also underlie changes in the writing, dissemination, and publishing processesbrought about by the advent of new digital and networking technologies. The first is the move towards openpublishing, producing and distributing data in the “public domain” or with Creative Commons(creativecommons.org) licenses that allow public consumption and comment through open-access journals orself-publishing. More radical still are trends in how research is shared and disseminated that emphasizes socialscholarship, sharing published or in-progress work via social media outlets. Such scholarship changes researchdissemination routes, peer review, and potential audiences for work (Greenhow & Gleason, 2014; Greenhow,Robelia, & Hughes, 2009). A second influence of technology has been to change the tools available foracademic collaborative writing. Today’s technologies for writing can transform everything from project andbibliographic organization to the nature and process of collaborative writing. A third influence of technologyhas been the rise of manuscript platforms that can alter how we review and publish our work. Authors nowsubmit their manuscripts online and can track the progress of their manuscripts throughout the reviewprocess. Because authors, reviewers, and editors record and archive information within the same onlinesystem, editors can track patterns in online activities, and these patterns can then be used to improve thejournal’s overall review and publishing process.

http://creativecommons.org

Confront Ethical Issues

Technology integration into educational psychologists’ contemporary work practices raises a host of ethicalissues, such as data security and human subjects issues (Moore & Ellsworth, 2014).

Data Security

In an increasingly digital and networked data environment, issues of data security have become moreprominent. For example, cloud computing is frequently cited as an appealing data protection option becauseof many obvious affordances—ease of use, scalability, shareability, easy access to data, and built in backups.Researchers’ use of cloud storage solutions, however, also raises ethical concerns associated with entrustingthird-party vendors with confidential subject data (Newton, 2010).

Unknowing Participants

Technology has introduced new ways of automatically recording data about people, behaviors, and patterns ofinteraction that considerably impact potential participants in a research study. First, technology hasintroduced video recording in many facets of everyday life, including through the widespread use of securitycameras, mobile device cameras, and webcams (Koeppel, 2011). Second, people’s behaviors online are beingrecorded, both knowingly and unknowingly, through the use of session variables (e.g., “cookies”), monitoringof behavior on websites, and studies of interactions that occur online. All of this automatically recorded datahas ethical implications for would-be researchers. For example, researchers studying individuals in socialnetworking sites may inadvertently access data from individuals in their participants’ network that they do nothave permission to access. Many studies using data from these auto-recorded sources are determined “IRB-exempt” (see below) because the behavior is “publicly observable” and therefore does not require the consent ofany participants in the research. That said, the very idea of what is (or is not) “publicly observable” in anetworked, connected world is contentious and open to scholarly and legal debate.

Instructional Review Board (IRB) Issues Related to Technology

Internet-based research also raises complex issues concerning human subject protections. Topics such asconfidentiality, recruitment, and informed consent become complicated when research is conducted online.For example, authentication of identity in online worlds is an issue and may inadvertently lead to conductingresearch on minors or vulnerable populations. Another potential issue with internet-based research is thatrequesting consent should not disrupt normal group activity; however, the very act of entering onlinecommunities or chat rooms to request consent can be perceived as disruptive. Finally, even apparentlyanonymous data can be mined to identify geographical location, and as data analytics tools become moreintelligent, personal variables (such as age and gender) may be used to identify participants.

Conclusion

Clearly, the work we do as educational psychologists has changed and will continue to evolve due to theadvent of new technologies. An important caveat, given this rapid rate of change, is that much of what wehave written here will appear outdated by the time this volume is published, not to mention five years after itspublication. What this means is that we have to approach all that we have written with a critical eye and alsoattempt, even while focusing on the latest tools and techniques, to keep our focus on key ideas that will standthe test of time. It was this concern with relevance that led us to structure this chapter along the eightcategories of work. Although the manner in which we go about our business may change, these eightcategories will remain important parts of what educational psychologists do.

Looking beyond the eight categories of work, we emphasize three key perspectives on the current literatureon technology and its specific role in what we do as educational psychologists. First, among these perspectivesis what Salomon and Almog (1998) called the “reciprocal relationship” between technology and educationalpsychology:

Technologies and prevailing psychological conceptions of learning, thinking, and instruction have always served and inspired each other inreciprocal ways. On the one hand, technologies in education have served to facilitate and realize the kinds of pedagogies that emanatedfrom the changing zeitgeists and from prevailing psychological conceptions. On the other hand, and possibly only recently, technologieshave been imported into education, challenging it and requiring novel psychological explanation and pedagogical justifications. (p. 222)

In other words, Salomon and Almog argue that there is a transactional, dialogic relationship between thepsychology of learning and the affordances and constraints of technologies, where each helps define the other(what they have described as “an ongoing duet”). Thus the pedagogical meaning of a technology emerges notjust from the tool (and its properties) but rather from its deep integration into the matrix of subject matter,learners, and classroom environments. As Bruce (1997) says, “A technology is a system of people, texts,artifacts, activities, ideology, and cultural meanings” (p. 5).

The second perspective highlights the ways in which technologies and theories of mind have co-evolvedover time—either to instantiate our current understandings of learning or, just as importantly, to seek modelsfor thinking about thinking. Our understanding of the human brain and its activity has been consistentlyinfluenced by metaphors of the current technology. These include pneumatic/hydraulic metaphors, such asthose used by Galen and Descartes, wherein the brain was considered a site for the mixing, storing, andredirection of “spirits” throughout the body to determine behavior and action. With the rise of the IndustrialRevolution, new machine metaphors came to be used where the brain was now considered a complexmechanical apparatus with levers, gears, and pulleys. In the early part of the twentieth century, with the rise oftelephone networks, the brain came to be seen as a switchboard with inputs, outputs, and signals. Morerecently, the advent of the digital computer led to the brain being viewed as a device for informationprocessing. The advent of the internet has paralleled visions of the brain as being a networked computer.

The third perspective illuminates how technologies provide important “boundary conditions” to testeducational and psychological theories. This involves providing new methodologies and new sources of data totest our theories as well as providing new tools to develop theory and to share our work with others.Technology can also provide novel pedagogical opportunities that offer a new “zone of possibility” (Kereluik,Mishra, Fahnoe, & Terry, 2013, p. 128) beyond our current psychological understandings, explanations, and

justifications. Because technologies develop so rapidly, often outpacing developments of our psychologicalconceptions, technology can pose important conceptual and theoretical challenges for educationalpsychologists. Suddenly, old and partly dormant issues, such as transfer, intentionality, and mindfulness, canbe brought again to the forefront as we develop novel conceptions and understandings of human behavior,learning, and instruction (Salomon & Almog, 1998).

These are truly exciting times for education—in large part distinguished by rapid changes in technologythat are changing almost all aspects of our professional lives as educators and educational scholars. We believethat this ongoing duet will continue into the future.

Note

The authors would like to thank Spencer Greenhalgh, Dr. Danah Henriksen, Dr. Michelle Hagerman, Rohit Mehta, and Joshua Rosenbergfor their assistance in writing this chapter.

References

Aggarwal, C. C. (Ed.). (2011). Social network data analytics. New York, NY: Springer.Alves, F., Pagano, A., & da Silva, I. (2010). A new window on translators’ cognitive activity: Methodological issues in the combined use of eye

tracking, key logging and retrospective protocols. Copenhagen Studies in Language, 38, 267–291.Anderson, A. F., & Bavelier, D. (2011). Action game play as a tool to enhance perception, attention and cognition. In S. Tobias & D. Bavelier

(Eds.), Computer games and instruction (pp. 307–330). Charlotte, NC: Information Age Publishing.Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine, 16(7). Retrieved from

http://archive.wired.com/science/discoveries/magazine/16-07/pb_theory (accessed February 18, 2015).Anfara, V. A., Jr., Brown, K. M., & Mangione, T. L. (2002). Qualitative analysis on stage: Making the research process more public.

Educational Researcher, 31(7), 28–38.Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7,

543–554.Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecology perspective. Human Development, 49,

193–224.Behrens, J. T., & Smith, M. L. (1996). Data and data analysis. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp.

949–989). New York: Macmillan.Bharanipriya, V., & Prasad, V. K. (2011). Web content mining tools: A comparative study. International Journal of Information Technology and

Knowledge Management, 4(1), 211–215.boyd, d., & Crawford, C. (2011). Six provocations for big data. In B. Dutton, B. Loader, V. Nash, & B. Wellman, A decade in Internet time:

Symposium on the dynamics of the Internet and society, September 2011. Symposium conducted at the meeting of the Oxford InternetInstitute, Oxford, England, UK.

Bransford, J. D., Brown, A. L., & co*cking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school. Washington, DC:National Academy Press.

Brown, D. (2002). Going digital and staying qualitative: Some alternative strategies for digitizing the qualitative research process. Forum:Qualitative Social Research, 3(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/index (accessed 18 February, 2015).

Bruce, B. C. (1997). Technology in social practice: Returning to Dewey’s conception of learning. Paper presented at the NSF/ARL/BI (NAB)workshop, Alexandria, VA.

Clarke, J., Dede, C., Ketelhut, D. J., & Nelson, B. (2006). A design-based research strategy to promote scalability for educational innovations.Educational Technology, 46(3), 27–36.

Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1),9–13.

Coiro, J., Knobel, M., Lankshear, C., & Leu, D. (2008). Central issues in new literacies and new literacies research. In J. Coiro, M. Knobel, C.Lankshear, & D. Leu (Eds.), Handbook of research on new literacies (pp. 1–21). New York, NY: Lawrence Erlbaum Associates.

Cormode, G., & Krishnamurthy, B. (2008). Key differences between Web 1.0 and Web 2.0. First Monday, 13(6).Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage.De Lucia, A., Francese, R., Passero, I., & Tortora, G. (2009). Development and evaluation of a system enhancing Second Life to support

synchronous role-based collaborative learning. Software: Practice and Experience, 39, 1025–1054.Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R., . . . Sherin, B. L. (2010). Conducting video research in the

learning sciences: Guidance on selection, analysis, technology, and ethics. Journal of the Learning Sciences, 19(1), 3–53.Duchowski, A. (2007). Eye tracking methodology: Theory and practice (Vol. 373). London, UK: Springer.Ellison, N. B., & boyd, d. m. (2013). Sociality through social network sites. In W. H. Dutton (Ed.), The Oxford handbook of Internet studies (pp.

151–172). Oxford, England: Oxford University Press.Ferdig, R. E., & Mishra, P. (2004). Emotional responses to computers: Experiences in unfairness, anger, and spite. Journal of Educational

Multimedia and Hypermedia, 13, 143–161.Fielding, N. G., & Lee, R. M. (2002). New patterns in the adoption and use of qualitative software. Field Methods, 14, 190–196.Fogg, B. J., & Nass, C. (1997). Silicon sycophants: The effects of computers that flatter. International Journal of Human-Computer Studies, 46,

551–561.Fry, J, & Talja, S. (2007). The intellectual and social organization of academic fields and the shaping of digital resources. Journal of Information

Science, 33, 115–133.Galesic, M., Tourangeau, R., Couper, M. P., & Conrad, F. G. (2008). Eye-tracking data: New insights on response order effects and other

cognitive shortcuts in survey responding. Public Opinion Quarterly, 72, 892–913.

http://archive.wired.com/science/discoveries/magazine/16-07/pb_theory

http://www.qualitative-research.net/index.php/fqs/index

Gentile, D. A. (2011). The multiple dimensions of video game effects. Child Development Perspectives, 5, 75–81.Gibbs, G. R., Friese, S., & Mangabeira, W. C. (2002). The use of new technology in qualitative research. Introduction to Issue 3(2) of FQS.

Forum: Qualitative Social Research, 3(2).Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational

Researcher, 40, 223–234.Greenhow, C., & Gleason, B. (2014). Social scholarship: Reconsidering scholarly practices in the age of social media. British Journal of

Educational Technology, 45, 309–402.Greenhow, C., & Li, J. (2013). Like, comment, share: Collaboration and civic engagement within social network sites. In C. Mouza & N.

Lavigne (Eds.), Emerging technologies for the classroom (pp. 127–141). New York, NY: Springer.Greenhow, C., Robelia, E., & Hughes, J. (2009). Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38,

246–259.Judele, R., Tsovaltzi, D., Puhl, T., & Weinberger, A. (2014). Collaborative learning in Facebook: Adverse effects of individual preparation. In

Proceedings of the Forty-Seventh Annual Hawaii International Conference on System Sciences (pp. 1616–1624). Washington, DC: IEEE.Kalos, M. H., & Whitlock, P. A. (2009). Monte Carlo methods. Weinheim, Germany: Wiley-VCH.Karle, J. W., Watter, S., & Shedden, J. M. (2010). Task switching in video game players: Benefits of selective attention but not resistance to

proactive interference. Acta Psychologica, 134, 70–78.Kereluik, K., Mishra, P., Fahnoe, C., & Terry, L. (2013). What knowledge is of most worth: Teacher knowledge for 21st century learning.

Journal of Digital Learning in Teacher Education, 29(4), 127–140.Knezek, G. A., & Christensen, R. (2014). Tools for analyzing quantitative data. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop

(Eds.), Handbook of research on educational communications and technology (4th ed., pp. 203–220). New York, NY: Springer.Koeppel, D. (2011). More people are using smartphones to secretly record office conversations (web blog post). Retrieved from

http://www.businessinsider.com/smartphones-spying-devices-2011-7 (accessed February 18, 2015).Kousha, K., & Thelwall, M. (2007). Google Scholar citations and Google web/URL citations: A multi-discipline exploratory analysis. Journal of

the American Society for Information Science and Technology, 58, 1055–1065.Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 2–3, 259–284.Lewins, A., & Silver, C. (2009). Choosing a CAQDAS package: A working paper (6th ed.). Surrey: University of Surrey.Linacre, J. M. (2000). Computer-adaptive testing: A methodology whose time has come. In S. Chae, U. Kang, E. Jeon, & J. M. Linacre (Eds.),

Development of computerized middle school achievement test (in Korean). Seoul, South Korea: Komesa Press.Luecht, R. M. (2005). Computer-adaptive testing. In B. S. Everitt, & D. Howell (Eds.), Encyclopedia of statistics in behavioral science. Hoboken,

NJ: John Wiley.Ma, J. & Nickerson, J. V. (2006). Hands-on, simulated, and remote laboratories: A comparative literature review. ACM Computing Surveys,

38(3).Manca, S., & Ranieri, M. (2013). Is it a tool suitable for learning? A critical review of the literature on Facebook as a technology-enhanced

learning environment. Journal of Computer Assisted Learning, 29, 487–504.Mayfield, E., & Rosé, C. P. (2013). LightSIDE: Open source machine learning for text. In M. D. Shermis & J. Burstein (Eds.), Handbook of

automated essay evaluation: Current applications and new directions (pp. 124–135). New York, NY: Routledge.Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and

review of online learning studies. Washington, DC: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development.Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and

Google Scholar. Journal of the American Society for Information Science and Technology, 58, 2105–2125.Mishra, P. (2006). Affective feedback from computers and its effect on perceived ability and affect: A test of the computers as social actor

hypothesis. Journal of Educational Multimedia and Hypermedia, 15, 107–131.Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College

Record, 108, 1017–1054.Moore, S. L., & Ellsworth, J. B. (2014). Ethics of educational technology. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.),

Handbook of research on educational communications and technology (4th ed., pp. 113–128). New York, NY: Springer.Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage.Mumby, P. J. (2002). Statistical power of non-parametric tests: A quick guide for designing sampling strategies. Marine Pollution Bulletin, 44,

85–87.Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation

Modeling, 9, 599–620.Nass, C., Fogg, B. J., & Moon, Y. (1996). Can computers be teammates? Journal of International Human-Computer Studies, 45, 669–678.

http://www.businessinsider.com/smartphones-spying-devices-2011-7

Nass, C., Moon, Y., & Carney, P. (1999). Are people polite to computers? Responses to computer-based interviewing systems. Journal ofApplied Social Psychology, 29, 1093–1109.

Nass, C., Moon, Y., & Green, N. (1997). Are machines gender neutral? Gender-stereotypic responses to computers with voices. Journal ofApplied Social Psychology, 27, 864–876.

National Endowment for the Arts (NEA). (2007). To read or not to read: A question of national consequence (Research report #47). Washington,DC: NEA.

Newton, J. (2010). The ethics and security of cloud computing. Austin, TX: International Legal Technology Association.Palmer, C. L. & Cragin, M. H. (2008). Scholarship and disciplinary practices. In B. Cronin (Ed.), Annual review of information science and

technology (Vol. 42, pp. 165–212). Medford, NJ: Information Today.Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications.Picciano, A. G., & Seaman, J. (2009). K-12 online learning: A 2008 follow-up of the survey of U.S. school district administrators. New York, NY:

The Sloan Consortium.RAND Reading Study Group. (2002). Reading for understanding: Toward an R&D program in reading comprehension. Santa Monica, CA:

RAND.Ranieri, M., Manca, S., & Fini, A. (2012). Why (and how) do teachers engage in social networks? An exploratory study of professional use of

Facebook and its implications for lifelong learning. British Journal of Educational Technology, 43, 754–769.Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge, MA: The MIT Press.Salomon, G., & Almog, T. (1998). Educational psychology and technology: A matter of reciprocal relations. Teachers College Record, 100(1),

222–241.Scalise, K., & Gifford, B. (2006). Computer-based assessment in e-learning: A framework for constructing “intermediate constraint” questions

and tasks for technology platforms. Journal of Technology, Learning, and Assessment, 4(6), 3–30.Shermis, M. D., & Burstein, J. C. (Eds.) (2003). Automated essay scoring: A cross-disciplinary perspective. Mahwah, NJ: Lawrence Erlbaum

Associates.Shermis, M. D., & Burstein, J. (Eds.) (2013). Handbook of automated essay evaluation: Current applications and new directions. New York, NY:

Routledge.Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 30–32.Sims, V. K., & Mayer, R. E. (2002). Domain specificity of spatial exercise: The case of video game players. Applied Cognitive Psychology, 16, 97–

115.Spiers, J. A. (2004). Tech tips: Using video management/analysis technology in qualitative research. International Journal of Qualitative Methods,

3(1), 57–61.Squire, K. (2006). From content to context: Videogames as designed experience. Educational Researcher, 35(8), 19–29.Stieff, M. & Wilensky, U. (2003). Connected chemistry: Incorporating interactive simulations into the chemistry classroom. Journal of Science

Education and Technology, 12, 285–302.Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.Tettegah, S., & Calongne, C. (2009). Identity, learning and support in virtual environments. Rotterdam, Netherlands: Sense Publishers.Thissen, D., & Mislevy, R. J. (2000). Testing algorithms. In H. Wainer, N. J. Dorans, D. Eignor, R. Flaugher, B. F. Green, R. J. Mislevy, L.

Steinberg, & D. Thissen (Eds.), Computerized adaptive testing: A primer (2nd ed., pp. 101–134). Mahwah, NJ: Lawrence Erlbaum Associates.Tobias, S., Fletcher, J. D., & Wind, A. P. (2014). Game-based learning. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.),

Handbook of research on educational communications and technology (4th ed., pp. 485–503). New York, NY: Springer.Walther, J. B. (2009). Theories, boundaries, and all of the above. Journal of Computer-Mediated Communication, 14, 748–752.Wilensky, U. (2003). Statistical mechanics for secondary school: The GasLab modeling toolkit. International Journal of Computers for

Mathematical Learning, 8, 1–41.Wilensky, U., & Reisman, K. (2006). Thinking like a wolf, a sheep or a firefly: Learning biology through constructing and testing

computational theories—an embodied modeling approach. Cognition & Instruction, 24, 171–209.Wilson, C., Sala, A., Puttaswamy, K. P. N., & Zhao, B. Y. (2012). Beyond social graphs: User interactions in online social networks and their

implications. ACM Transactions on the Web, 6(4), Article 17.

4The Prospects and Limitations of Latent Variable Models

in Educational PsychologyBENJAMIN NAGENGAST

ULRICH TRAUTWEIN1

University of Tübingen, Germany Latent variable models, in particular, structural equation modeling (SEM), are a staple of research ineducational psychology. SEM combines measurement models, which provide links between observedresponses to items and unobserved latent variables, and structural models, which represent the multivariaterelations between latent variables and manifest variables (Jöreskog, 1970; see Matsueda, 2012, for an overviewof the history of SEM). The main application of latent variable models in educational psychology lies in thedevelopment and validation of measures of core constructs. Second, SEM is used to study and test relationsbetween latent variables in structural models. Often, these structural relations are not only used descriptivelybut are also given a causal interpretation (e.g., in mediation analyses). There have been many methodologicaladvances with respect to measurement models, structural models, and causal inference that have the potentialto impact the ways in which researchers in educational psychology use SEM to analyze their data. In thischapter, we critically review the scope and limitations of applications of SEM in educational psychology andhighlight the challenges that researchers will face in the upcoming decade.

Recent numbers show that such a review is timely. In the last decade, researchers in educational psychologyhave taken advantage of the increased availability of large datasets, which have allowed researchers to applycomplex modeling techniques such as path analysis, hierarchical linear modeling, or SEM to questions ineducational psychology. A recent review (Reinhart, Haring, Levin, Patall, & Robinson, 2013) showed that, in2000, only 12 out of 134 (or 9.0%) of all articles published in five leading educational journals had used suchmodeling techniques. These numbers increased to 57 out of 141 (or 40.4%) in 2010. This increase was at leastpartly driven by significant advances in latent variable modeling. The emergence of generalized latent variablemodeling (Muthén, 2002; Skrondal & Rabe-Hesketh, 2004) as a unifying framework for many previouslyunrelated methods has significantly increased the scope of problems that can be addressed. An additionalreason for the growing popularity of latent variable modeling techniques is their greatly increased availability.During the first decade of the new century, software implementations of SEM became more accessible andeasier to use. This development has considerably lowered the threshold for the adoption of SEM in appliedresearch.

The increased use of latent variable techniques, however, is also a challenge for the field of educationalpsychology. In order to be able to take full advantage of emerging methodological advances (Marsh & Hau,2007), researchers must stay current with the rapidly evolving methodology, its opportunities, and its

limitations. At the same time, reviews suggest that the methodological standards in modern educationalpsychology often exceed what is being taught in graduate and postgraduate training programs (e.g., Aiken,West, & Millsap, 2008) and that the “collective quantitative proficiency” (Henson, Hull, & Williams, 2010,p. 233) in the field is rather low. Indeed, this leads to many applications of latent variable models that do notfully exploit their strengths, but also to unjustified conclusions that are based on the results of these models(Foster, 2010a; Reinhart et al., 2013). In particular, the uncritical causal interpretation of path coefficientsderived from latent variable models has repeatedly been pointed out as problematic (Foster, 2010a; Freedman,1987; Reinhart et al., 2013). Researchers in educational psychology appear to be slow at adopting the notionthat research design matters more for the validity of causal inferences than a specific statistical method (Cook,Steiner, & Pohl, 2009; Rubin, 2008). All too often, latent variable models are used in research designs that aretoo weak to support the intended conclusions.

In this chapter, we will focus on developments in latent variable modeling and methodology over the courseof the last 10 years. We will begin by reviewing developments in measurement models, focusing ongeneralized latent variable modeling, measurement invariance, and exploratory measurement models. Next, wewill discuss advances in structural models (multilevel SEMs and non-linear models). Finally, we will focus oncausal inference in quasi-experimental designs and mediation modeling. Our discussion will be illustrated withexamples from educational psychology that highlight methodological challenges and demonstrate thecontributions and limitations of latent variable models in addressing such challenges. We have attempted tolimit the technical details of our presentation and to point interested readers toward accessible and relevantsources whenever possible.

Measurement Models

Most constructs in educational psychology cannot be directly observed but rather have to be inferred on thebasis of indicators that are often imperfect or unreliable (Lord & Novick, 1968; Steyer & Eid, 2001). Forexample, students’ math competencies have to be inferred on the basis of math problems; academic self-concept, the self-perception of academic abilities (Marsh & Shavelson, 1985), has to be inferred from answersto questionnaire items; and instructional quality has to be inferred from student ratings of their teachers’behavior (e.g., Kunter et al., 2013; Wagner, Göllner, Helmke, Trautwein, & Lüdtke, 2013). Researchers ineducational psychology (often implicitly) invoke the latent variable theory of measurement (Markus &Borsboom, 2013; see Michell, 1997, for a discussion of other definitions of measurement), which “rests on thespecification of a functional relation between the latent variable and its indicators” (p. 63) in order to locate aperson within the latent space. The observed items are not deterministically influenced by the construct butare affected by a wide range of unsystematic influences that are subsumed under measurement error (Lord &Novick, 1968). Thus, educational psychologists often are faced with the challenge of separating measurementerror and true score variance (Lord & Novick, 1968).

Confirmatory factor analysis (CFA) as implemented in SEM measurement models has proven to be aversatile tool for this purpose. It assumes that the covariances between observed item responses can beattributed to an underlying set of latent variables (e.g., Jöreskog, 1969). Parameters such as the factor loadings,variances, and covariances of latent variables and residual variances can then be obtained. Figure 4.1aillustrates a confirmatory factor model with three correlated factors. Influential applications of CFA ineducational psychology include, for example, the assessment of goal orientations (Elliott & McGregor, 2001;Elliott, Murayama, & Pekrun, 2011; Midgley et al., 1998), integrative models of educational motivation (e.g.,Martin, 2007), and academic self-concept (e.g., Marsh & Hocevar, 1985).

In the following section, we will review three methodological advances in SEM measurement models thathave significantly affected (or have the potential to significantly affect) the way educational psychologistsanalyze their data: (a) the emergence of generalized latent variable modeling; (b) the stronger focus onestablishing measurement invariance; and (c) exploratory structural equation modeling (ESEM).

Figure 4.1 Path diagrams of a confirmatory factor model (a) and an exploratory factor model (b) with nine indicators variables (Y1 to Y9) andthree correlated latent factors (η1 to η3)

Generalized Latent Variable Modeling

Historically, CFA required the indicator variables (e.g., responses to questionnaire items) to be continuousvariables with normally distributed residuals (e.g., Bollen, 1989; Jöreskog, 1970). Obviously, this assumptionis often violated in measures in educational psychology. Achievement tests typically employ items scored asright or wrong, thus yielding dichotomous indicators, and questionnaire studies typically employ Likert scales,which yield ordered-categorical indicators. For a long time, researchers had two options for dealing with theseviolations of distributional assumptions. They could ignore them and treat indicator variables as continuous,leading to potential biases in parameter estimates and fit statistics, particularly for data with a limited numberof categories and non-symmetric distributions (e.g., DiStefano, 2002; Muthén & Kaplan, 1985, 1992; Olsson,1979); alternatively, they could construct item parcels (e.g., parallel test versions) that satisfied distributionalassumptions but potentially masked important misfit (Bandalos & Finney, 2001; Little, Rhemtulla, Gibson,& Schoemann, 2013; Marsh, Lüdtke, Nagengast, Morin, & Davier, 2013). Software implementations ofmodels for categorical data were often restricted to a single dimension, thus limiting their usefulness forvalidating multiple constructs at the same time (e.g., Reckase, 1997; Wirth & Edwards, 2007).

The development of so-called generalized latent variable models allows researchers to take the nature ofindicator variables into account systematically. Generalized latent variable models (Muthén, 2002; Rabe-Hesketh, Skrondal, & Pickles, 2004; Skrondal & Rabe-Hesketh, 2004) can incorporate a wider range ofindicator variables (e.g., dichotomous, ordered-categorical, unordered-categorical, censored, and countindicators) in measurement models. Thus, these models present a major advance in latent variablemethodology. These extensions allow a variety of other latent variable traditions to be subsumed under theumbrella of SEM, including item response theory models that are prominent in educational assessment andmeasurement (e.g., van der Linden & Hambleton, 1997). Thus, relations between latent variables that have

100

multiple types of indicators can be modeled. Conceptually, generalized latent variable models include linkfunctions in the measurement model. They link a normally distributed latent response variable to the actualresponses on the indicators.

Generalized latent variable models are now available in several software programs. One often-usedestimator for models with ordered-categorical and dichotomous indicator variables, the robust weighted least-squares estimator (denoted WLSMV in Mplus) (Muthén, du Toit, & Spisic, 1997), has demonstratedflexibility and efficiency even with moderate sample sizes (e.g., Flora & Curran, 2004). Consequently,applications in educational psychology are beginning to appear (e.g., Bulotsky-Shearer, Dominguez, & Bell,2012; Guay, Morin, Litalien, Valois, & Vallerand, 2015; Waasdorp, Bradshaw, & Duong, 2011).

Measurement Invariance

After establishing the factor structure of an instrument, it is important to establish its invariance across groupsor time points (in longitudinal studies). Differences in measurement models between groups suggest that theinstrument functions differently across groups: people with the same value on the latent variable but who arefrom different groups will show different response behaviors (e.g., see Millsap, 2011, for an overview). In thiscase, comparisons of observed scale scores and latent variable means will be significantly complicated.Consider the example of gender differences in motivational constructs that are discussed in educationalpsychology (e.g., Jacobs, Lanza, Osgood, Eccles, & Wigfield, 2002; Watt et al., 2012). In order to interpretsuch differences substantively, it is necessary that equivalent levels in the latent motivational constructs aremanifested similarly in the item responses for girls and boys. If there were items that boys might answer morefavorably due to reasons unrelated to their motivational beliefs (e.g., because the item taps typical male self-stereotypes), mean differences in the scale scores could be inflated. In a similar way, items could be related tothe latent variable in different ways in each group, further complicating comparisons. It is thus critical toestablish measurement invariance as a precondition for group comparisons.

Measurement invariance is tested by implementing increasingly more restrictive multiple-group SEMs(e.g., gender, experimental groups, countries; see e.g., Millsap, 2011). Taxonomies of measurement invarianceare given by Widaman and Reise (1997), Meredith (1993), and Marsh et al. (2009). Similar taxonomies couldalso be applied when repeatedly measuring the same respondents at multiple time points (e.g., Vandenberg &Lance, 2000).

The configural invariance model (Widaman & Reise, 1997) assumes that the factor structure is equal acrossgroups. It is implemented as a multigroup SEM with the same measurement structure in each group.However, the values of the factor loadings, intercepts, residual variances, factor variances, and covariances arenot restricted to equality across groups. If this model does not fit the data, there are fundamental differencesin the factor structure across groups that make comparisons of the latent variables difficult or impossible. Infact, finding such discrepancies (e.g., between experimental groups) might be an interesting research findingin itself as it points to substantive differences in the ways in which the items are related to underlyingconstructs.

If the configural invariance model as described above fits the data well, the next step is to test for weakmeasurement invariance (Meredith, 1993). In this model, all factor loadings are constrained to equality acrossgroups. If the model holds, a change in the latent variable affects the expected answers to the indicators in the

101

Box 4.1

same way in all groups. Weak measurement invariance is required for meaningful comparisons of variance–covariance matrices of factors across the groups. A comparison of factor means (e.g., for calculating meandifferences in experimental designs) is appropriate only if strong measurement invariance (Meredith, 1993)holds. In this model, item intercepts are constrained to be equal across groups in addition to the factorloadings. If this restriction holds, there will be no systematic differences in average item responses between thegroups that are not due to differences in the latent variables. Finally, it is often of interest to test for strictmeasurement invariance (Meredith, 1993). In this model, the residual variances of items are fixed to equalityacross groups in addition to the factor loadings and item intercepts. Comparisons of raw scale scores arepossible only if strict measurement invariance holds.

In general, researchers in educational psychology have started to acknowledge the importance ofmeasurement invariance facilitated by its increased availability in software packages. Examples includeresearch on academic motivation (e.g., Grouzet, Otis, & Pelletier, 2006; Guay et al., 2015), studentengagement (e.g., Wang, Willett, & Eccles, 2011), or the cross-cultural invariance of core constructs ineducational psychology (e.g., Nagengast & Marsh, 2013) and academic self-concept (e.g., Brunner, Keller,Hornung, Reichert, & Martin, 2009). However, in applications, the assumptions of measurement invarianceare not always explicitly spelled out and often go untested, in cross-cultural studies in particular (e.g., Chen,2008). In Box 4.1, we present an example of tests of measurement invariance of school engagement acrossgender and race/ethnicity (Wang et al., 2011).

Testing the measurement invariance of school engagement across gender and race/ethnicity(Wang et al., 2011)

102

Research question

School engagement has been discussed as an important precondition for academic success. Theoreticalmodels (e.g., Fredricks, Blumenfeld, & Paris, 2004) posit three components of engagement (behavior,emotion, and cognition). Can these dimensions (and further subfacets) be empirically identified? Is thestructure of engagement invariant with respect to gender and ethnic groups, thus allowing forcomparisons between these groups?

103

Methodological challenge

Comparisons across different gender and ethnic groups require that strong measurement invarianceholds (i.e., that factor loadings and item intercepts are not influenced by the respective groups).

104

Method

Multigroup SEM was applied to test for measurement invariance separately for gender andrace/ethnicity using data from 1,103 ninth-grade students (52% girls; 56% African American, 32%European American, 12% biracial or from other ethnic minorities) from 23 public schools in Maryland.Students were assessed on six scales intended to measure school engagement. Each component ofengagement was represented by two scales, leading to a hierarchical factor model with three higher-orderfactors (behavioral, emotional, and cognitive engagement), with two first-order factors each.

105

Results

The CFA model with six first-order and three second-order factors fit the data well and generalizedacross the gender and race groups (configural invariance). In addition, there was evidence of strongmeasurement invariance (invariance of the factor loadings of the first- and second-order factor loadingsand intercepts of indicators and first-order factors) permitting mean comparisons on the second-orderfactors.

106

Interpretation

Boys had lower scores in behavioral and emotional engagement. European Americans scored lower onemotional engagement but reported higher behavioral engagement compared with African Americans.There were no group differences on cognitive engagement. These comparisons were possible onlybecause the strong invariance model held for the second-order factors.

Exploratory Structural Equation Modeling

CFA often relies on the independent cluster model (Thurstone, 1947): each indicator is allowed to load onone and only one latent variable. Cross-loadings (i.e., when an indicator loads substantially on more than onelatent variable) are typically not included. In addition, a loading structure needs to be specified a priori inCFA. Researchers have to indicate which latent variable underlies which indicators. Thus, the measurementmodel formally encodes prior assumptions about the factor structure. These assumptions can turn out to beincompatible with the observed data, requiring changes to the measurement model. Also, at the beginning ofthe scale construction process, the underlying dimensionality of the manifest variables is unknown. In thesecases, CFA falls short.

Exploratory measurement models (Browne, 2001) have recently been re-examined as alternatives to CFA(Asparouhov & Muthén, 2009). An exploratory measurement model posits a factor model that restricts onlythe dimensionality of the indicators but freely estimates all cross-loadings (Figure 4.1b). For a long time,exploratory measurement models could not be easily included in structural models and lacked many of theadvantages of SEM, such as tests of model fit, standard errors for loadings, or the ability to include correlatedresiduals. Recent developments in ESEM (Asparouhov & Muthén, 2009; Dolan, Oort, Stoel, & Wicherts,2009) have integrated exploratory measurement models into a broader SEM framework. Introductions toESEM are given by Asparouhov and Muthén (2009) and Morin, Marsh, and Nagengast (2013).

Measurement models in ESEM (Asparouhov & Muthén, 2009) require only the specification of thehypothesized number of factors. An initial solution is obtained by fitting an unconstrained factor model(Jöreskog, 1969). As in exploratory factor analysis, this solution is then rotated to one of many possiblealternative equivalent loading patterns. Most popular rotation algorithms (see Browne, 2001) operationalizeThurstone’s (1947) simple structure criterion and try to obtain a solution where most items have high loadingson a single factor. As an alternative, target rotation (Browne, 2001) can be used to rotate the factors as closelyas possible to a pre-specified arbitrary loading pattern (Asparouhov & Muthén, 2009; Dolan et al., 2009). Asin exploratory factor analysis, it is possible to use both orthogonal (i.e., uncorrelated factors) and oblique (i.e.,correlated factors) rotation procedures.

The most important strengths of ESEM are two extensions of exploratory factor analysis: ESEM factorscan be used in the structural models as dependent or predictor variables and can be related to manifestcovariates and latent factors established in other exploratory or confirmatory measurement models. Allparameters in the structural model depend on the rotation algorithm and change when a different rotation ischosen (Asparouhov & Muthén, 2009). Second, ESEM can be used to jointly rotate different sets of

107

Box 4.2

exploratory factors and to test the invariance of exploratory measurement models across multiple groups andmultiple time points (Asparouhov & Muthén, 2009; Dolan et al., 2009) by including constraints in therotation process. In Box 4.2, we present an example of tests of measurement invariance across two groups ofraters (students and teachers) using ESEM. The most important shortcoming of ESEM is rotationalindeterminacy: When using exploratory measurement models, there is an indefinite number of rotated factorsolutions that fit the data equally well (e.g., Browne, 2001). In ESEM, this not only affects parameters in themeasurement model such as factor loadings and factor correlations, but also influences parameters in thestructural model (see Asparouhov & Muthén, 2009). As a result, conclusions about structural relations dependon the chosen rotation algorithm, and more complex ESEM models are often difficult to fit.

Morin et al. (2013) distinguished two uses of ESEM. First, the ESEM framework can be used in anexploratory way to establish the dimensionality of a set of manifest variables without firm knowledge of thefactor structure. In this context, a series of models with an increasing number of factors is estimated andcompared according to model fit indices and the likelihood ratio test (e.g., Jöreskog, 1969) or traditionalapproaches for establishing the number of factors (e.g., Cattell, 1966; Horn, 1965; Kaiser, 1960). Afterestablishing the factor structure, its invariance across groups and its stability across time points can be tested.Second, the ESEM framework can be used in a confirmatory way to analyze existing instruments that have aknown (simple) factor structure but that do not fit a confirmatory measurement model very well (see e.g.,Marsh et al., 2009, 2010). Here, the use of ESEM is justified if the exploratory measurement model fitsconsiderably better than the more restrictive confirmatory measurement model.

Agreement between students’ and teachers’ perspectives of self-regulated learning and mathcompetence—an exploratory use of ESEM (Friedrich, Jonkmann, Nagengast, Schmitz, &Trautwein, 2013)

108

Research question

Being able to correctly diagnose students’ abilities and motivational states is an important skill forteachers who want to adapt their instruction to their students’ needs. Can teachers correctly infer theirstudents’ abilities for self-regulated learning, or are their perceptions colored by the perceivedcompetency levels of their students? How do teachers’ and students’ perceptions of students’ self-regulated learning ability and competency compare?

109

Methodological challenge

The dimensionality of teachers’ ratings of students’ self-regulated ability and competency is unclear.Meaningful comparisons of teacher and student ratings require measurement invariance across the twosets.

110

Method

ESEM (Asparouhov & Muthén, 2009) was applied with separate measurement models for teacher andstudent ratings controlling for negatively worded items with correlated residuals using data from 73 mathteachers and their 1,289 fifth-grade students from German lower-track schools. Students and teachersrated competencies on sets of similar items.

111

Results

A three-factor solution differentiating pre-actional self-regulated learning, actional self-regulated learning,and perceived math competence supported the prediction that teachers can in principle differentiatestudents’ abilities to self-regulate from perceptions of competence. The joint model with students’responses (that showed a similar loading structure) revealed relatively low agreement between studentsand teachers, in particular for self-regulated behavior.

112

Interpretation

ESEM allowed comparisons to be made with regard to the dimensionality and invariance of theexploratory factor structure of the self-regulation and perceived competence scales across teachers andstudents. A CFA would not have been warranted due to uncertainties about the dimensionality of theitems. The use of ESEM highlighted the exploratory nature of the data analysis but nevertheless allowedfor strong tests of the congruence of the factor structure of student and teacher ratings.

The options for exploratory and confirmatory uses of ESEM in educational psychology are manifold. Many

scales in educational psychology do not follow the independent cluster model (e.g., the Academic MotivationScale, Guay et al., 2015; the NEO-Five-Factor Inventory, Marsh et al., 2010; the Motivation andEngagement Wheel, Marsh, Liem, Martin, Morin, & Nagengast, 2011a) and cannot be fit adequately usingconventional CFA. Cross-loadings are a common occurrence, and ignoring (or removing) them can lead tosubstantial changes in the interpretation of constructs (e.g., Guay et al., 2015; Marsh et al., 2009, 2010,2011a, 2011b). By placing exploratory measurement models in a broader SEM framework that includes testsof measurement invariance, correlated residuals, and tests of model fit, ESEM provides a principled way ofdealing with these issues.

Conclusion

SEMs are well entrenched in scale construction and validation in educational psychology. The advances wediscussed in measurement models have made their way into educational psychology to varying degrees.Whereas tests for measurement invariance that are based on confirmatory—and to a lesser extent, exploratory—measurement models are being taken more seriously, generalized latent variable models for dichotomousand ordered-categorical indicators have yet to find widespread use in educational psychology. An increasedexposition of these methods will likely add to their adoption. ESEM promises to provide a useful alternativewhen confirmatory measurement models yield unsatisfactory fit and when instruments of unknowndimensionality are included in larger SEMs.

113

Structural Models

Conventionally, the SEM structural model consists of multiple linear regression models that describe theregressive dependencies between latent variables and covariates (see e.g., Jöreskog, 1970). However, simplemultiple regression is not appropriate for some research questions in educational psychology. In particular,many theoretically important questions involve the interplay of multiple levels, such as influences of classroomclimate on learning outcomes and motivation (e.g., Murayama & Elliot, 2009). Similarly, many theories ineducational psychology include moderation effects or non-linear relations between constructs. In the followingsection, we present multilevel SEMs and SEMs with non-linear effects that address these issues.

Multilevel Structure—How do We Account for the Clustered Structure of Educational Data?

Research in educational psychology is often conducted in a multilevel context: Students are clustered withinclassrooms and schools. This structure presents two important challenges. Substantively, research questionsare often tied to the interplay of different levels of nesting, for example, when studying the effect ofinstructional quality on student outcomes (e.g., Kunter et al., 2013), the effect of classroom composition onachievement and motivation (e.g., Marsh & Hau, 2003), or the effects of classroom goal structures andindividual goals (e.g., Murayama & Elliot, 2009). Methodologically, the clustering structure can violate theassumption of independent residuals, leading to biased standard errors (for an accessible introduction, seeSnijders & Bosker, 2011).

Multilevel models have been used for more than 20 years to address these substantive research questionsand to obtain correct standard errors. However, measurement error in the dependent and predictor variablescannot be easily controlled in conventional multilevel models, potentially biasing regression coefficients (e.g.,Harker & Tymms, 2004). In addition, conventional multilevel models are restricted to one outcome variable,making it difficult to test complex structural models (see e.g., Bauer, Preacher, & Gil, 2006). Over the courseof the last decade, easy-to-use implementations of multilevel SEMs have become increasingly available (e.g.,Muthén & Muthén, 1998–2012; Rabe-Hesketh et al., 2004). A good and concise introduction to differentmodeling frameworks for multilevel SEM is given by Rabe-Hesketh, Skrondal, and Zheng (2012).

Statistically, multilevel SEMs decompose the total variance–covariance matrix of the indicators into level-specific components (Mehta & Neale, 2005). This allows the simultaneous specification of measurement andstructural models at different levels. Thus, multilevel SEM has particular strengths for the analysis of effectsof learning environments whose properties are often difficult to assess directly. Instead, student ratings areused to assess constructs such as instructional quality (e.g., Fauth, Decristan, Rieser, Klieme, & Büttner, 2014;Wagner et al., 2013) or classroom climate (e.g., Mainhard, Brekelmans, & Wubbels, 2011). These ratings arethen aggregated at the classroom level and interpreted as properties of the teacher or classroom. It is quitecommon to find that the factor structure differs between the student and the classroom level (e.g., Schweig,2014).

Substantively, the effects of aggregated variables on other measures are often of great interest (e.g., Marsh& Hau, 2003). For this purpose, two possible effects can be distinguished (e.g., Enders & Tofighi, 2007;Lüdtke et al., 2008; Snijders & Bosker, 2011). Contextual effects are effects of the aggregated variable whenindividual differences in this variable are controlled (Enders & Tofighi, 2007; Marsh et al., 2012a). Climate

114

Box 4.3

effects (or between effects) are the effects of the aggregated variable without controlling for individualdifferences (Enders & Tofighi, 2007; Marsh et al., 2012a). The estimation of contextual and climate effects isfraught with difficulties. When individual measures are unreliable, contextual effects can be spuriously inflated(Harker & Tymms, 2004). Additional unreliability arises when the students are only a sample of all studentsin a classroom or when student ratings are intended to measure a construct at a higher level, such as classroomclimate (Lüdtke et al., 2008; Marsh et al., 2012a).

Although contextual and climate effects are highly relevant in educational psychology, the correct analyticalstrategy in multilevel SEMs has only recently been conceptually clarified. Lüdtke et al. (Lüdtke, Marsh,Robitzsch, & Trautwein, 2011; Marsh et al., 2009, 2012a) introduced a taxonomy that classifies multilevelSEMs with aggregated variables according to two dimensions: whether they control measurement error byemploying multiple indicators and whether they control sampling error by using a latent aggregationprocedure to correct the covariance matrix at the classroom level. The conventional multilevel model can beclassified as a doubly manifest model: It does not control measurement error as it is based on single items (orscale scores), and it does not control sampling error in the aggregation of individual responses as it uses theobserved cluster means as predictors. The doubly latent model represents the other extreme: It uses multipleindicators to identify latent predictor variables and controls for sampling error in the cluster means bycorrecting the upper-level covariance matrix. The latent manifest and the manifest latent model control onlyone source of error at a time, either measurement error in the constructs or sampling error in the aggregationof variables to the upper level.

What models are most appropriate in applications? Marsh et al. (2012a) argued that the appropriate modelspecification depends on the nature of the construct. When the effect of classroom composition is of primaryinterest and the construct has a clear meaning at the individual level (e.g., when average prior achievement isused to characterize a learning environment), the conventional analytical strategy of using the observed clustermeans as predictors is appropriate. If all students in a classroom have been assessed, there is no sampling error,and the observed cluster means reflect the learning environments appropriately. However, when individualratings are used to assess a property of a higher-order unit (e.g., when students rate characteristics of theirteacher), correcting for sampling error is the most appropriate analytical strategy. In this case, the studentratings are indicators of a latent characteristic: Each individual rating will be an imperfect reflection of theconstruct at the higher level, and this source of error will need to be controlled to obtain unbiased effects.

Marsh et al. (2012a) further argued that the two different types of variables require the interpretation ofdifferent coefficients in multilevel models. The effects of classroom composition are best represented bycontextual effects as they represent the effect of the context over and above individual differences. We presentan example of an application of a multilevel SEM for the analysis of the big-fish-little-pond effect (Marsh &Hau, 2003), a prime example of a contextual effect, in Box 4.3. The effects of climate variables, characteristicsof the higher-level unit assessed with individual ratings, are represented with climate or between effects.Individual differences in the perception of the higher-level unit do not represent true score variance but ratherunique person differences and interactions of the individuals and the higher-level unit. Hence, the effects ofthe aggregated ratings should not be controlled for these individual differences (Marsh et al., 2012a).

Studying the big-fish-little-pond effect (BFLPE) with multilevel SEM (Nagengast and

115

Marsh, 2012)

116

Research question

How are academic self-concept and scientific career aspirations of 15-year-old students affected byschool-average achievement, a contextual variable in a multilevel system? Research on the BFLPEpredicts negative effects of school-average achievement after controlling for individual achievementdifferences. Do these findings generalize across 56 culturally diverse countries?

117

•

Methodological challenge

Measurement error: How to take multiple indicators of the outcome variables (academic self-conceptand career aspirations) into account?Sampling error: How to account for unreliability in the school-average achievement scores due toassessing only a limited sample of students per school?

118

Method

A doubly latent multilevel SEM (Lüdtke et al., 2011; Marsh et al., 2009) with multiple outcome-variable indicators and latent aggregation of the individual achievement measures was applied at theschool level using data from 398,750 students assessed as part of Programme for International StudentAssessment (PISA) 2006.

119

Results

Clear negative contextual effects of school-average achievement were found on academic self-conceptand career aspirations in science, generalizing across the 56 countries.

120

Interpretation

Multilevel SEM helped to statistically control for the impact of the nesting of students within schools.In contrast to conventional multilevel models, it was possible to control for measurement error in thedependent variable and to take into account unreliability in school-average achievement due to samplingerror, thus leading to a more precise estimate of the BFLPE for academic self-concept and careeraspirations in science.

Multilevel SEM has begun to be picked up by researchers in educational psychology. Examples include

studies of instructional quality (Fauth et al., 2014; Wagner et al., 2013), the frame-of-reference effects onacademic self-concept (Marsh et al., 2014; Nagengast & Marsh, 2012), and of teachers’ professionalcompetencies (Kunter et al., 2013). However, it is not a panacea and will not be appropriate and applicable inall contexts. One of its main problems is the large sample size requirement at the higher level of nesting. Thedoubly latent model in particular can be run reliably only with samples of over 50 classrooms (Lüdtke et al.,2011). Even then, partial correction models provide more efficient, less variable parameter estimates (Lüdtkeet al., 2011). The sample size requirements also limit the number of indicator variables and the complexity ofstructural models that can be implemented. In addition, there is still some need to clarify rules for assessingmodel fit in multilevel SEM (see Ryu & West, 2009) and an ongoing debate about the importance of cross-level measurement invariance (Jak, Oort, & Dolan, 2013; Wagner et al., 2013). Alternatives to using themultilevel decomposition of variables such as the design-based correction of standard errors and fit statistics(Muthén & Satorra, 1995, cf., Gardiner, Luo & Roman, 2009; Subramanian & O´Malley, 2010) might bemore suitable in some situations. Hence, although the potential for applications of multilevel SEM ineducational psychology is high, the future will show how widespread the use of this model will be.

The Issue of Non-linearity—How do We Address Moderation Effects?

Moderation effects feature prominently in many theories in educational psychology. For example, theoreticalaccounts that postulate aptitude–treatment interactions (Cronbach & Snow, 1977) assume that the effects ofeducational interventions vary with aptitude levels. Classical models of expectancy-value theory assume thatexpectancy and value multiplicatively determine motivation (for a review, see Feather, 1982). Control-valuetheory of achievement emotions assumes that control and value beliefs determine achievement emotions incomplex non-linear patterns (Pekrun, 2006). However, moderation effects are notoriously difficult to establishempirically.

To illustrate this problem, we consider an influential modern version of expectancy-value theory asdeveloped by Eccles (see Eccles (Parsons), 1983). Eccles’ expectancy-value theory assumes that academicchoices, engagement, and ultimately achievement are influenced by two kinds of subjective beliefs: theexpectancy of success on a task and the subjective value ascribed to a task (Eccles (Parsons), 1983; Wigfield &Eccles, 2000). Since its initial formulation (Eccles (Parsons), 1983), the expectancy-value framework has beenempirically validated in many subjects and for many outcome variables (see Wigfield & Eccles, 2000, for areview). However, there is one specific theoretical assumption that sets the modern expectancy-value theory

121

Box 4.4

apart from most of its historic predecessors (e.g., Atkinson, 1957; Tolman, 1938, 1955). Interestingly, neithertheoretical accounts nor empirical tests of modern expectancy value seem to have tested interaction effects toaccount for the non-compensatory relation of expectancy and value in predicting achievement and motivation(see Feather, 1982, for the prominence of this assumption in classical expectancy-value models). For thehighest motivation, both task value and expectancy of success should be high.

Nagengast et al. (2011) argued that the main reason for the disappearance of the non-compensatory,multiplicative relation of expectancy and value was methodological. Traditionally, expectancy and value hadbeen manipulated in experimental designs (e.g., Atkinson, 1957; Tolman, 1955). In recent tests, however,both beliefs are typically assessed using multi-item questionnaires. This resulted in a change in the statisticalmethodology: Instead of analysis of variance, researchers have relied on multiple regression models (e.g.,Eccles (Parsons), 1983) or have used SEM to control for measurement error (e.g., Meece, Wigfield, & Eccles,1990). However, measurement error in the predictor variables will lead to biased regression weightsparticularly affecting the product variables used to test interaction effects (Blanton & Jaccard, 2006;Busemeyer & Jones, 1983). Hence, interaction effects of unreliable variables will be severely underestimated inmultiple regression models and easily discarded. Conventional SEMs, on the other hand, allow for lineareffects only in the structural model (e.g., Bollen, 1989), making it impossible to estimate interactions betweenlatent variables and limiting their use for testing interactions between grouping variables and latent variablesin multigroup models (Mayer, Nagengast, Fletcher, & Steyer, 2014; Sörbom, 1974).

Latent variable models with non-linear effects can be used to test for interactions between latent variables.However, it is very challenging to identify and estimate the effects of latent product variables, and this is themain reason why these models have only recently become available to applied researchers (Kenny & Judd,1984; Klein & Moosbrugger, 2000). Conceptually, there are two approaches for estimating structural equationmodels with latent interactions and latent quadratic effects. Product-indicator approaches identify the latentproduct variables by creating products of indicators and can be specified in conventional SEMs. Distribution-analytic approaches use the implications of interaction and quadratic effects in the structural model for thedistributions of the observed variables (Kelava et al., 2011) and require specialized software.

The first product-indicator approach (Kenny & Judd, 1984) required the specification of a large number ofnon-linear constraints—a rather cumbersome and error-prone exercise. In addition, there were only a fewSEM frameworks that allowed such constraints to be implemented at that time. Further developments stayedtrue to the original principle of using products of indicators but introduced considerable simplifications to theactual model specifications. The extended unconstrained approach (Kelava & Brandt, 2009; Marsh, Wen, &Hau, 2004), the current state of the art, requires only a single constraint in an interaction model and can bespecified in most conventional SEM software packages. Technical advice for specifying latent interactionmodels with this approach was given by Marsh, Wen, Nagengast, and Hau et al. (2012b) and Marsh, Wen,Hau, and Nagengast (2013). In Box 4.4, we present an example of the use of SEMs with latent interactionsbased on the product-indicator approach applied to expectancy-value theory.

Analyzing expectancy-value interactions with non-linear structural equation models(Nagengast et al., 2011)

122

123

Research question

How do the expectancy of success in science-related fields and the degree to which a person believes thatscience has value interact to predict career aspirations and engagement in extracurricular activities? Isthere evidence for a synergistic relation, that is, are the effects of expectancy and value particularly strongwhen both constructs are high? How do these effects generalize across a set of 57 culturally diversecountries?

124

Methodological challenges

Measurement error in scale scores attenuates estimates of interaction effects in moderated multipleregression models. Conventional SEMs cannot estimate interaction effects between latent variables.Latent interaction models allow for unbiased estimates of the interaction between expectancy and value.

125

Method

The unconstrained product-indicator approach (Marsh et al., 2004) to latent interactions was used tomodel the regression of career aspirations and extracurricular activities on expectancy of success, value ofscience, and their interaction (all measured with multiple items). Data were drawn from the backgroundquestionnaires from PISA 2006.

126

Results

Clear evidence for an interaction of expectancy of success in science and the intrinsic value of science wasfound for both career aspirations and engagement in extracurricular activities. Multigroup analysesshowed that these effects generalized across the 57 countries that participated in PISA 2006.

127

Interpretation

The latent interaction analyses revealed synergistic effects of value and expectancy of success. Forstudents who attributed low levels of intrinsic value to science, expectancy of success did not predictengagement in extracurricular activities and had only a weak relation to career aspirations. For studentswho attributed high levels of intrinsic value to science, academic self-concept was a strong predictor ofboth the engagement in extracurricular activities and career aspirations. The latent interaction modelmade it possible to obtain unbiased estimates of the interaction effects.

Despite their flexibility, product-indicator approaches have some statistical limitations. The formation of

product-indicators violates the assumption of normality and makes the interpretation of model fit statisticsand fit indices difficult (Jöreskog & Yang, 1996). Distribution-analytic approaches provide a principledalternative by modeling the implied non-normality of the indicators of the latent dependent variable (Kelavaet al., 2011). The latent moderated structural equations approach (Klein & Moosbrugger, 2000) is the mostpopular distribution-analytic approach and is included in commercial SEM software (Muthén & Muthén,1998–2012). Other distribution-analytic approaches (e.g., Klein & Muthén, 2007) are more robust but alsoless efficient (Kelava et al., 2011).

SEMs with latent interactions carry great potential for researchers in educational psychology, as manytheories include moderation effects and rely on latent variables to operationalize core constructs. Despite theirpotential, there have been only a few applications in educational psychology (e.g., Holzberger, Philipp, &Kunter, 2014; Nagengast et al., 2011; Trautwein et al., 2012). It is likely that the next few years will bring anincreased application of these models.

Conclusion

Advances in the structural model of SEMs have considerably broadened their scope for addressing researchquestions in educational psychology in recent years. However, many of these advances have yet to make abroader impact in applied research. Our examples illustrate the scope of potential applications of multilevelSEM and SEMs with non-linear structural models. However, we note that conventional applications of thesemodels are mostly for descriptive purposes. Causal interpretations of SEMs are much harder to justify, as wewill discuss in the next section.

128

Causal Inference

Large-scale research in educational psychology is often confronted with “real-world data” from observationalstudies or quasi-experiments in which students self-select to different conditions. Whereas this increasesexternal validity, the internal validity of conclusions is severely in doubt (Shadish, Cook, & Campbell, 2002).The “gold standard” (e.g., Rubin, 2008, p. 1350) for increasing internal validity, the randomized experiment,is often very difficult or impossible to implement (e.g., in the case of differential effects of learningenvironments; Becker, Lüdtke, Trautwein, Köller, & Baumert, 2012) or would be outright unethical (e.g., inthe case of grade retention; Hong & Raudenbush, 2006). In addition, uncovering the (causal) mechanism thatlinks an experimental manipulation to an outcome variable (i.e., analyses of mediation effects) is often of greatinterest and constitutes one of the main uses of latent variable models in educational psychology.

Indeed, latent variable models are often uncritically used as tools for causal inference and prescriptivestatements in educational psychology. Reinhart et al. (2013) noted that the proportion of causalinterpretations and prescriptive policy recommendations based on latent variable models increasedsubstantially between 2000 and 2010. However, when it comes to causal inference, latent variable models haveseveral limitations that require cautious interpretations. Foster (2010a) argued rightfully that theimplementation of a complex model does not imply that the parameters of this model also have a causalinterpretation. However, it is not the use of latent variable models per se that is problematic; it is the use ofresearch designs such as observational studies and quasi-experimental designs that limit justifiable conclusions.However, when it comes to the analyses of mediation effects, even a randomized design does not guaranteethat indirect effects obtained from a path analytic model have a causal interpretation. In the following section,we will review the counterfactual model of causality (Rubin, 1973, 1974, 1977) and describe how it informsthe treatment of selection effects in quasi-experimental designs. We will then discuss the limits andassumptions of causal interpretations in mediation models.

Uncovering Causal Effects from Quasi-experimental Designs

In non-experimental research designs, a large number of variables are often related to the selection of studentsto treatment conditions, making it difficult to distinguish selection processes from treatment effects (Rubin,1973, 1974, 1977). For example, single-sex schooling is often hypothesized to have a positive impact onacademic achievement and aspirations (e.g., Lee & Bryk, 1986). Yet, students attending single-sex schools arevery different from students attending coeducational schools. Typically, they come from more affluentbackgrounds and have higher achievement even before entering a single-sex school. These selectiondifferences have to be taken into account if the efficacy of single-sex education is to be fairly evaluated.Typically, researchers in educational psychology deal with selection effects by including large sets of potentialconfounders in multiple regression models (e.g., Lee & Bryk, 1986; Marsh, 1989; in studies of single-sexeducation). However, regression-based adjustment techniques have some severe shortcomings when used astools for causal inference (e.g., Schafer & Kang, 2008). These shortcomings become apparent when they areconsidered within formal theories of causal inference, such as the counterfactual model of causality (Neyman,1923/1990; Rubin, 1973, 1974, 1978).

Typically, the counterfactual model of causality is introduced in the context of two distinct treatment

129

groups (e.g., Morgan & Winship, 2007; Rubin, 1977; Schafer & Kang, 2008), referred to as the treatmentgroup and the control group. The treatment has to be manipulable at least in principle (Rosenbaum & Rubin,1983). The following thought experiment illustrates the model (e.g., Rubin, 1974; Rosenbaum & Rubin,1983): Each subject is assumed to have two potential outcomes, Y0 and Y1. The potential outcome Y1 indicates

the value of the outcome variable that would be observed if a participant was assigned to the controlcondition. The potential outcome indicates the value of the outcome variable that would be observed if theparticipant was assigned to the treatment condition. The difference between the two potential outcomes Y1 −

Y0 is the individual causal effect, the relative effect of the treatment on the individual. In many contexts, it

would be highly desirable to know the individual causal effect, for example, for assigning students to the mosteffective remedial training program or for adapting learning arrangements to the specific strengths of astudent. However, only one of the potential outcomes can ever be observed, creating the “fundamentalproblem of causal inference” (Holland, 1986, p. 947).

Within the counterfactual model of causality, various average causal effects in the total population, insubpopulations, or in treatment groups can be derived as expected values of the individual causal effects(Rosenbaum & Rubin, 1983; Rubin, 1973, 1974, 1977; Schafer & Kang, 2008). In a completely randomizedexperiment where all subjects have the same probability of being assigned to the treatment group, the averagecausal effect is identified by the observed mean difference between the treatment and the control group(Rubin, 1974). However, if there are systematic selection processes, the simple mean difference, the primafacie effect (Holland, 1986; Steyer, Gabler, von Davier, Nachtigall, & Buhl, 2000), does not identify theaverage causal effect.

In this case, the assumption of strong ignorability (Rosenbaum & Rubin, 1983; Rubin, 1978) is commonlyinvoked to identify the average causal effect. Technically, strong ignorability is the assumption of stochasticindependence of the potential outcome variables and the treatment variable conditional on the covariates(Rosenbaum & Rubin, 1983). Strong ignorability will be justifiable when all variables that influence theassignment of subjects to treatment groups and the outcome are observed. If this is the case, it is possible toimplement adjustment methods that are based on modeling the outcome variable (e.g., analysis of covariance:ANCOVA) or on modeling the probability of being assigned to the treatment condition, the propensity score(Rosenbaum & Rubin, 1983; Schafer & Kang, 2008).

In principle, both ANCOVA and propensity score methods are appropriate adjustment techniques as longas all relevant covariates are considered (Rosenbaum & Rubin, 1983) and the model is correctly specified(Cochran & Rubin, 1973; Schafer & Kang, 2008). However, propensity score methods have a number ofadvantages over conventional regression-based estimators. Similar to randomized experiments, they separatethe design stage and the analysis stage in causal inference for observational studies (Rubin, 2005). Thepropensity score approach allows researchers to come up with an optimal model for treatment assignment—the design—without having access to the outcome variable (Rubin, 2001). Furthermore, the specification ofthis model can be optimized with respect to the criterion of covariate balance that is grounded in thecounterfactual model of causality (Rosenbaum & Rubin, 1983). A correctly specified propensity score modelbalances the distributions of observed covariates between the treatment conditions. Imbalance (i.e., thedifferential distribution of covariates between the treatment and the control group) is indicative ofmisspecifications of the model for treatment assignment (see Stuart, 2010; Thoemmes & Kim, 2011). In

130

Box 4.5

addition, methods based on the propensity score also allow tests of overlap of the propensity scoredistributions to assess the assumption of common support, which is a precondition for meaningfulcomparisons of subjects in the control and the treatment group (Stuart, 2010; Thoemmes & Kim, 2011).Educational psychology has been comparatively slow in adopting the counterfactual model of causality as aconceptual and analytical framework (see Foster, 2010a; Thoemmes & Kim, 2011). Recently however, severalhigh-impact publications have used propensity score approaches to study, for example, the causal effects ofkindergarten retention (e.g., Hong & Raudenbush, 2006; Hughes, Chen, Thoemmes, & Kwok, 2010), high-intensity instruction (Hong & Raudenbush, 2008), differential learning environments (e.g., Becker et al.,2012), or the effects of working part-time during high school (e.g., Bachman, Staff, O’Malley, Schulenberg,& Freedman-Doan, 2011; Monahan, Lee, & Steinberg, 2011; Nagengast, Marsh, Chiorri, & Hau, 2014).Box 4.5 presents an example.

What is the potential of latent variable models when it comes to causal inference from observational studiesand quasi-experiments? It should be obvious from the discussion above that using SEM to analyze data fromquasi-experiments and observational studies will not per se solve the problems of causal inference (see alsoMartin, 2011). “Design trumps analysis” (Rubin, 2008, p. 808) for observational studies, and a careful accountof selection processes is required. Nevertheless, SEMs have the potential to contribute to some of the issuesthat affect causal inference. Within-study comparisons (Cook et al., 2009) suggest that measurement error inthe covariates is an important impediment to causal inference. If covariates are unreliable, adjustmenttechniques such as ANCOVA and propensity score matching will yield biased estimates of average effects.Multigroup SEMs can be used to control for measurement error in covariates that biases effect estimates inregression-based adjustment. Within educational psychology, multigroup multilevel SEM is particularlyuseful as it also allows contextual covariates to be incorporated with appropriate controls for both samplingerror and measurement error (Mayer et al., 2014; Nagengast, 2009). Controlling for measurement error inpropensity score models is more challenging, but some first SEM-based approaches are starting to appear(Raykov, 2012). SEMs are also useful tools for data analysis after participants have been matched on the basisof the propensity score, for example, in growth curve analyses (e.g., Goos, Van Damme, Onghena, Petry, &de Bilde, 2013).

Studying the effects of early ability grouping on students’ psychosocial development (Becker etal., 2014)

131

Research question

How does early transition into academically selective schools affect the development of academic self-concept, peer relations, school satisfaction, and school anxiety?

132

Methodological challenge

Students who stay in primary school longer (for a total of 6 years) differ from students who move to aselective academic track school after Grade 4 with respect to their achievement and the core dependentvariables. This selection effect needs to be adequately controlled to investigate the institutional effect ofacademically selective schools.

133

Method

Propensity score matching that controlled for group differences in the four outcome variables (academicself-concept, peer relations, school satisfaction, and school anxiety) and 20 background variables,including standardized achievement tests, grades, and socioeconomic status, were applied in a sample of155 early-entry and 3,169 regular students. Pretests and background variables were assessed at the end ofGrade 4. Outcome measures were taken at the end of Grade 5.

134

Results

There were large selection effects. At the end of Grade 4, students who entered academically selectiveschools reported a higher academic self-concept, better peer relations, and less school anxiety. Inaddition, they also had better test scores, better grades, and came from a more affluent socioeconomicbackground. After controlling for selection effects with propensity score matching, there were substantialnegative effects of early entry into selective schools on academic self-concept, peer relations, and schoolanxiety.

135

Interpretation

Propensity score matching allowed baseline differences to be controlled between students who enteredselective schools early and those who stayed in primary schools longer. The results converged across threedifferent matching approaches that provided a comparable balance of the covariates.

Mediation—How do We Test Hypotheses About Processes?

Mediation analysis is one of the main applications of path models in educational psychology. For example, inthe second issue of Journal of Educational Psychology in 2014 (volume 106, the current issue at the time thischapter was written), nine out of 18 articles included a mediation analysis (De Meyer et al., 2014; Dicke et al.,2014; Fuchs, Geary, Fuchs, Compton, & Hamlett., 2014a, Fuchs, Schumacher, et al., 2014b; Harackiewicz etal., 2014; Liem, Martin, Anderson, Gibson, & Sudmalis, 2014; Orkibi, Ronen, & Assoulin, 2014; Reeve &Lee, 2014; Stieff, Dixon, Ryu, Kumi, & Hegarty, 2014). Researchers use mediation analyses to uncover theprocesses that underlie the effect of a predictor on an outcome. Researchers in educational psychology areoften interested in studying the effects of learning environments on student outcomes. The identification ofmediating processes is of special interest in this area as it could lead to a better understanding of thephenomenon and suggest how interventions can be implemented. For example, grades have been discussed asan important mediator of frame-of-reference effects such as the BFLPE (e.g., Trautwein & Baeriswyl, 2007).The explicit (or implicit) goal is to obtain a causal interpretation of the resulting “mediating effects.” For thispurpose, the predictor is ideally a treatment to which students have been randomly assigned. Nevertheless, inlarge-scale applications, researchers also often use mediation analyses that are based on quasi-experimentaldesigns or observed predictor variables such as achievement (e.g., Nagengast & Marsh, 2012). However, thecausal interpretation of parameters in mediation models rests on strong assumptions that are often not easilyfulfilled, even in randomized experiments, let alone in quasi-experimental designs or observational studies(e.g., Imai, Keele, & Tingley, 2010; Sobel, 2008; Valeri & VanderWeele, 2013; VanderWeele, 2010;VanderWeele & Vansteelandt, 2009).

Let us first consider how a typical mediation analysis proceeds. Figure 4.2 shows a path diagram for amediation analysis (cf. Baron & Kenny, 1986). The total effect of the predictor variable (denoted as X inFigure 4.2) on the outcome variable (denoted as Y in Figure 4.2) is decomposed into a direct effect(represented by the arrow a) and an indirect effect transmitted by the mediator variable M. In a linear SEM,the indirect effect is the product of the path coefficients b, which links the predictor variable to the mediator,and c, which links the mediator to the outcome variable. A reduction in the coefficient of the direct path fromthe predictor to the outcome variable after including the mediator is interpreted as evidence of a mediationeffect (e.g., MacKinnon, 2008). Of course, researchers in educational psychology often have to deal with amultilevel structure, which adds another layer of complexity to mediation analyses. Although multilevel SEMhas emerged as a joint framework for multilevel mediation analysis, the best way of testing and interpreting amultilevel mediation hypothesis involving higher-level variables has yet to be determined (see e.g., Pituch &Stapleton, 2011, 2012; Preacher, Zyphur, & Zhang, 2010; Preacher, Zhang, & Zyphur, 2011; VanderWeele,

136

2010).

Figure 4.2 Path diagram of a mediation analysis. X = predictor variable; Y = outcome variable; M = mediator; a, b, c = path coefficients

In the last decade, methodological research has shown that causal interpretations of direct and indirecteffects in both single-level and multilevel mediation analyses are seldom defensible (e.g., Imai et al., 2010;Sobel, 2008; VanderWeele & Vansteelandt, 2009). Although these findings challenge the validity ofconclusions from conventional and multilevel mediation analyses, they have not led to a substantial change inhow educational psychologists analyze and interpret their data. As a formal presentation of causal mediationanalyses is beyond the scope of this chapter, we will provide a brief introduction to the main problems in theremainder. Interested readers are directed to Sobel (2008) and VanderWeele and Vansteelandt (2009), whopresent excellent discussions of causal mediation analysis.

VanderWeele and Vansteelandt (2009) use the counterfactual model of causality to distinguish threepotential causal effects that are to be estimated in mediation analysis. The controlled direct effect represents theexpected effect of the treatment variable if the mediator was held constant at a specific value in the population.The natural direct effect reflects the effect of the treatment if the mediator was held at its value in the controlgroup (i.e., if the effect of the treatment on the mediator was blocked). The natural indirect effect reflects theeffect of a change in the mediator if the treatment was held constant in the treatment group. Without furtherassumptions, none of these effects could be identified by the parameters of a linear mediation model.

The direct path from predictor to outcome (path a in Figure 4.2) has a causal interpretation only if there areno unmeasured confounders of the treatment–outcome relation and no unmeasured confounders of themediator–outcome relation (Valeri & VanderWeele, 2013; VanderWeele & Vansteelandt, 2009). An intuitiveway to understand this proposition is that, although subjects can be randomized to treatment conditions, thevalues of the mediator will obviously not be randomly assigned. Including the mediator as a predictor of theoutcome variable can also open up the door for new confounders of the treatment–outcome relation similar tosuppression effects so that a causal interpretation of this direct path would also be questionable (Steyer,Mayer, & Fiege, in press). Hence, causal interpretations of the direct effect in a mediation analysis require thecontrol of all potential confounders of the treatment–outcome relation conditional on the mediator and of allconfounders of the mediator–outcome relation (VanderWeele & Vansteelandt, 2009). In addition, a causalinterpretation of the direct effect rests on the assumption that there are no unmodeled interactions orquadratic effects between the predictor and the mediator (Valeri & VanderWeele, 2013; VanderWeele &Vansteelandt, 2009). On the basis of these assumptions, it is possible to estimate the controlled direct effect.

137

However, this effect is seldom of interest in applications.

The causal identification of natural direct and indirect effects requires additional assumptions, specificallythat there are no unmeasured confounders of the treatment–mediation relation and that there are noadditional mediators that could be affected by the treatment (VanderWeele & Vansteelandt, 2009). Suchadditional mediators would be manifested as confounders of the mediator–outcome relation. Whereasrandomization guarantees that there are no confounders of the treatment–mediator relation, it is very difficultto guarantee that there are no confounders of the mediator–outcome relation. Even in simple linear mediationmodels with continuous outcomes, an unbiased estimation of the indirect effect will require the inclusion ofconfounders of the mediator–outcome relation. Things get even more complicated in designs without randomassignment to treatment conditions when confounders of the treatment–mediator relation have to becontrolled. Furthermore, the statistical implementation of a mediation analysis and the causal interpretation ofmediation effects need to go beyond conventional SEM when mediator variables are not continuous and whenthere are moderation effects (e.g., when the treatment–outcome relation is moderated by the mediator orwhen the treatment affects the mediator–outcome relation). Although some approaches to causal mediationanalysis (e.g., Imai et al., 2010; Muthén, 2011; Valeri & VanderWeele, 2013) address some of the issuesintroduced above, there is currently no consensus on the most appropriate method and the assumptionsrequired to make it work (see e.g., Imai et al., 2010; Valeri & VanderWeele, 2013). Even the definition ofcausal mediation effects has not been decisively clarified and remains contested between researchers (e.g.,Pearl, 2012).

Conclusion. Structural parameters in latent variable models are all too often given a causal interpretation andused to justify policy implications (Reinhart et al., 2013). As a field, educational psychology would be wellserved to fully embrace formal frameworks for causal inference (e.g., Rubin’s causal model) and to use theinsights derived from these theories to strengthen research design and data analysis. Employing good researchdesigns would imply that randomized designs be embraced in field settings as well (see e.g., Harackiewicz,Rozek, Hulleman, & Hyde, 2012; Hulleman & Harackiewicz, 2009, for examples from motivational research)but also that other techniques be applied in field studies such as regression discontinuity designs (Imbens &Lemieux, 2008), instrumental variable techniques (Heckman & Vytlacil, 2007), or propensity score methods,as discussed above, all of which are prominent in other social sciences. Uncritically relying on a causalinterpretation of structural model parameters in SEM, however, will not be sufficient for moving the fieldforward.

138

Outlook: Prospects of Latent Variable Models in Educational Psychology

In this chapter, we reviewed the actual and potential uses of advances in latent variable modeling ineducational psychology. In doing so, we focused on measurement models, structural models, and causalinference. Unsurprisingly, latent variable models are not a panacea for all data analytic challenges, andparticular caution is needed when causal inference instead of mere descriptive analyses is desired. In thisoutlook, we discuss some of the prospects and future directions for the use of latent variable models ineducational psychology.

Latent variable models are popular tools for scale construction and scale development in educationalpsychology. Tests of measurement invariance are well-entrenched research practices in educational psychology,although there seems to be the potential for a greater use of measurement models that are based on generalizedlatent variable modeling, particularly for psychosocial constructs. ESEM offers a viable alternative toconfirmatory measurement models when the dimensionality of indicators is unclear or when cross-loadingsneed to be taken into account. However, it also comes with limitations regarding the interpretability offactors. Future developments, such as Bayesian SEM (Muthén & Asparouhov, 2012), promise to furtherexpand analytical options but also demand that users have a high level of statistical sophistication.Furthermore, discussions about the fundamentals of validity theory and measurement (e.g., Markus &Borsboom, 2013) have the potential to fundamentally affect the way educational psychologists think aboutdeveloping and validating their measures.

With respect to advances in the structural model, multilevel SEM carries great promise for researchers ineducational psychology. However, the sample requirements make it unlikely that doubly latent models will beimplemented as a standard procedure. Studying the tradeoff between the accuracy and variability of theestimates (Lüdtke et al., 2011) and finding ways to deal with small sample sizes at higher levels of analysis willbe important methodological research topics. Further work is needed on computational simplifications forestimating cross-level interactions between latent variables and random effects on factor loadings. In addition,conceptual clarifications as to when and whether it is appropriate to decompose variances and observations ina full-blown multilevel SEM are needed (see e.g., Rabe-Hesketh et al., 2012).

Non-linear SEMs can be directly employed to test moderation hypotheses, which are central in educationalpsychological theories. They promise stronger tests of interaction effects and the chance to better aligntheoretical predictions and data analytic methods. Future developments will extend the flexibility of thesemodels through semiparametric approaches (Bauer, 2005) and corrections for non-normally distributedvariables (Kelava & Nagengast, 2012; Kelava, Nagengast, & Brandt, 2014).

When it comes to causal inference in quasi-experimental designs and observational studies, the choice of aresearch design that accounts for selection effects is more important than the choice of a specific analyticaltechnique. Causal inference, first and foremost, requires that all relevant confounders be measured andmodeled. Latent variable models have their role in modeling (e.g., by controlling for measurement error andalleviating assumptions of conventional statistical models such as the heterogeneity of residual variances).

The causal interpretation of path coefficients, particularly in mediation models, requires even strongerassumptions that often remain implicit. Even in randomized experiments, a causal interpretation of direct andindirect effects is problematic. In non-experimental settings and in cross-sectional studies in particular, causal

139

interpretation will almost always be unwarranted. Longitudinal designs alone do not guarantee that a causalinterpretation of parameters is possible (Foster, 2010b). Mediation analyses and path analysis are so wellentrenched as methods for establishing causal processes that the fundamental critique of existing approachesshould lead to changes in the way data are collected, analyzed, and interpreted if the field does not want tolose credibility compared with fields such as economics or sociology, which take causal inference moreseriously (see e.g., Foster, 2010a, 2010b). However, a consensus about the most appropriate framework andmethods for causal mediation analysis has yet to be reached.

Future trends in educational psychology are likely to increase the need for an informed use of latent variablemodels. Increasingly, large datasets from panel studies, international large-scale assessments, and state-wideassessments include measures that are relevant for research in educational psychology. However, bringingthem to fruition for addressing research questions requires that issues of measurement, multilevel structure,and causality be addressed in an informed way. Otherwise, educational psychology easily runs the risk ofoverstating the potential implications while not using the available information in the data to its full potential.Large-scale randomized designs are also likely to be used more frequently for the evaluation of interventioneffects in classrooms or schools. Standards set by institutions such as the What Works Clearinghouse (2008)make randomized studies, potentially with randomization at the classroom level, a requirement for achievingthe highest standards of evidence. Obviously, questions of causal inference and research design will thus havea new bearing on educational psychology, but well-constructed measures and their invariance across treatmentgroups and classrooms will also be required. Finally, there will likely be a desire to study the processesunderlying intervention effects, and this issue ties back to the problems of causal mediation analysis.Measurement issues (e.g., invariance, unidimensionality) are also likely to become more relevant in small-scaleexperimental work on learning and teaching. In addition, it is this part of educational psychology that mostoften uses mediation analysis to test process hypotheses in experimental designs. Thus, issues of the causalinterpretation of mediation effects will likely impact researchers significantly and might lead to areconsideration of research designs and analytical methods. Finally, the emergence of intensive longitudinaldata (e.g., in experience-based sampling studies; Hektner, Schmidt, & Csikszentmihalyi, 2006) andbehavioral measures obtained from computerized learning environments pose new challenges for measurementand causal inference that will likely bring an increased use of modeling techniques.

The challenge for educational psychologists is to keep abreast of methodological developments in order tobe able to identify the most appropriate research design and statistical approach for addressing a substantiveresearch question (Marsh & Hau, 2007). Substantive research ideas and knowledge need to be combined witha sound methodological understanding to advance the field as a whole (Marsh & Hau, 2007). The warningsagainst the limited quantitative proficiency of the field (Henson et al., 2010) should be taken seriously andshould inspire educational psychologists to recognize the value of carefully designed research over advancedmodeling techniques (Foster, 2010a). This emphasis should be applied in the editing and reviewing of papersas well. On the other hand, choosing a statistical model will always involve compromises. In realisticallycomplex settings, it will almost never be possible to represent all aspects of the data collection process (e.g.,the multilevel structure, violations of distributional assumptions) in a statistical model. The challenge foreducational psychology as a field is to select strong and informative research designs, assess constructs withpsychometrically sound measures, choose statistical models that represent the specificities of the design, but

140

carefully consider the limitations of the research design and the analytical approaches chosen in order toadvance knowledge of learning, teaching, and education.

141

Note

We thank Eric Anderman, Derek Briggs, Andrew Martin, and Norman Rose for comments on earlier versions of the chapter.

142

References

Aiken, L. S., West, S. G., & Millsap, R. E. (2008). Doctoral training in statistics, measurement, and methodology in psychology: Replicationand extension of Aiken, West, Sechrest, and Reno’s (1990) survey of PhD programs in North America. American Psychologist, 63, 32–50.

Asparouhov, T., & Muthén, B. O. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal,16, 397–438.

Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64, 359–372.Bachman, J. G., Staff, J., O’Malley, P. M., Schulenberg, J. E., & Freedman-Doan, P. (2011). Twelfth-grade student work intensity linked to

later educational attainment and substance use: New longitudinal evidence. Developmental Psychology, 47, 344–363.Bandalos, D. L., & Finney, S. J. (2001). Item parceling issues in structural equation modeling. In G. A. Marcoulides & R. E. Schumacker

(Eds.), New developments and techniques in structural equation modeling (pp. 269–296). Mahwah, NJ: Erlbaum.Baron, R. M., & Kenny, D. A. (1986) The moderator-mediator variable distinction in social psychological research: conceptual, strategic and

statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.Bauer, D. J. (2005). A semiparametric approach to modeling nonlinear relations among latent variables. Structural Equation Modeling, 12, 513–

535.Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel

models: New procedures and recommendations. Psychological Methods, 11, 142–163.Becker, M., Lüdtke, O., Trautwein, U., Köller, O., & Baumert, J. (2012). The differential effects of school tracking: Do academic-track schools

make students smarter? Journal of Educational Psychology, 104, 682–699.Becker, M., Neumann, M., Tetzner, J., Böse, S., Knoppick, H., Maaz, K., . . . Lehmann, R. (2014). Is early ability grouping good for high-

achieving students’ psychosocial development? Effects of the transition into academically selective schools. Journal of Educational Psychology,106, 555–568.

Blanton, H., & Jaccard, J. (2006). Tests of multiplicative models in psychology: A case study using the unified theory of implicit attitudes,stereotypes, self-esteem, and self-concept. Psychological Review, 113, 155–166.

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150.Brunner, M., Keller, U., Hornung, C., Reichert, M., & Martin, R. (2009). The cross-cultural generalizability of a new structural model of

academic self-concepts. Learning and Individual Differences, 19, 387–403.Bulotsky-Shearer, R. J., Dominguez, X., & Bell, E. R. (2012). Preschool classroom behavioral context and school readiness outcomes for low-

income children: A multilevel examination of child- and classroom-level influences. Journal of Educational Psychology, 104, 421.Busemeyer, J. R., & Jones, L. E. (1983). Analysis of multiplicative combination rules when the causal variables are measured with error.

Psychological Bulletin, 93, 549–562.Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276.Chen, F. F. (2008). What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural

research. Journal of Personality and Social Psychology, 95, 1005–1018.Cochran, W. G., & Rubin, D. B. (1973). Controlling bias in observational studies: A review. Sankhya-A, 35, 417–446.Cook, T. D., Steiner, P., & Pohl, S. (2009). How bias reduction is affected by covariate choice, unreliability, and mode of data analysis: Results

from two types of within-study comparisons. Multivariate Behavioral Research, 44, 828–847.Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. Oxford, UK: Irvington.De Meyer, J., Tallir, I. B., Soenens, B., Vansteenkiste, M., Aelterman, N., Van den Berghe, L., . . . & Haerens, L. (2014). Does observed

controlling teaching behavior relate to students’ motivation in physical education? Journal of Educational Psychology, 106, 541–554.Dicke, T., Parker, P. D., Marsh, H. W., Kunter, M., Schmeck, A., & Leutner, D. (2014). Self-efficacy in classroom management, classroom

disturbances, and emotional exhaustion: A moderated mediation analysis of teacher candidates. Journal of Educational Psychology, 106, 569–583.

DiStefano, C. (2002). The impact of categorization with confirmatory factor analysis. Structural Equation Modeling, 9, 327–346.Dolan, C. V., Oort, F. J., Stoel, R. D., & Wicherts, J. M. (2009). Testing measurement invariance in the target rotated multigroup exploratory

factor model. Structural Equation Modeling: A Multidisciplinary Journal, 16, 295–314.Eccles (Parsons), J. S. (1983). Expectancies, values, and academic behaviours. In J. T. Spence (Ed.), Achievement and achievement motivation (pp.

75–146). San Francisco, CA: W. H. Freeman.Elliot, A. J., & McGregor, H. A. (2001). A 2×2 achievement goal framework. Journal of Personality and Social Psychology, 80, 501–519.Elliot, A. J., Murayama, K., & Pekrun, R. (2011). A 3×2 achievement goal model. Journal of Educational Psychology, 103, 632.Enders, C., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological

143

Methods, 12, 121–138.Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014). Student ratings of teaching quality in primary school: Dimensions and

prediction of student outcomes. Learning and Instruction, 29, 1-9.Feather, N. T. (1982). Expectancy-value approaches: Present status and future directions. In N. T. Feather (Ed.), Expectations and actions:

Expectancy-value models in psychology (pp. 395–420). Hillsdale, NJ: Erlbaum.Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal

data. Psychological Methods, 9, 466–491.Foster, E. M. (2010a). The U-shaped relationship between complexity and usefulness: A commentary. Developmental Psychology, 46, 1760–

1766.Foster, E. M. (2010b). Causal inference and developmental psychology. Developmental Psychology, 46, 1454–1480.Freedman, D. A. (1987). As others see us—a case-study in path-analysis. Journal of Educational Statistics, 12, 101–128.Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of

Educational Research, 74, 59–109.Friedrich, A., Jonkmann, K., Nagengast, B., Schmitz, B., & Trautwein, U. (2013). Teachers’ and students’ perceptions of self-regulated

learning and math competence: Differentiation and agreement. Learning and Individual Differences, 27, 26–34.Fuchs, L. S., Geary, D. C., Fuchs, D., Compton, D. L., & Hamlett, C. L. (2014a). Sources of individual differences in emerging competence

with numeration understanding versus multidigit calculation skill. Journal of Educational Psychology, 106, 482–498.Fuchs, L. S., Schumacher, R. F., Sterba, S. K., Long, J., Namkung, J., Malone, A., . . . & Changas, P. (2014b). Does working memory

moderate the effects of fraction intervention? An aptitude–treatment interaction. Journal of Educational Psychology, 106, 499–514.Gardiner, J. C., Luo, Z. & Roman, L. A. (2009). Fixed effects, random effects and GEE: What are the differences. Statistics and Medicine, 28,

221–239.Goos, M., Van Damme, J., Onghena, P., Petry, K., & de Bilde, J. (2013). First-grade retention in the Flemish educational context: Effects on

children’s academic growth, psychosocial growth, and school career throughout primary education. Journal of School Psychology, 51, 323–347.Grouzet, F. M., Otis, N., & Pelletier, L. G. (2006). Longitudinal cross-gender factorial invariance of the Academic Motivation Scale. Structural

Equation Modeling, 13, 73–98.Guay, F., Morin, A. J. S., Litalien, D., Valois, P., & Vallerand, R. J. (2015). Application of exploratory structural equation modeling to evaluate

the Academic Motivation Scale. Journal of Experimental Education, 83, 51–82.Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde, J. S. (2014). Closing the social class

achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106, 375–389.Harackiewicz, J. M., Rozek, C. S., Hulleman, C. S., & Hyde, J. S. (2012). Helping parents to motivate adolescents in mathematics and science:

An experimental test of a utility–value intervention. Psychological Science, 23, 899–906.Harker, R., & Tymms, P. (2004). The effects of student composition on school outcomes. School Effectiveness and School Improvement, 15, 177–

199.Heckman, J. J., & Vytlacil, E. J. (2007). Econometric evaluation of social programs, part I: Causal models, structural models and econometric

policy evaluation. In J. J. Heckman & E. E. Leamer (Eds.), Handbook of econometrics (Vol. 6, pt. 2, pp. 4779–4874). Amsterdam, theNetherlands: North-Holland.

Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2006). Experience sampling method: Measuring the quality of everyday life. ThousandOaks, CA: Sage.

Henson, R. K., Hull, D. M., & Williams, C. S. (2010). Methodology in our education research culture. Educational Researcher, 39, 229–240.Holland, P. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960.Holzberger, D., Philipp, A., & Kunter, M. (2014). Predicting teachers’ instructional behaviors: The interplay between self-efficacy and intrinsic

needs. Contemporary Educational Psychology, 39, 100–111.Hong, G., & Raudenbush, S. W. (2006). Evaluating kindergarten retention policy. Journal of the American Statistical Association, 101, 901–910.Hong, G., & Raudenbush, S. (2008). Causal inference for time-varying instructional treatments. Journal of Educational and Behavioral Statistics,

33, 333–362.Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185.Hughes, J., Chen, Q., Thoemmes, F., & Kwok, O. (2010). An investigation of the relationship between retention in first grade and

performance on high stakes tests in 3rd grade. Educational Evaluation and Policy Analysis, 32, 166–182.Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326, 1410–1412.Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309–334.Imbens, G., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142, 615–635.Jacobs, J. E., Lanza, S., Osgood, D. W., Eccles, J. S., & Wigfield, A. (2002). Changes in children’s self-competence and values: Gender and

144

domain differences across grades one through twelve. Child Development, 73, 509–527.Jak, S., Oort, F. J., & Dolan, C. V. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel

data. Structural Equation Modeling, 20, 265–282.Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.Jöreskog, K. G. (1970). A general method for analysis of covariance structures. Biometrika, 57, 239–251.Jöreskog, K. G., & Yang, F. (1996). Nonlinear structural equation models: The Kenny-Judd model with interaction effects. In G. Markoulides

& R. Schumacker (Eds.), Advanced structural equation modeling (pp. 57–87). Mahwah, NJ: Erlbaum.Kaiser, H.F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.Kelava, A., & Brandt, H. (2009). Estimation of nonlinear latent structural equation models using the extended unconstrained approach. Review

of Psychology, 16, 123–131.Kelava, A., & Nagengast, B. (2012). A bayesian model for the estimation of latent interaction and quadratic effects when latent variables are

non-normally distributed. Multivariate Behavioral Research, 47, 717–742.Kelava, A., Nagengast, B., & Brandt, H. (2014). A nonlinear structural equation mixture modeling approach for non-normally distributed

latent predictor variables. Structural Equation Modeling, 21, 468–481.Kelava, A., Werner, C. S., Schermelleh-Engel, K., Moosbrugger, H., Zapf, D., Ma, Y., Cham, H., Aiken, L., & West, S. G. (2011). Advanced

nonlinear latent variable modeling: Distribution analytic LMS and QML estimators of interaction and quadratic effects. Structural EquationModeling: A Multidisciplinary Journal, 18, 465–491.

Kenny, D. A., & Judd, C. M. (1984). Estimating the nonlinear and interactive effects of latent variables. Psychological Bulletin, 96, 201–210.Klein, A. G., & Moosbrugger, H. (2000). Maximum likelihood estimation of latent interaction effects with the LMS method. Psychometrika,

65, 457–474.Klein, A. G., & Muthén, B. O. (2007). Quasi maximum likelihood estimation of structural equation models with multiple interaction and

quadratic effects. Multivariate Behavioral Research, 42, 647–674.Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on

instructional quality and student development. Journal of Educational Psychology, 105, 805–820.Lee, V., & Bryk, A. (1986). Effects of single-sex secondary schools on student achievement and attitudes. Journal of Educational Psychology, 78,

381–395.Liem, G. A. D., Martin, A. J., Anderson, M., Gibson, R., & Sudmalis, D. (2014). The role of arts-related information and communication

technology use in problem solving and achievement: findings from the programme for international student assessment. Journal of EducationalPsychology, 106, 348–363.

Little, T. D., Rhemtulla, M., Gibson, K., & Schoemann, A. M. (2013). Why the items versus parcels controversy needn’t be one. PsychologicalMethods, 18, 285–300.

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Oxford, UK: Addison-Wesley.Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011). A 2x2 taxonomy of multilevel latent contextual models: Accuracy and bias

tradeoffs in full and partial error-correction models. Psychological Methods, 16, 444–467.Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A

new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203–229.MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Erlbaum.Mainhard, M. T., Brekelmans, M., & Wubbels, T. (2011). Coercive and supportive teacher behaviour: Within- and across-lesson associations

with the classroom social climate. Learning and Instruction, 21, 345–354.Markus, K. A., & Borsboom, D. (2013). Frontiers of test validity theory: Measurement, causation, and meaning. New York: Routledge.Marsh, H. W. (1989). Effects of attending single-sex and coeducational high schools on achievement, attitudes, behaviors, and sex differences.

Journal of Educational Psychology, 81, 70–85.Marsh, H. W., Abduljabbar, A. S., Parker, P. D., Morin, A. J. S., Abdelfattah, F., & Nagengast, B. (2014). The big-fish-little-pond effect in

mathematics: A cross-cultural comparison of U.S. and Saudi Arabian TIMSS responses. Journal of Cross-Cultural Psychology, 45, 777–804.Marsh, H. W., & Hau, K. (2003). Big-fish-little-pond effect on academic self-concept. A cross-cultural (26-country) test of the negative effects

of academically selective schools. American Psychologist, 58, 364–376.Marsh, H. W., & Hau, K.-T. (2007). Applications of latent-variable models in educational psychology: The need for methodological-

substantive synergies. Contemporary Educational Psychology, 32, 151–170.Marsh, H. W., & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First-and higher order factor

models and their invariance across groups. Psychological Bulletin, 97, 562–582.Marsh, H. W., Liem, G. A. D., Martin, A. J., Morin, A. J. S., & Nagengast, B. (2011). Methodological measurement fruitfulness of

exploratory structural equation modeling (ESEM): New approaches to key substantive issues in motivation and engagement. Journal of

145

Psychoeducational Assessment, 29, 322–346.Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the Big Five

factor structure through exploratory structural equation modeling. Psychological Assessment, 22, 471–491.Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S. & Köller, O. (2012a). Classroom climate and

contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47(2), 106–124.Marsh, H. W., Lüdtke, O., Nagengast, B., Morin, A. J. S., & Davier, Von, M. (2013). Why item parcels are (almost) never appropriate: Two

wrongs do not make a right—camouflaging misspecification with item parcels in CFA models. Psychological Methods, 18, 257–284.Marsh, H. W., Muthén, B. O., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory structural

equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16,439–476.

Marsh, H. W., Nagengast, B., Morin, A. J. S., Parada, R. H., Craven, R. G., & Hamilton, L. R. (2011b). Construct validity of themultidimensional structure of bullying and victimization: An application of exploratory structural equation modeling. Journal of EducationalPsychology, 103, 701–732.

Marsh, H.W., & Shavelson, R. (1985). Self-concept: Its multifaceted, hierarchical structure. Educational Psychologist, 20, 107–125.Marsh, H. W., Wen, Z., & Hau, K.-T. (2004). Structural equation models of latent interactions: Evaluation of alternative estimation strategies

and indicator construction. Psychological Methods, 9, 275–300.Marsh, H.W., Wen, Z., Hau, K.-T., & Nagengast, B. (2013). Structural equation models of latent interaction and quadratic effects. In G. R.

Hanco*ck & R. O. Mueller (Eds.), Structural equation modeling. A second course (2nd ed., pp. 395–438). Greenwich, CT: Information Age.Marsh, H.W., Wen, Z., Nagengast, B., & Hau, K.-T. (2012b). Structural equation models of latent interactions. In R.D. Hoyle (Ed.),

Handbook of structural equation modeling (pp. 436–458). New York: Guilford Press.Martin, A. J. (2007). Examining a multidimensional model of student motivation and engagement using a construct validation approach. British

Journal of Educational Psychology, 77, 413–440.Martin, A. J. (2011). Prescriptive statements and educational practice: What can structural equation modeling (SEM) offer? Educational

Psychology Review, 23, 235–244.Matsueda, R. L. (2012). Key advances in the history of structural equation modeling. In R. D. Hoyle (Ed.), Handbook of structural equation

modeling (pp. 17–42). New York: Guilford Press.Mayer, A., Nagengast, B., Fletcher, J., & Steyer, R. (2014). Analyzing average and conditional effects with multigroup multilevel structural

equation models. Frontiers in Psychology, 5, 304.Meece, J. L., Wigfield, A., & Eccles, J. S. (1990). Predictors of math anxiety and its influence on young adolescents course enrolment

intentions and performance in mathematics. Journal of Educational Psychology, 82, 60–70.Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10, 259–284.Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543.Michell, J. (1997). Quantitative science and the definition of measurement in psychology. British Journal of Psychology, 88, 355–383.Midgley, C., Kaplan, A., Middleton, M., Maehr, M. L., Urdan, T., Anderman, L. H., . (1998). The development and validation of scales

assessing students’ achievement goal orientations. Contemporary Educational Psychology, 23, 113–131.Millsap, R.E. (2011). Statistical approaches to measurement invariance. New York: Routledge.Monahan, K. C., Lee, J. M., & Steinberg, L. (2011). Revisiting the impact of part-time work on adolescent adjustment: Distinguishing

between selection and socialization using propensity score matching. Child Development, 82, 96–112.Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. Cambridge: Cambridge

University Press.Morin, A.J.S., Marsh, H.W., & Nagengast, B. (2013). Exploratory structural equation modeling: An introduction. In G. R. Hanco*ck & R. O.

Mueller (Eds.), Structural equation modeling: A second course, 2nd ed. Greenwich, CT: IAP.Murayama, K., & Elliot, A. (2009). The joint influence of personal achievement goals and classroom goal structures on achievement-relevant

outcomes. Journal of Educational Psychology, 101, 432–447.Muthén, B. O. (2002). Beyond SEM: General latent variable modeling. Behaviormetrika, 29, 81–117.Muthén, B. O. (2011). Applications of causally defined direct and indirect effects in mediation analysis using SEM in Mplus. Retrieved from

http://www.statmodel.com/papers.shtml (accessed February 18, 2015).Muthén, B. O., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory.

Psychological Methods, 17, 313–335.Muthén, B. O., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British

Journal of Mathematical and Statistical Psychology, 38, 171–189.Muthén, B. O., & Kaplan, D. (1992). A comparison of some methodologies for the factor analysis of non-normal Likert variables: A note on

146

http://www.statmodel.com/papers.shtml

the size of the model. British Journal of Mathematical and Statistical Psychology, 45, 19–30.Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.Muthén, B. O., & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316.Muthén, B. O., du Toit, S. H. C., & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent

variable modeling with categorical and continuous outcomes. Retrieved from http://www.gseis.ucla.edu/faculty/muthen/articles/Article_075.pdf(accessed February 18, 2015).

Nagengast, B. (2009). Causal inference in multilevel designs. Unpublished doctoral dissertation. Jena: Friedrich-Schiller-Universität, Faculty ofSocial and Behavioral Sciences.

Nagengast, B., & Marsh, H. W. (2012). Big fish in little ponds aspire more: Mediation and cross-cultural generalizability of school-averageability effects on self-concept and career aspirations in science. Journal of Educational Psychology, 104, 1033–1053.

Nagengast, B., & Marsh, H.W. (2013). Motivation and engagement in science around the globe: Testing measurement invariance withmultigroup structural equation models across 57 countries using PISA 2006. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.),Analysis of international large-scale assessment data. New York: Taylor & Francis.

Nagengast, B., Marsh, H.W., Chiorri, C., & Hau, K.-T. (2014). Character building or subversive consequences of employment during highschool: Causal effects based on propensity score models for categorical treatments. Journal of Educational Psychology, 106, 584–603.

Nagengast, B., Marsh, H. W., Scalas, L. F., Xu, M. K., Hau, K.-T., & Trautwein, U. (2011). Who took the “x” out of expectancy-valuetheory? A psychological mystery, a substantive-methodological synergy, and a cross-national generalization. Psychological Science, 22, 1058–1066.

Neyman, J. (1923/1990). On the application of probability theory to agricultural experiments. Essay on principles. Statistical Science, 5, 465–480.Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika, 44, 443–460.Orkibi, H., Ronen, T., & Assoulin, N. (2014). The subjective well-being of Israeli adolescents attending specialized school classes. Journal of

Educational Psychology, 106, 515–526.Pearl, J. (2012). The mediation formula: A guide to the assessment of causal pathways in non-linear models. Prevention Science, 13, 426–436.Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and

practice. Educational Psychology Review, 18, 315–341.Pituch, K. A., & Stapleton, L. M. (2011). Hierarchical linear and structural equation modeling approaches to mediation analysis in randomized

field experiments. In M. Williams & P. Vogt (Eds.), The SAGE handbook of innovation in social research methods (pp. 590–619). ThousandOaks, CA: SAGE.

Pituch, K. A., & Stapleton, L. M. (2012). Distinguishing between cross- and cluster-level mediation processes in the cluster randomized trial.Sociological Methods & Research, 41, 630–670.

Preacher, K. J., Zhang, Z., & Zyphur, M. J. (2011). Alternative methods for assessing mediation in multilevel data: the advantages of multilevelSEM. Structural Equation Modeling, 18, 161–182.

Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. PsychologicalMethods, 15, 209–233.

Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multilevel structural equation modelling. Psychometrika, 69, 167–190.Rabe-Hesketh, S., Skrondal, A. & Zheng, X. (2012). Multilevel structural equation modeling. In R. D. Hoyle (Ed.), Handbook of structural

equation modeling (pp. 512–531). New York: Guilford Press.Raykov, T. (2012). Propensity score analysis with fallible covariates: A note on a latent variable modeling approach. Educational and Psychological

Measurement, 72, 715–733.Reckase, M. D. (1997). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 25–36.Reeve, J., & Lee, W. (2014). Students’ classroom engagement produces longitudinal changes in classroom motivation. Journal of Educational

Psychology, 106, 527–540.Reinhart, A. L., Haring, S. H., Levin, J. R., Patall, E. A., & Robinson, D. H. (2013). Models of not-so-good behavior: Yet another way to

squeeze causality and recommendations for practice out of correlational data. Journal of Educational Psychology, 105, 241–247.Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41.Rubin, D. B. (1973). The use of matching and regression adjustment to remove bias in observational studies. Biometrics, 29, 185–203.Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66,

688–701.Rubin, D. B. (1977). Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics, 2, 1–26.Rubin, D. B. (1978). Bayesian-inference for causal effects: The role of randomization. Annals of Statistics, 6, 34–58.Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services &

Outcomes Research Methodology, 2, 169–188.

147

http://www.gseis.ucla.edu/faculty/muthen/articles/Article_075.pdf

Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100,322–331.

Rubin, D. B. (2008). For objective causal inference, design trumps analysis. The Annals of Applied Statistics, 2, 808–840.Ryu, E., & West, S. G. (2009). Level-specific evaluation of model fit in multilevel structural equation modeling. Structural Equation Modeling,

16, 583–601.Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological

Methods, 13, 279–313.Schweig, J. (2014). Cross-level measurement invariance in school and classroom environment surveys. Educational Evaluation and Policy

Analysis, 36, 259–280.Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental design for generalized causal inference. Boston:

Houghton Mifflin.Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal and structural equation models. Boca

Raton, FL: Chapman & Hall/CRC.Snijders, T.A.B., & Bosker, R.J. (2011). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Thousand Oaks,

CA: Sage.Sobel, M. (2008). Identification of causal parameters in randomized studies with mediating variables. Journal of Educational and Behavioral

Statistics, 33, 230–251.Sörbom, D. (1974). A general method for studying differences in factor means and factor structure between groups. British Journal of

Mathematical and Statistical Psychology, 27, 229–239.Steyer, R., & Eid, M. (2001). Messen und Testen (2. Aufl.). Berlin: Springer.Steyer, R., Gabler, S., von Davier, A. A., Nachtigall, C., & Buhl, T. (2000). Causal regression models I: Individual and average causal effects.

Methods of Psychological Research Online, 5, 39–71.Steyer, R., Mayer, A. & Fiege, C. (in press). Causal inference on total, direct, and indirect effects. In A. C. Michalos (Ed.), Encyclopedia of

quality of life and well-being research. New York: Springer.Stieff, M., Dixon, B. L., Ryu, M., Kumi, B. C., & Hegarty, M. (2014). Strategy training eliminates sex differences in spatial problem solving in

a STEM domain. Journal of Educational Psychology, 106, 390–402.Stuart, E. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 1–21.Subramanian, S. V., & O´Malley, A. J. (2010). Modeling neighborhood effects: The futility of comparing mixed and marginal approaches.

Epidemiology, 21, 475–478.Thoemmes, F., & Kim, E. S. (2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research,

46, 90–118.Thurstone, L. L. (1947). Multiple factor analysis. Chicago, IL: University of Chicago.Tolman, E. C. (1938). The determiners of behavior at a choice point. Psychological Review, 45, 1–35.Tolman, E. C. (1955). Principles of performance. Psychological Review, 62, 315–326.Trautwein, U., & Baeriswyl, F. (2007). Wenn leistungsstarke Klassenkameraden ein Nachteil sind: Referenzgruppeneffekte bei

Übergangsentscheidungen [When high-achieving classmates put students at a disadvantage: Reference group effects at the transition tosecondary schooling]. Zeitschrift für Pädagogische Psychologie, 21, 119–133.

Trautwein, U., Marsh, H. W., Nagengast, B., Lüdtke, O., Nagy, G., & Jonkmann, K. (2012). Probing for the multiplicative term in modernexpectancy–value theory: A latent interaction modeling study. Journal of Educational Psychology, 104, 763–777.

Valeri, L., & VanderWeele, T. J. (2013). Mediation analysis allowing for exposure-mediator interactions and causal interpretation: Theoreticalassumptions and implementation with SAS and SPSS macros. Psychological Methods, 18, 137–150.

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, andrecommendations for organizational research. Organizational Research Methods, 3, 4–70.

van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.VanderWeele, T. J. (2010). Direct and indirect effects for neighbourhood-based clustered and longitudinal data. Sociological Methods & Research,

38, 515–544.VanderWeele, T. J., & Vansteelandt, S. (2009). Conceptual issues concerning mediation, interventions and composition. Statistics and its

Interface, 2, 457–468.Waasdorp, T. E., Bradshaw, C. P., & Duong, J. (2011). The link between parents’ perceptions of the school and their responses to school

bullying: Variation by child characteristics and the forms of victimization. Journal of Educational Psychology, 103, 324–335.Wagner, W., Göllner, R., Helmke, A., Trautwein, U., & Lüdtke, O. (2013). Construct validity of student perceptions of instructional quality is

high, but not perfect: Dimensionality and generalizability of domain-independent assessments. Learning and Instruction, 28, 1–11.

148

Wang, M.-T., Willett, J. B., & Eccles, J. S. (2011). The assessment of school engagement: Examining dimensionality and measurementinvariance by gender and race/ethnicity. Journal of School Psychology, 49, 465–480.

Watt, H. M., Shapka, J. D., Morris, Z. A., Durik, A. M., Keating, D. P., & Eccles, J. S. (2012). Gendered motivational processes affectinghigh school mathematics participation, educational aspirations, and career plans: a comparison of samples from Australia, Canada, and theUnited States. Developmental Psychology, 48, 1594–1611.

What Works Clearinghouse (2008). Evidence standards for reviewing studies (version 1.0). Washington, DC: What Works Clearinghouse.Widaman, K.F., & Reise, S.P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use

domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuseresearch (pp. 281–324). Washington, DC: American Psychological Association.

Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68–81.Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.

149

Part IIFunctional Processes for Learning

150

5Learning as Coordination

Cognitive Psychology and Education

DANIEL L. SCHWARTZ

Stanford University

ROBERT GOLDSTONE

Indiana University It is human nature to create dichotomies—mine versus yours, hot versus cold. Dichotomies usefully structureand simplify the world. They can also lead people astray. Aesop’s fable of The Satyr and the Man captures thisrisk:

A Man was walking in the woods on a very cold night. A Satyr came up to him. The Man raised both hands to his mouth and kept onblowing at them.

“What do you do that for?” asked the Satyr.“My hands are numb with the cold,” said the Man, “and my breath warms them.”Later, the Satyr saw the Man again. The Man had a bowl of steaming soup. The Man raised a spoon of soup to his mouth. He began

blowing upon it.“And what do you do that for?” asked the Satyr.The Man said, “The soup is too hot, and my breath will cool it.”The Satyr shouted, “The Man blows hot and cold with the same breath!”The Satyr ran away. He was afraid the Man was a demon.

Each pole of the dichotomy contains a truth—the man’s breath warmed his hands and cooled his soup. Theproblem is that the satyr treated the categories of hot and cold as mutually exclusive and did not seek a deeperanalysis. Instead, he became agitated and fled the possibility of a unifying explanation.

Education has produced its share of dichotomies: abstract versus concrete, memorizing versusunderstanding, teacher-centered versus student-centered, authentic tasks versus decomposed practice,efficiency versus innovation, and many more. Often the categories of such dichotomies become mutuallyexclusive alternatives, and people advocate for one versus the other. Since at least the behaviorism of B. F.Skinner (1986), scholars have argued whether discovery or entrainment is better for learning (e.g., Kirschner,Sweller, & Clark, 2006; Tobias & Duffy, 2009). The so-called math and reading wars are strong examples ofheated polarization in education.

More prevalent and less extreme than heated debates, people simply accept the definition of one category asthe negation of another, as in the case of active versus passive learning. People do not flee like the satyr, butthey do not seek a deeper analysis either. On deeper analysis, familiar categories of learning, often taken asmutually exclusive, have underlying mechanisms that can make them complementary. So rather than choosingone or the other, the best strategy is to choose both.

151

The chapter follows a central thesis: A major task of teaching and instruction is to help learners coordinatecategories of cognitive processes, capabilities, and representations. While nature confers basic abilities,education synthesizes them to suit the demands of contemporary culture. So, rather than treating categories oflearning and instruction as an either–or problem, the problem is how to coordinate learning processes so theycan do more together than they can alone. This thesis, which proposes a systems level analysis, is not thenorm when thinking about teaching and learning. More common is the belief that learning involvesstrengthening select cognitive processes rather than coordination across processes. Our chapter, therefore,needs to develop the argument for learning as coordination. To do so, we introduce findings from the field ofcognitive psychology.

Cognitive psychology focuses on the mechanisms of mind and brain that determine when and how peoplesolve problems, make decisions, interpret situations, remember, learn, and adapt. There are many reviews ofcognitive psychology as it relates to education (e.g., Koedinger, Booth, & Klahr, 2013; Pashler et al., 2007).There are also cognitively minded books for education (Bransford, Brown, & co*cking, 2000; Mayer, 1987),cognitive psychology textbooks (Anderson, 2000), and excellent free online resources (www.learnlab.org/re‐search/wiki/index.php/Main_Page). These all introduce the central constructs of cognitive psychology,including attention, different forms of memory, expertise, problem-solving strategies, schemas, and more.Many topics originally investigated by cognitive psychology have matured to the point that they now havetheir own chapters in this Handbook and do not need further coverage here (e.g., see Chapters 9 and 15).Therefore, the goal of the present chapter is not to provide an encyclopedic review. Instead, the primary goalis to provide framing and examples for how to view learning from a cognitive perspective that is relevant toquestions of teaching and instruction ranging from reading to math. A second goal is to introduce cognitiveneuroscience, which is increasingly a part of the cognitive psychology tool kit. We show where neurosciencecan complement behavioral analyses.

The first section of the chapter considers the natural human tendency towards categorization with a specialfocus on reconsidering one of the most influential categorical frameworks in education—Bloom’s taxonomy.The next section presents a view of the mind and brain that helps to indicate why mutually exclusivecategories of learning are problematic. The third section presents the heart of the thesis: A major goal ofschool-based instruction is to help learners coordinate different cognitive processes in the service of culturalgoals such as being able to read. The section is populated with examples from research on the teaching andlearning of math and reading. The remaining sections provide two examples of common dichotomies,including memorization versus understanding and concreteness versus abstraction. The examples provide aglimpse into how cognitive processes that putatively occupy the poles of a dichotomy can work in concert.The conclusion considers dichotomania more generally and offers a tentative prescription.

152

http://www.learnlab.org/research/wiki/index.php/Main_Page

Categorical Thinking and Education

Before developing our alternative to dichotomous thinking, it is worth understanding the power of categoriesand boundaries, which make dichotomies possible. Boundaries appear throughout cognition. At the lowestlevels, vision has dedicated neural circuitry that detects the edges that separate one object from another.Rainbows present to us a continuous range of wavelengths, yet we tend to see rainbows as consisting of sevendistinct bands of color. At the highest levels, people intentionally impose boundaries. Political systems dependon fabricated social boundaries that often become physical ones. Creating boundaries is fundamental to thehuman experience (Medin, Lynch, & Solomon, 2000) and reaches from basic perception to culturalorganization.

Categories follow from boundaries; they collect those things that fall within a physical or conceptualboundary. Categories simplify and stabilize an otherwise ever-changing world. The category of “self” appliesduring dinner and when waking up, even though one is quite different at those two time points. Withoutcategories, experience would be a flow of inchoate sensations without organizational structure. Oncecategories are fixed mentally, people de-emphasize differences among members of a category, and accentuatedifferences across categories (Goldstone & Hendrickson, 2010; Harnad, 1997).

Language is an important contributor to category formation (Boroditsky, 2001; Lupyan, 2008). Whenspeaking, it is impossible to convey the totality of experience and all the subtle variations one might beexperiencing right now at this very second. Language fixes the flow of experience into categories. Throughlanguage, people can reflect upon and communicate categories. Lawyers’ carefully worded statements, politicalplatforms, and the movement toward non-sexist and non-discriminatory language are all motivated by therealization that the words we use do not just label our experiences, but also shape and warp these experiences.Being labeled as a member of a category, by a stereotype for instance, can have large effects on how peopleexperience and perform in the world (Steele, 1997). Humans create categories, and categories create humans(McDermott, 1993).

Given their centrality in human thought, categorization schemes can be extremely powerful. An importantgoal of education is to help students learn cultural and scientific categorization schemes (e.g., republics,taxonomies). Categories, even imperfect ones, can advance science. They offer initial hypotheses that driveresearch that may even eventually replace the original categories. On the negative side, once a categorizationscheme is in place, it can be difficult to displace. It took over a thousand years to overhaul the categories ofAristotelian physics with the modern conception of force. People still spontaneously develop Aristoteliancategories to understand physical phenomena, and it takes substantial instruction to displace those naïvemisconceptions (Hestenes, Wells, & Swackhamer, 1992).

Bloom’s taxonomy of educational outcomes (Figure 5.1) provides an example of the strengths andweaknesses of categorization schemes (Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956). On the positiveside, the taxonomy was a brilliant effort to create an assessment framework. It helped educators focus on amore differentiated set of outcomes than the coarse observation that a student “learned.” The taxonomydescribes a pyramid of the following order, going from bottom to top: memory (called “knowledge” backthen), comprehension, application, analysis, synthesis, and evaluation. More recently, some scholars have putin a new top layer, labeled creativity. The pyramid was seminal in pointing out that there are learning

153

outcomes that go beyond the repetition of behavior, which was the prevailing behaviorist perspective at thetime.

On the negative side, many people interpret the categories as forming a prerequisite structure. Studentsmust first learn the lower-order skills at the bottom of the pyramid (memory), before engaging in the skills atthe top of the pyramid (evaluation). This interpretation fuels a back-to-basics mentality, so that studentsshould memorize before trying to apply their learning usefully. However, the science of learning does notsupport this interpretation. For example, comprehension occurs above memory in the taxonomy, but peoplecan remember ideas better when they comprehend them (Bransford & Johnson, 1972). Making memories aprerequisite for comprehension does not work very well. Similarly, having students learn a new topic in anapplication context is a useful way to help them simultaneously learn the facts and evaluate their applications(Barron et al., 1998).

Figure 5.1 Bloom’s taxonomy of cognitive outcomes is a framework for analyzing learning outcomes from 70 years ago (Bloom et al., 1956).Contemporary research does not support the implied ordering that people should learn the bottom of the pyramid before engaging the top ofthe pyramid

Bloom’s taxonomy neatly captures the strengths and weaknesses of categorizations in education. It is acompelling and intuitive categorization scheme, and as such, it has had tremendous influence on practitionersand scientists alike. At the same time, it has been difficult to change, despite 70 years of subsequent researchthat challenges the pyramidal structure. Moreover, people use the categories in ways that violate their intent.Bloom’s taxonomy is an assessment framework for evaluating instructional outcomes. It is not a framework forlearning or designing instruction, but people still use it that way.

154

The Distributed Nature of Cognition

One of the important qualities of cognition is that different categories of thinking comprise distributed andoverlapping subprocesses at another lower level of description (Rumelhart, McClelland, & the PDP ResearchGroup, 1986). For instance, subtraction and multiplication are separate categories of mathematical operation,and each requires its own set of mental steps to compute an answer. It seems safe to say that when people aredoing subtraction, they have “shut off” multiplication. However, at a lower level of analysis, they are engagingmany of the same underlying processes for both types of computation. What appears to be different at onelevel of analysis is not so different at another. We provide an example by introducing brain research that usesfunctional magnetic resonance imaging (fMRI).

Brain cells are alive and therefore always active to some degree. If one simply looked at the activation of thebrain for any category of thought, all the cells would be active. The constant activation of the brain makes foran interesting methodological problem, because it is not possible to say that one cognitive process (set of cells)is on, and another cognitive process is off. To solve this problem, brain research examines relative changes tolevels of activation.

The MRI machine used for brain research is the same machine that doctors can use to collect images of softtissues, such as a torn knee cartilage. For knee injuries, the machine records structural data on the shape anddensity of tissue. When used for fMRI, the scanner can detect changes in blood flow within the brain. Whenpeople complete a task, some of the brain cells do more work than others do. These working cells need to bereplenished with oxygenated blood, and the fMRI picks up the changes in the blood flow. fMRI does notcapture the firing of the neurons when people are completing the task, but rather the increase in blood flowafter the task (about 2 seconds later).

fMRI research depends on comparing the amount of local blood flow for different tasks. When a region ofthe brain receives more blood, scientists infer it has been more active. A study by Lee (2000) demonstrates atypical research strategy. People completed subtraction tasks and multiplication tasks. The fMRI recordedbrain activity during the two tasks. The investigators studied the average activation patterns across the brainfor the subtraction tasks, as well as the activation patterns for the multiplication task. The whole brain isactive for both tasks, but the researcher wanted to find out which brain regions are selectively more active forone task versus the other. To find out, the investigator took the activation patterns for subtraction andremoved the activation patterns in common with multiplication. In other words, the scientist statisticallyremoved all the activation for subtraction that was common with multiplication. The leftover activationindicates which parts of the brain are involved preferentially in subtraction compared to multiplication. Theresearchers then flipped the comparison. They took the brain activity for the multiplication task, and removedactivity that was in common with the subtraction task. Figure 5.2 shows the results. The black regionsindicate areas that are more active for multiplication than subtraction, and the white areas show the areas thatare more active for subtraction than multiplication.

The circled intraparietal sulcus (IPS) region was more active for subtraction. The IPS is also involved invarious spatial attention tasks and judgments about the size of things (Uddin et al., 2010). One interpretationis that people are consulting some form of spatial representation—a mental number line—when doingsubtraction (Dehaene, Piazza, Pinel, & Cohen, 2003). They are making sense of the relative magnitudes of

155

the numbers while completing the subtraction on symbolic digits. In contrast, the area indicated as AG(angular gyrus) is more active for multiplication. This region is also involved in the retrieval of factual, verbalmemories. Thus, people seem to rely on the semantics of quantity (e.g., size and order) for subtraction, andthey appear to rely on rote memory for multiplication, consistent with the idea they are consulting memorizedmultiplication tables.

Figure 5.2 Regions of the brain are selectively active for different mathematics tasks. The figure shows three different slices of the brain. Areasin white are more active for subtraction than multiplication. Areas in black are more active for multiplication than subtraction. The circledregion labeled AG indicates the rough location of the angular gyrus. The circled regions labeled IPS indicate the rough location of theintraparietal sulcus. (Adapted from Lee, K. (2000). Cortical areas differentially involved in multiplication and subtraction: A functionalmagnetic resonance imaging study and correlation with a case of selective acalculia. Annals of Neurology, 48(4), 657–661.)

Given these results, it may be easy to feel the pull of dichotomous thinking. For instance, one might wantto conclude that when people are doing multiplication, their sense of magnitude (the IPS) is shut down. Onemight even go further to make the reckless conclusion that the proper method of multiplication instruction isto emphasize the memorization of verbal math facts without regard for a sense of magnitude. We take a closerlook at IPS activation to see why these are mistaken conclusions.

Cochon, Cohen, van de Moortele, & Dehaene (1999) compared brain activation during multiplication withactivation when staring at a small cross on the screen. Figure 5.3 indicates the IPS is very active formultiplication compared to doing a non-mathematical task. When interpreting this new result, one mightnow conclude that it is important for multiplication facts to be tightly connected with one’s sense ofmagnitude. This is a very different conclusion from a dichotomous interpretation of the results in Figure 5.2.

In summary, cognitive processes are always “on” to some degree. It is a mistake to view them asdichotomous, where one process excludes another. It is tempting to do so, because dichotomous categoriessimplify the world. On closer inspection, however, exclusive categories often hide a deeper truth aboutcognition, much as the satyr’s assumptions about hot and cold hid a deeper truth.

156

Nature Confers Cognitive Processes; Education Coordinates Them

A major goal of typical education is to coordinate evolutionarily conferred abilities into ensembles that canachieve culturally relevant goals. Whereas most everyone learns to speak and interpret language, educationcoordinates our evolutionarily bestowed linguistic capacity with the visual system so that people can also readlanguage. Similarly, education coordinates the IPS, largely implicated in spatial attention, so it can contributeto mathematical thinking. Evolution bestowed humans with the ability to coordinate and re-coordinatecognitive and neural processes.

A nice example comes from a study by Mackey, Miller Singley, and Bunge (2013). The authors comparedbrain changes among students who did or did not take a course that provided training for the Law SchoolAdmission Test (LSAT) exam (the entrance test for law schools). The LSAT is rich in hypothetical thinking,which requires one to set aside the facts that one knows, and instead, draw logical conclusions based on thestated premises in the problem. That is why it is called “hypothetical reasoning.” The effect of the LSATtraining was to coordinate the prefrontal and parietal regions. One interpretation is that the prefrontal regionslearned to suppress spontaneous memory intrusions from the parietal regions, so people would rely on thepremises and logic rather than their memories. Learning to deactivate memory retrieval is useful for doing thetypes of tasks that appear in the LSAT, a cultural invention.

The reader may have entertained the analogy that learning is like strengthening a muscle. A better analogywould be learning to dance. Dancing requires the coordination of many muscles, as well as the strengtheningof the muscles in response to one another. Strengthening without coordination is ineffective. Woltz, Gardner,and Bell (2000), for instance, found that if people already know how to do one set of computation steps verywell, they may display more errors when performing a new, related computation compared to a person whohas less initial experience. Even though the seasoned subjects had strengthened some relevant computation“cognitive muscles,” the coordination was wrong.

157

Figure 5.3 The intraparietal sulcus (IPS) is active for multiplication tasks relative to staring at a fixation cross. (Adapted from Cochon, F.,Cohen, L., van de Moortele, P. F., & Dehaene, S. (1999). Differential contributions of the left and right inferior parietal lobules to numberprocessing. Journal of Cognitive Neuroscience, 11(6), 617–630. Reprinted by permission of MIT Press Journals. Copyright 1999 MassachusettsInstitute of Technology.)

Consider the case of learning to type. One approach might be to have people strike a key faster and fasterwhen they see the relevant letter. For instance, one sees the letter “t” appear on a screen and then types theletter “t” as quickly as possible. It is not hard to imagine a fun little computer game that could train this kindof response. It fits the muscle analogy, where one emphasizes the strengthening of an isolated skill. Typingprograms, however, do not take this approach. Instead of helping people learn how to type each letter asquickly as possible, these programs help people coordinate multiple keystrokes. The bottleneck in typing ishow well people can coordinate their fingers to handle collections of letters (i.e., words). Moreover, peoplealso need to coordinate the movements of their eyes with their hands, if they are typing from a document.They need to look ahead by just the right amount to anticipate how to coordinate their fingers for thetransition from one word to the next. This fits the dance analogy, where one emphasizes the coordination ofactivity. Education is more about teaching the brain to dance than teaching it to lift weights.

Dichotomous thinking brings with it a focus on single cognitive processes, often to the exclusion of others.This can lead to tenacious misconceptions. One major misconception may be the belief in learning styles. Thebelief is that different people have different favored cognitive abilities, and therefore, instruction should matcha person’s favored cognitive ability. To be sure, there are individual differences in some foundationalcapacities. For instance, some people are better at mentally manipulating spatial information than others(Hegarty & Waller, 2005), and there are researchers who work on strengthening these very specific skills(Feng, Spence, & Pratt, 2007). However, this does not support the claim that, therefore, people with highspatial ability should receive instruction spatially, which is the immediate implication of some of the researchon learning styles. Despite a thriving belief in learning styles, their effects must be small, because there issurprisingly little evidence to support the idea that people with different native strengths should receivedifferent types of instruction (Pashler, McDaniel, Rohrer, & Bjork, 2009). When people claim they are visuallearners, they may be claiming that they can interpret spatial information more easily, or perhaps, they aresaying that they do not like to read very much, which is a motivation issue. Regardless, when one thinks oflearning the important content taught in schools, it often depends on the coordination of the linguistic,spatial, conceptual, attention, memory, and other systems.

158

Examples of Learning as Coordination

Learning to coordinate is foundational to the biology of the brain as it adapts to new information. At thecellular level, brain cells “learn” to coordinate their signals with one another. All learning requires coordinationat the cellular level. The neurons need to communicate to accomplish work. Learning comprises an increaseand decrease in the number and strength of connections among neurons, so they can coordinate theircommunication more effectively for specific tasks. Of course, knowing this fact does not get one very far inthinking about the macro-level of learning that teachers handle in classroom instruction. Therefore, in thissection we provide some examples of coordination for the types of tasks and learning found in schools. Theexamples come from reading, mathematics, and conceptual change.

Learning to Read

A crisp example of the role of learning as coordination involves reading. By the time that children are learningto read, they have extensive vocabularies. They can detect words in sound, and they can use these sound-basedwords to retrieve their meanings from memory. Arrow 1 in Figure 5.4 indicates this coordination of hearingand memory. With reading, children now have the challenge of hooking up their visual system to theirauditory system, as indicated by arrow 2. They need to learn that the look of a set of letters (a written word)corresponds to a sound. Establishing this coordination takes time, because the children need to learn how tosee and hear the letters. Over time and with many hundreds of hours of practice, people begin to establishcoordination between sight and meaning. They develop a link directly between the look of a word and itsmeaning, as indicated by arrow 3.

Figure 5.4 Circuits that enable reading. Evolutionarily conferred language circuits map between word sounds and meaning. It requires expliciteducation to coordinate the activity of the visual system so that reading language also becomes possible

The link between vision and word meaning becomes automatic with practice. When seeing a word, it ishard to ignore its meaning, as shown by the Stroop task (Stroop, 1935) in Figure 5.5. Moreover, people canread without having to sound out words, which enables them to read much faster. An interesting fact is thatonce sight and meaning have been coordinated, people do not lose the coordination between sight and sound.For instance, if you run into a word that you do not know immediately, you may notice that you subvocalizethat word—you are sounding it out in your head in the hopes that arrow 1 will help find the meaning, becauseyou cannot find the direct link from sight to meaning. It is informative to note that population variability in

159

the ability to speak language is much lower than the variability found in reading. This is because readingdepends on the special coordinating arrangements of culture and school, whereas speaking and understandingoral language are conferred by nature.

Approximate Addition

The significance of well-coordinated processes also appears in mathematics tasks. Tsang, Dougherty,Deutsch, Wandell, and Ben-Shachar (2009) investigated children’s abilities to do approximate addition. Inapproximate addition, people receive an addition problem and have to choose which of two answers is closerwithout computing the answer exactly. Given 27 + 14, is 40 or 60 closer to the answer? The task is anexperimental version of the standard “estimate the answer” assignment in school. Tsang had childrencomplete numerous problems and found that there were reliable individual differences in children’sperformance.

Figure 5.5 The Stroop effect. The task is to say the color of the word (black), but people automatically read and retrieve the meaning of theword (white), which slows down their time to complete the task of saying black

The researchers then took measures of the brain’s white matter using MRI. The white matter consists offibers or tracts that connect regions of gray matter that reside on the surface of the brain. The gray matter isresponsible for different types of computations, whereas the white matter helps distal brain regionscommunicate. Figure 5.6 shows the brains of two children and the white-matter tract of interest (anteriorsuperior longitudinal fasciculus: aSLF) for the approximate addition task. Children who had a more coherenttract connecting the two areas of the brain were also the ones who did better on the approximate additiontask. The implication is that they were better able to coordinate the computations between different brainregions.

At this fine level of granularity, the coordination of different processes appears as biological, and one canask whether and what types of educational experiences might improve the structure of these specific biologicalpathways. The researchers did not address this question. A likely hypothesis is that the children need toengage in tasks that co-activate and force the coordination of the two areas of gray matter to drive changes inthe connective white matter (see Scholz, Klein, Behrens, & Johansen-Berg, 2009).

Conceptual Change in Mathematics

Conceptual change refers to major shifts in how people think of a situation or problem (see also Chapter 18,this volume). For instance, young children change from a conception of a flat earth to a round one (Vosniadou& Brewer, 1992). Conceptual change in mathematics provides a strong example of learning as coordination.

160

To an adult, the digit “5” coordinates multiple quantitative meanings seamlessly. For instance, 5 can refer tocardinality—five total things. It can refer to ordinality—fifth in a series. It can also refer to magnitude—5 isbigger than 3. Infants, and many animals, have innate abilities for each of these separate meanings of number.They can differentiate between two and three objects at a rapid glance; they can tell whether something comesbefore or after something else; and they can judge larger and smaller. The task of instruction is to coordinatethese different abilities to make an integrated concept of number. For instance, Griffin, Case, and Siegler(1994) created a kindergarten curriculum that involved board games where students had to translate betweenthe different meanings. They might roll a die and count the total number of dots (cardinality). They wouldthen move their character on the game board the same number of spaces forward, thereby translating betweencardinality and ordinality. They might then have to decide who has more total spaces so far, translatingbetween magnitude and ordinality. These researchers found that children who played the coordinating gamesdid better in first grade the following year compared to children who played games that tried to improve eachsense of quantity independently (e.g., just counting dots to find cardinality without translating the results intoordinal position or to make a magnitude comparison).

Figure 5.6 White-matter tracts connect distant surfaces of the brain. These two brains show differences in the anterior superior longitudinalfasciculus (aSLF) white-matter tracts that connect two regions of the brain that coordinate to complete approximate addition tasks. (For visualclarity, the aSLF tracts are shown in black and the many other white-matter tracts of the brain have been removed from the image.) (Courtesyof Dr. Jessica Tsang, based on data collected in Tsang et al., 2009.)

When people learn fundamentally new concepts, they need to re-coordinate the relations betweenevolutionarily old neural circuits. Dehaene and Cohen (2007) proposed that people “exapt” neural circuits forcultural purposes through a process of cortical recycling. Exapt means that a structure originally evolved toserve one function is borrowed to serve another. For example, the visual circuits responsible for finediscrimination of natural phenomena become repurposed to identify symbolic letters. Blair, Tsang, andSchwartz (2013) looked for evidence of borrowing primitive perceptual computations in the context of amathematical conceptual change; namely, learning the integers.

The integers introduce the negative numbers and zero to the natural numbers. The understanding ofnegative numbers is unlikely to have been conferred by nature, given that they are a recent invention (Varma& Schwartz, 2011). The integers also depend on the introduction of new mathematical structure in the formof the additive inverse: X + –X = 0. The authors asked what innate abilities were exapted to handle theadditional structure of the negative numbers. Adults had to decide the mid-point of two digits, for instance, 2and 10 (answer: 6), –6 and 2 (answer: –2). As the digits became more symmetric about zero, people answeredmore quickly. For instance, people could solve –5 and 7 faster than –3 and 9. This was true even if peopleheard the digits rather than seeing them on a screen. Interestingly, they also found that brain regions

161

associated with detecting visual symmetry (e.g., visual area V5) became more active for the more conceptually

symmetric problems. Based on this evidence, it appears that people exapt their abilities to detect symmetry tohelp make sense of the integers, which can be conceptualized as symmetric about zero.

The authors went a step further to determine if this finding had implications for instruction. They created acurriculum for fourth graders that emphasized symmetry (Figure 5.7) so that students could coordinate theirinnate abilities with symmetric structures to understand the negative numbers. They found that thiscurriculum led to superior abilities to solve novel integer problems compared to current instructional models,which do not help students coordinate their knowledge of natural numbers and symmetry to build anunderstanding of integers.

162

Memory and Understanding

We now turn to the discussion of dichotomies that may be familiar to the reader. The poles of thesedichotomies reflect important cognitive processes and outcomes. The risk is that people treat the poles asmutually exclusive and argue for one over the other. We begin with the distinction between memory andunderstanding. These processes need each other. For instance, a common technique in classrooms around theworld is to have students activate their prior knowledge before a lesson. “Do you remember hugging your dog?Did you notice the warmth? That is because a dog is a mammal, and mammals are warm-blooded.” Activatingprior knowledge is an example of retrieving memories to help one understand new ideas better. In this case,understanding depends on memory. A second common instructional technique is to ask people to makesentences out of new words. Making a meaningful sentence with a new word will help people remember theword. In this case, memory depends on understanding. Despite the obvious interdependence of memory andunderstanding, they are often placed into the following exaggerated opposition:

Figure 5.7 Hands-on materials created to emphasize the symmetry of positive and negative materials. Children received integer additionproblems (e.g., 5 + –6) that they modeled by setting out positive and negative blocks about the zero point. To find the answer, they clapped theblocks together, folding up from the zero point. The number of extra blocks on either side gives the answer

Rote memorization ↔ Deep understanding

People need to memorize important recurrent facts. Knowing the fact families in math is a great asset forsolving problems that depend on factoring. Remembering is faster than problem solving, and for manyproblems, speed matters. Being able to remember an answer also frees up cognitive resources useful forunderstanding broader aspects of a problem. Similarly, people need understanding. If one truly understands,then one can recreate what may be forgotten. There are important differences between memorization andunderstanding, but as fits our argument, they work better in coordination than in isolation. We begin with abrief review of the memory literature, and then the literature on understanding. We then consider why thecoordination of memory and understanding is important for the transfer of learning from one setting toanother.

Memory

Memory is one of the most intensely studied and theorized domains within cognitive psychology. How can

163

people gain memories without limit, yet still remember the right memory at the right time and at blazingspeeds? For example, here is a random word—“peach.” You probably recalled the right fruit in about 0.6–0.75seconds. Given how many memories you have about so many different things, it is a stunning achievement.

Humans have many distinct memory systems, each specializing in a different type of information. At anextreme, one can consider the immune system to be a type of memory. When people receive bone marrowtransplants, doctors kill the existing marrow and then replace it. As a result, the immune system “forgets” allthe diseases it has encountered and it needs to relearn. For cognitive phenomena, there are multiple memorysystems, and recent evidence suggests that each requires separate sleep cycles to help consolidate the memoriesof the day (Stickgold, 2005). For instance, given a typing lesson, people will type faster after sleeping on thelesson than they did at the end of the typing lesson. If people’s sleep is interrupted during the specific cycleassociated with this form of procedural memory, they will not perform better in the morning.

Gaining a memory depends on two processes. One is encoding the memory, or “getting it in there.” Theother is retrieval, or “getting it back out.” We consider each briefly.

Encoding involves laying down the initial trace of a memory. Ideally, the way one encodes a memory willimprove the chances of remembering it later, and this is an important emphasis of good instruction. There area number of study techniques for improving memory. One class of strategies relies on the meaning of whatone is trying to learn. For instance, connecting a new idea to a pre-existing idea improves encoding. If you aretrying to learn a new phone number, it helps to find familiar mathematical patterns. Given 422-8888, onemight improve the encoding of the phone number by thinking, “4 divided by 2 makes 2, and adding them upmakes 8 of which there are 4 again.” This works much better than just repeating the phone number, which isa recipe for forgetting as soon as one stops repeating the digits. In general, the depth of processing (Craik &Lockhart, 1972) and the relevance of elaboration (Stein & Bransford, 1979) predict the success of memoryencoding. The more you think about a new idea and relate it to other ideas in meaningful ways, the better thechances of remembering it. It is as if you are laying down lots of neural roads, so it is easier to get back to theidea from other ideas. A second class of general encoding strategy—spaced practice—works regardless of thecontent of what one is learning (Cepeda et al., 2009). If one plans to work on memorizing words for a total of10 minutes, it is better to use five separate sessions of 2 minutes each rather than one big session of 10minutes. Cramming for a test is a bad way to create memories for a lifetime.

The second process of memory is retrieval, which involves bringing the memory back out. Retrieving amemory increases the chances of being able to retrieve it again later. A seminal demonstration comes from the“generation” effect (Slamecka & Graf, 1978). People received word pairs in one of two conditions. In the readcondition, the words were presented completely, for instance, FAST: RAPID. In the generate condition, thewords were presented as FAST: R_P_D. People knew the words had to be synonyms, and they could easilygenerate the missing letters to generate “rapid.” People read or generated very many words. A short time later,they recalled as many of the words as possible. The generate condition remembered more of the words. Onepossible explanation is that the generate task required working a little harder to remember the word “rapid”during the task, which made subsequent retrieval a little easier. The importance of retrieval practice hasresurfaced recently as the testing effect (Karpicke & Blunt, 2011). Taking a test, which requires retrievingmemories, improves the chances of retrieving those memories later, for example, on a future test. Of course,the implication is not necessarily that students should take repeated tests, but rather, they should practice

164

remembering what they know. If one wants to learn using flashcards, it is better to try to remember what is onthe other side of the flashcard than just turning it over to see the answer.

With practice, the neural coordination of memories changes (McClelland, McNaughton, & O’Reilly,1995). An example comes from children solving simple mental addition and subtraction problems.Behaviorally, children are very accurate at all ages, but they answer more quickly as they develop moreexperience with math facts. Figure 5.8 shows changes in brain activity with development (Rivera, Reiss,Eckert, & Menon, 2005). The bottom panel highlights areas that decrease activity. As children gainexperience, they do not rely on the prefrontal areas of the brain as much. Among other things, the prefrontalarea is responsible for the deliberate control of processing. With experience, the children do not need to do asmuch deliberate control to help them put their memories together to come up with the answer. The top panelshows areas of the brain that become more active for the arithmetic tasks as children gain more experience.With experience, these parietal areas become responsible for holding the relevant memories, and children canaccess them directly with little deliberate effort. Tasks that once required flexible but cognitively costlyexecutive control come to be executed by quickly retrieving stored memories (see also Chapter 19, thisvolume).

At the behavioral level, a clear case of memory transformation involves skill acquisition (Anderson, 1982).Acquiring skills involves a transition from declarative to procedural memory. Declarative memory refers tothings you can say, and procedural memory refers to things that you do. Imagine that you are learning tochange lanes while driving. At first, you followed declarative instructions—“check your blind spot, turn onyour blinker, check your blind spot, turn the wheel . . . ” With practice, you no longer needed to rely on theseverbal memories. Instead, you developed procedural memory. You can tell because you do not need to talk toguide yourself through the steps. Instead, you can just execute them. After even more practice, these skillsbecome automated. They require very little cognitive control or attention to execute. For instance, you canchange lanes while also talking to your passenger. Because you do not need to pay attention to the skillexecution, you can pay attention to talking. The way this works is that all the steps become “chunked”together so that one quickly leads to the next, and it requires little cognitive control to coordinate thetransition from one step to another. They become one big step.

165

Figure 5.8 Changes in how the brain solves mental addition problems as children develop. The top panel shows that children increasingly relyon parietal regions to solve addition problems, and the bottom panel shows that they decreasingly rely on executive control from the prefrontalcortex to accomplish the tasks. (Adapted from Rivera, S. M., Reiss, A. L., Eckert, M. A., & Menon, V. (2005). Developmental changes inmental arithmetic: Evidence for increased functional specialization in the left inferior parietal cortex. Cerebral Cortex, 15(11), 1779–1790. Bypermission of Oxford University Press.)

The transition from declarative to procedural memory has been an important guide for the design of manycurricula. One of the most notable involves computerized “cognitive tutors” (Anderson, Corbett, Koedinger,& Pelletier, 1995). These intelligent computer programs track a student’s progress. By monitoring how wellthe learner is performing on various tasks, it can infer whether the child has developed chunked proceduralknowledge. If not, the program can back up to provide the student with relevant practice.

Understanding and Analogy

A common objection to memory-focused models of instruction is that students may learn to recall or execute askill, but they may not understand it. For instance, students who memorize math steps may not reallyunderstand what those math steps mean. But, what counts as “understanding?”

The definition of “understanding” has been a subject of philosophical investigation since at least the time ofSocrates. In cognitive psychology, different investigators choose different ways to operationalizeunderstanding that are most relevant to their topic of study. (The term “operationalize” means that oneindicates which measureable behaviors provide evidence for a given mental state or process.) For instance, aresearcher who studies mathematics learning may operationalize understanding as the ability to verbally justifythe generality of a particular claim (does the operation of addition always make a greater total quantity?). Aperson who studies language acquisition may operationalize understanding as the ability to identify thereferent of a word (given the word dog, point to the right thing). Thus, there is no single definition ofunderstanding. Nevertheless, it has been possible to make important empirical advances.

One major advance has been the distinction between surface features and deep structures. A deep structureis a set of necessary relations that characterize what is the same across many instances (e.g., mammal: warm-blooded, hair). A surface feature is a property that may or may not be important (e.g., red hair). A classic

166

example comes from Chi, Feltovich, and Glaser (1981). They had novices and experts categorize physics

problems. Experts grouped spring and inclined-plane problems together, whereas novices did not. The expertsidentified that the problems shared the deep structure of being about potential energy. To the novices, thesesituations seemed completely different, because one involved springs and one involved inclined planes, whichdo not look alike. In mathematics and science education, helping students learn the deep structure isimportant. A phenomenon’s deep structure is typically what verbal principles and formulas describe.

Oftentimes, people rely on surface features while learning, and this causes them to miss the deep structure.In a telling study, students learned the probability formulas for computing combinations and permutations(Ross, 1987). (As a reminder, imagine pulling two chips from a bag of red and blue chips. There are threepossible combinations: two reds, two blues, or one red and one blue. Permutations further consider thepossible orderings: red → red, blue → blue, red → blue, blue → red.) In the study, the students learned tocompute the number of combinations using marbles as the example, and they learned permutations using carsas the example. On the posttest, students received combination and permutation problems. They used thecombination formula for problems about marbles—regardless of whether the problem called for findingcombinations or permutations of marbles. Similarly, if a problem involved cars, the students used thepermutation formula whether it was appropriate or not. The students had memorized the formulas just fine.The problem was that they relied on the surface features of the problems (e.g., cars or marbles) to decidewhich formula to use. They did not learn to recognize the deep structure of combinations and permutations,which holds up regardless of cars or marbles.

Analogies capitalize on the distinction between deep structure and surface features. Consider theabbreviated test question:

Deluge is to Droplet as:

(a) Landslide is to Pebble(b) Cloudburst is to Puddle

Many people choose (b) as the answer, because it shares the surface feature of being about water. Answer (a)can be construed as a better answer because it shares the same deep structure as the prompt, which might besummarized as “many harmless events can accumulate into a disaster.” Being able to work with the deepstructure of a situation is one useful operationalization of understanding.

Transfer and Induction—Where Understanding and Memory Work Together

Research on transfer highlights the importance of coordinating memory and understanding. Transfer refers tothe use of prior learning in a new situation. Liberal education is predicated on the notion of transfer, becausestudents learn in school, but they need to use this learning outside of school. In contrast, training-orientedinstruction often does not need to consider issues of transfer. The application context typically shares the samesurface and deep features as the original learning conditions. Training airline pilots in a simulator does notraise large transfer challenges, because the simulated co*ckpit is very similar to the co*ckpit of the plane; theyshare the same surface and deep features.

Transfer depends on the coordination of memory and understanding. For instance, in the preceding study,the students remembered the procedures for computing permutations and combinations. This was insufficient

167

Table 5.1

for effective transfer, however. They did not understand the deep structure of situations that call for the use of

one or the other formula. Of course, had the students never memorized the procedures, they would not havehad a formula to transfer either. How can we help students both memorize and understand?

One solution is to rely on inductive learning (Holland, Holyoak, Nisbett, & Thagard, 1986). Inductionrefers to the process by which people use multiple instances to create a new category or rule. (It contrasts withdeduction, where people start with a rule or category, and determine what instances are possible.) Throughinduction, people may find the deep structure that unifies discrete memories and generalizes to new situations.

Discrete memories do not transfer well, because they typically apply to a single situation. For instance,memorizing 3 + 1 = 4 will not help solve 4 + 1. However, if people memorize that 3 + 1 = 4, 4 + 1 = 5, 5 + 1 =6, and so forth, they might induce the rule that “any number plus 1 equals the next number in order.”Induction is an important way that people generalize from the instances they have encountered and go beyondthe information given (Bruner, 1957).

People are always inducing patterns from their memories and experience. However, they may not inducewhat we consider most important. For instance, given the series of +1 problems above, a student mightcorrectly but inappropriately induce, “this teacher really likes to give problems with a 1 in them.” Througheducation, we want people to induce particular patterns that generalize well, not idiosyncratic ones. Learningthrough analogy is a powerful way to help people induce targeted understanding from a set of instances.

A classic study on learning from analogy clarifies the role of induction in coordinating memory andunderstanding for transfer. Gick and Holyoak (1983) tried to determine what would help people solveDuncker’s radiation problem, short of giving them the answer. Here is the problem:

A patient has a tumor that needs to be irradiated. If the doctor uses a beam that is powerful enough to kill the tumor, it will kill healthycells as it passes on the way to the tumor. If the doctor uses a radiation beam that is weak enough that it will not hurt healthy cells, then itwill not kill the tumor. What can the doctor do?The answer: The doctor can use multiple weak beams from different angles that simultaneously converge on the tumor.

To see what would help people solve this problem, the researchers constructed several analogs to theradiation problem. For example, in one analog, a general wanted to attack a fortress and had to split up histroops to converge from different angles so they would not be too heavy for any one bridge. In another,firefighters needed to use multiple bucket brigades to douse a fire. In some cases, the researchers alsodescribed the general principle, “Split up forces to converge on a central target.” Given these elements, theresearchers tried different combinations to see which ones would support transfer to the radiation problem.

The Effects of Induction and Explanation on Transfer

Percent who Solved the Radiation Problem Read Principle Did not Read Principle

Received no analog 28% 18%

One analog 32% 29%

Two analogs 62% 52%

Data from Gick and Holyoak (1983)

College students were randomly assigned to one of several conditions. One factor was the number ofanalogies included in the packet: zero, one, or two of the analogs (fortress and fire problems). A second factorwas whether or not the packet included a statement of the principle. On the last page of all the packets was

168

the radiation problem. Table 5.1 shows the percentage of students who solved the radiation problem at theend of the packet. (Students who received neither the analogs nor the principle received filler materials intheir packet and served as the control condition.)

The most notable findings involve the first column. Students who were told the correct principle withoutreceiving any examples did not transfer very well. One interpretation of this result might be that the studentsdid not understand the principle without an example. However, students who received one analog (anexample) plus the statement of the principle did not do much better. Why would a single example with aprinciple be ineffective for transfer? The principle indicated the deep structure of the problem, and thestudents had an example to help make sense of the principle. It cannot simply be that the students did notknow the deep structure.

The students who received two examples (analogs) did much better, with or without a statement of theprinciple. (The students who received the two examples without the principle were often able to induce theprinciple from the two analogs, so they did not need to read the principle.) One possible reason that the singleanalog and the principle did not work very well is that students did not learn the range of variation that mightappear for this particular principle. For example, those students who only learned about the story of attackingthe fortress from multiple bridges, even with the statement of the principle, had no way of knowing that it canapply to lots of situations. The surface features, while incidental to the deep structure, are still important fortransfer. Remembering variability of the surface features allows people to appreciate that the deep structurethey understand can apply to many situations. Thus, memory of several instances and understanding worktogether, and one without the other does not work very well for transfer.

The value of coordinating memory and understanding for transfer yields some simple instructionalprescriptions. Loewenstein, Thompson, and Gentner (2003) describe a study where they found that askingbusiness students to find the analogous structure between case studies led to superior learning compared to acondition where students handled each case separately without looking for the common structure. Providingtwo analogous examples works well for transfer, but students need encouragement to induce the commondeep structure that unifies otherwise discrete examples (Schwartz, Chase, Oppezzo, & Chin, 2011).

Despite the simplicity of helping students induce the deep structure across instances, there is aninstructional tendency to use single examples plus a statement of the rule. Felder and Silverman (1988) notedthat almost all engineering professors claim to use deductive instruction methods when teaching others—going from general rule statements to specific instantiations—even though they often themselves use inductivelearning methods, proceeding from particulars to generalities. Additionally, people may neglect the potentialof using analogies to support induction. In a review that compared instruction cross-nationally, Richland, Zur,and Holyoak (2007) found that U.S. teachers tended not to capitalize on the use of analogy compared toteachers in Hong Kong and Japan. Perhaps, by understanding how memory and understanding coordinate,educators will take more advantage of analogical induction.

169

Concrete and Abstract

A long-standing distinction, at least since the time of Plato, is the dichotomy between concrete and abstractmental representations. The idea is that concrete thinking is tied to the perceptual-motor particulars of asituation, whereas abstract operations rise above immediate experience to use logical relations and hypotheticalthinking. The dichotomy is relevant to instruction, because people often favor one over the other mode ofoperation. For example, in California, the state’s science curriculum commission proposed legislation thatwould limit hands-on learning to “no more than 20 to 25 percent” of instructional time. This resulted in anoutcry from educators and business people, and the final legislation reversed the proposal to “at least 20 to 25percent” of science instruction using hands-on material (2004, www.cascience.org/csta/leg_criteria.asp).

A similar distinction between perception and abstraction occurs in intuitive frameworks for thinking aboutthe brain: The brain is an information-processing system that takes in information from the senses andtransforms it in different ways along a series of processing stages. The early stages of information processingare “low-level.” For vision, these early stages would include extracting edges from a scene, creating contoursfor objects, determining stable colors not influenced by lighting conditions, and segregating objects from theirbackgrounds. “High-level” stages are further “downstream” in the flow of information processing, executedonly after a considerable amount of sensory and perceptual processing has completed. High-level processeswould involve cognitive actions such as inferring likely career choices of a friend, deciding where to search foran answer, and creating a new diagram for representing relations among cognitive functions.

The intuitive separation of low-level and high-level operations fuels many of the dichotomies found ineducation, such as Bloom’s taxonomy. One key dichotomy might be represented as follows:

Perception ↔ Conception

As with all dichotomies, there is some truth to it. The brain is not a hom*ogeneous lump. It is organizedinto spatially distinct modules with specialized functions. Some of the best-articulated modules are thosedealing with perception, and these perceptual modules are often the first to be activated in response to astimulus. In contrast, concepts do not need to be stimulus-driven. One can bring to mind the concept of “dog”without seeing or hearing a dog. Moreover, concepts have abstract and logical relations to other concepts,such as “not a cat.”

Despite the important distinctions between concepts and percepts, they are not dichotomous. Grossanatomical considerations indicate a high degree of coordination between low-level perceptual processes andhigh-level conceptual processes. First, there are strong down-stream connections from perception toconception, so that stimulation of the perceptual system gives rise to relevant concepts. Second, there are alsoreciprocal up-stream connections from conceptual levels of analysis to the perceptual system (Lamme &Roelfsema, 2000). The tight integration is manifest when considering the time course of identifying an objectsuch as a dog’s tail. As stimulus information arrives from the senses, populations of neurons complete low-level analysis, such as edge detection, to extract important perceptual features. At 120 milliseconds, theseneural populations operate similarly, regardless of whether one is explicitly paying attention to the object, andwhether one has knowledge that there is a dog attached to the object. In this time window, the neurons areengaged in pre-attentive, unconscious perceptual processing. Yet, at 160 milliseconds, the same neural

170

http://www.cascience.org/csta/leg_criteria.asp)

populations will respond to high-level information about where to pay attention and the knowledge of thecontext of the object (it is a dog). It is this latter activity that people consciously experience—“it is the tail of adog” (Fahrenfort, Scholte, & Lamme, 2007). Thus, if one wants to know whether a neuron is “strictlyperceptual,” the answer depends on when one asks the question as much as it depends on which neuron isinvolved.

These broad considerations of neural architecture echo in the functional behavior of people. People adjustthe processing of lower-level regions so that they are better adapted to the needs of higher-level cognition.Our perceptual systems are surprisingly adaptive, even late in life, and they typically adapt so that we canperform high-demand tasks more effectively (Fahle & Poggio, 2002).

The renowned philosopher Quine (1977) argued that advanced scientific thought must dispense withnotions of perceptual similarity as the basis for its categories (see also Chapter 1, this volume). The argumentseems plausible at first sight, and it has some similarity to the idea that understanding depends on findingdeep relational structures rather than relying on surface features. Our perceptual systems might mislead us intobelieving that samples of fool’s gold (pyrite) are true gold. Better to rely on the periodic table and the physicalchemistry of the elements. There is a good deal of appeal to this argument. However, if one used thepossibility of error as a reason to discard perception from scientific thinking, one would also have to throw outconception, because people’s concepts are frequently wrong as well (e.g., McCloskey & Kohl, 1983). It is thepossibility of error that creates the possibility of learning and discovery.

Fittingly, people can learn to perceive. Learning is not confined to abstract matters such as F = ma.Perception can be educated and augmented so it complements conceptual thinking. People can look for subtleproperties that distinguish fool’s gold from the real thing. People can supplement their biological perceptualapparatus with tools such as microscopes, hardness scales, and quantitative measurements of malleability.

Education and experience change the processing of the perceptual system (Goldstone, 1998). Some of thesechanges can occur in perceptual areas relatively early in the brain’s information-processing stream. Forinstance, consider expert perception. Researchers measured the electrophysiological activity of dog and birdexperts while they looked at pictures of dogs and birds (Gauthier, Tarr, & Bubb, 2010). For dog experts,enhanced electrical activity occurred 164 milliseconds after the presentation of dog, but not for a bird.Reciprocally, bird experts showed quick activation for bird pictures but not for photographs of dogs. This is animpressively fast processing effect of expertise given that transmitting a simple electrical signal from one endof a neuron to the other requires about 10 milliseconds. For brain evidence of experience-driven changes toperception, Furmanski, Schluppeck, and Engel (2004) used fMRI to measure brain activity before and after 1month of practice with detecting hard-to-see lines. Practice increased responses in the primary visual cortex(area V1) and the degree of change correlated with detection performance. Bao, Yang, Rios, and Engel (2010)found changes in electrical activity as fast as 50–70 milliseconds after stimulus onset. Perception, even at thefirst stages of information uptake, can be educated. In fact, there is evidence that auditory training canproduce differential responses in sensory receptors, such as the cochlea (Puel, Bonfils, & Pujol 1988), asensory organ just inside of the eardrum. Perceptual changes are found at many different neural loci and ageneral rule seems to be that brain regions associated with early perceptual analysis are implicated in finer,more detailed, and generally less transferable knowledge (Ahissar & Hochstein, 1997).

Even learning abstract topics such as algebra can be improved by harnessing perceptual learning in

171

instruction. A nice example comes from Kellman, Massey, and Son’s (2010) perceptual learning modules. Thealgebra learning modules have high feedback and minimal explicit instruction. They try to develop students’sensitivity at noticing preserved structures in equations across algebraic transformations. For example, studentsare given trials on which they must determine that 6y – 17 = 32 – 5x is a valid transformation of 6y + 5x – 17 =32, but that neither 6y – 17 = 32 + 5x nor 6y – 17 = 32 – x – 5 are. Although this kind of training might seemlike “mere symbol pushing,” the argument from perceptual learning is that by training students to see contrastsbetween valid and invalid algebraic transformations, they come to naturally perceive or induce the underlyingstructure of algebra.

Lawrence Barsalou has presented a particularly influential account of the grounding of conception inperception in the form of perceptual symbols theory (Barsalou, 1999). By this account, conceptual knowledgeinvolves activating brain areas dedicated for perceptual processing. When a concept is brought to mind,sensorimotor areas of the brain are reactivated. Even abstract concepts, such as truth and negation, may begrounded in complex perceptual simulations of combined physical and introspective events. Interestingly,Barsalou’s research shows that when people engage in perceptual simulations, their understandings of aconcept are likely to be richer and more flexible compared to when they do not. Reasoning based onperception is the “smart,” not “stupid” stuff. This result is echoed by studies showing that students who showgreater mathematical competence are more, not less, likely to engage in perceptual solutions to algebraic tasks(Goldstone, Landy, & Son, 2010). For example, students who exhibit relatively good mastery of mathematicsare more likely to solve problems such as x – 2 = 7, by imagining the 2 moving from the left side of theequation to the right side, turning into a +2 as it does so. Rather than viewing perceptual processes asantagonistic to proper formal thought, it is precisely by properly executing these perceptual processes thatformally sanctioned reasoning is achieved effectively.

A major challenge of school-based instruction is helping students coordinate the abstract, symbolicrepresentations of culture with the perceptual world of experience. Glenberg, Gutierrez, Japuntich, andKaschak (2004), for instance, noted that young readers often do not construct a mental model of what they arereading, but instead, they are just saying the words aloud. To help, the researchers had young childrenmanipulate figurines to correspond with each sentence they read (e.g., “the man went into the barn”). Theythen told the children they should do this in their head when reading. This improved reading comprehensionlater, even when the children no longer manipulated figurines physically. An important challenge for aneducationally relevant cognitive psychology is to develop new theories and evidence that helps guide freshinstructional efforts to coordinate perception, action, and conception (Goldin-Meadow & Beilock, 2010).Simply juxtaposing a concrete and abstract representation may not be sufficient for people to learn tocoordinate their perceptual-motor abilities with their symbolic ones.

172

Dichotomania

We have sampled a pair of familiar dichotomies. There are others. For instance, a common dichotomy is thedistinction between passive versus active learning, which appears in the college instruction literature (e.g.,Prince, 2004). Passive learning largely refers to sitting in a large lecture listening to a professor’s exposition,whereas active learning refers to being engaged in problem solving during class. Dichotomies with familyresemblances include learning by doing versus being told, as well as discovery learning versus directinstruction. The intuition that students can learn more effectively when they are experientially engaged, or atleast not being crushed by tedious exposition, is worthwhile. At the same time, a tremendous amount oflearning occurs through reading and hearing explanations, for example through mass media, the internet, andbooks. Experience and explanation each has its place. So again, the task is how to coordinate these differenttypes of learning. Experiential activities can provide direct engagement of a phenomenon or problem, whereaslectures and readings can provide explanations of those experiences in ways that students are unlikely todiscover on their own. On this model, one way to coordinate active and passive learning is to use activeexperience to create a time for telling (Schwartz & Bransford, 1998). For instance, Arena and Schwartz(2014) had students play a modified version of the arcade game Space Invaders that prepared them to thenlearn a formal treatment on statistical distributions. By itself, the game showed little direct benefit forlearning, but when combined with a formal exposition, students learned more from the exposition thanotherwise equivalent students who had not played the game.

What can we do about all these dichotomies? It may be useful to notice that many of the dichotomies makeone of the poles of the dichotomy something construable as “true understanding.” Rote memorization wascontrasted with conceptual understanding. Attention to surface features was contrasted with attention to deepprinciples. Low-level perception was contrasted with high-level abstract reasoning. It is a recurrent motif tocontrast the upper reaches of human thought with the lower capabilities shared by animals.

Overcoming dichotomania requires a more humble mindset. First, as we have proposed, what separateshumans from animals is the ability to coordinate cognitive processes in concert with cultural demands andopportunities. Through this process of coordination, both the “bottom” and the “top” of cognition refashionone another. Humans are adaptive, and this should be the emphasis of our thinking about learning, not howone type of thinking is superior to another.

Second, it is important to appreciate that even true understandings are always partial and fragmentary. Anoteworthy attitude shared by many accomplished scholars is their insistence on how much they, and we, donot yet understand. The dichotomous endpoint of “true understanding” is illusory, and a realization of thismay yield a less disparaging attitude toward the purported opposite pole. True and complete understandingcertainly has its attractions over more brittle and biologically constrained forms of intelligence, but the latterhave the distinct advantage of actually existing.

As part of a more humble attitude towards posing dichotomies, one also needs an attitude towardsbecoming more knowledgeable. The acceptance of dichotomies presupposes fixed poles, when they may notbe fixed but rather grow with respect to one another. Creating dichotomous categories may be an importantfirst step in making intellectual advances; it is native to human thought (Smith & Sera, 1992). Nevertheless,one should avoid becoming a satyr and running away from opportunities to grow beyond the opposition.

173

In an analogous case, Carol Dweck (2012) has observed that people differ in their implicit views—theirmindsets—about the origins of human ability. Some people with a “fixed” theory believe that ability is largelyinnate. In contrast, those with a “growth” theory believe that ability results from hard work. A “fixed” mindsethas an analogous structure to dichotomania. A fixed mindset presupposes there are poles of “smart” and “not-so-smart” people, and there is no path from one to another. For dichotomania, one may feel that there aremutually exclusive cognitive processes, some being better than others, and with no bridge between.

Just as people who adopt a “growth” theory are more likely to achieve actual success, so our understandingof learning may be more successful if we adopt a growth theory. Such a perspective does not focus on the widegap between the endpoints of putative dichotomies, but rather considers how different processes can be placedinto productive relations. The point is to reflect not only on our lofty positions as intelligences capable ofinfinite flexibility, but also on how we can get to that point using finite means. By this account, properlyharnessed and coordinated memory, perception, action, habit formation, and attention processes can growinto a well-organized system that we take as showing improved educational outcomes.

174

Acknowledgment

The writing of this chapter was supported by the National Science Foundation under grant nos. SMA-0835854, EHR-0910218, as well as the Institute of Education Sciences, U.S, Department of EducationR305A1100060. Any opinions, findings, and conclusions or recommendations expressed in this material arethose of the authors and do not necessarily reflect the views of the granting agencies.

175

References

Ahissar, M., & Hochstein, S. (1997). Task difficulty and the specificity of perceptual learning. Nature, 387, 401–406.Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369–406.Anderson, J. R. (2000). Cognitive psychology and its implications (5th ed.). New York: Worth.Anderson, J. R., Corbett, A. T., Koedinger, K., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of Learning Sciences, 4, 167–

207.Arena, D. A., & Schwartz, D. L. (2014). Experience and explanation: Using videogames to prepare students for formal instruction in statistics.

Journal of Science Education and Technology, 23, 538–548.Bao, M., Yang, L., Rios, C., and Engel, S. A. (2010). Perceptual learning increases the strength of the earliest signals in visual cortex. Journal of

Neuroscience, 30, 15080–15084.Barron, B. J., Schwartz, D. L., Vye, N. J., Moore, A., Petrosino, A., Zech, L., Bransford, J. D., & CTGV. (1998). Doing with understanding:

Lessons from research on problem- and project-based learning. Journal of the Learning Sciences, 7, 271–312.Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577–660.Blair, K. P., Tsang, J. M., & Schwartz, D. L. (2013). The bundling hypothesis: How perception and culture give rise to abstract mathematical

concepts in individuals. In S. Vosniadou (Ed.), International handbook of research on conceptual change II (pp. 322–340). New York: Taylor &Francis.

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: Handbook I: Cognitivedomain. New York: David McKay.

Boroditsky, L. (2001). Does language shape thought? Mandarin and English speakers’ conceptions of time. Cognitive Psychology, 43, 1–22.Bransford, J. D., Brown, A. L., & co*cking, R. R. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National

Academy Press.Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall.

Journal of Verbal Learning & Verbal Behavior, 11, 717–726.Bruner, J. S. (1957). Going beyond the information given. In H. Gruber, G. Terrell, & M. Wertheimer (Eds.), Contemporary approaches to

cognition (pp. 258–290). Cambridge, MA: Harvard University Press.Cepeda, N. J., Coburn, N., Rohrer, D., Wixted, J. T., Mozer, M. C., & Pashler, H. (2009). Optimizing distributed practice. Experimental

Psychology, 56(4), 236–246.Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive

Science, 5, 121–152.Cochon, F., Cohen, L, van de Moortele, P. F., & Dehaene, S. (1999). Differential contributions of the left and right inferior parietal lobules to

number processing. Journal of Cognitive Neuroscience, 11(6), 617–630.Craik, F. I., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior,

11(6), 671–684.Dehaene, S., & Cohen, L. (2007). Cultural recycling of cortical maps. Neuron, 56(2), 384–398.Dehaene, S., Piazza, M., Pinel, P., & Cohen, L. (2003). Three parietal circuits for number processing. Cognitive Neuropsychology, 20(3/4/5/6),

487–506.Dweck, C. S. (2012). Mindset: How you can fulfill your potential. London: Constable & Robinson.Fahle, M., & Poggio, T. A. (2002). Perceptual learning. Cambridge, MA: MIT Press.Fahrenfort, J., Scholte, H., & Lamme, V. (2007). Masking disrupts reentrant processing in human visual cortex. Journal of Cognitive

Neuroscience, 19(9), 1488–1497.Felder, R. M., & Silverman, L. K. (1988). Learning and teaching styles in engineering education. Engineering Education, 78, 674–681.Feng, J., Spence, I., & Pratt, J. (2007). Playing an action video game reduces gender differences in spatial cognition. Psychological Science, 18(10),

850–855.Furmanski, C. S., Schluppeck, D., and Engel, S. A. (2004). Learning strengthens the response of primary visual cortex to simple patterns.

Current Biology, 14, 573–578.Gauthier, I., Tarr, M. J., and Bubb, D. (Eds.) (2010). Perceptual expertise: Bridging brain and behavior. Oxford: Oxford University Press.Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15, 1–38.Glenberg, A. M., Gutierrez, T., Levin, J. R., Japuntich, S., & Kaschak, M. P. (2004). Activity and imagined activity can enhance young

children’s reading comprehension. Journal of Educational Psychology, 96, 424–436.Goldin-Meadow, S., & Beilock, S. L. (2010). Action’s influence on thought: The case of gesture. Perspectives on Psychological Science, 5, 664–

674.

176

Goldstone, R. L. (1998). Perceptual learning. Annual Review of Psychology, 49, 585–612.Goldstone, R. L., & Hendrickson, A. T. (2010). Categorical perception. Interdisciplinary Reviews: Cognitive Science, 1, 65–78.Goldstone, R. L., Landy, D. H., & Son, J. Y. (2010). The education of perception. Topics in Cognitive Science, 2, 265–284.Griffin, S. A., Case, R., & Siegler, R. S. (1994). Rightstart: Providing the central conceptual prerequisites for first formal learning of arithmetic

to students at risk for school failure. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 25–49).Cambridge, MA: MIT Press.

Harnad, S. (1997). Categorical perception. Cambridge: Cambridge University Press.Hegarty, M., & Waller, D. (2005). Individual differences in spatial abilities. The Cambridge handbook of visuospatial thinking (pp. 121–169).

New York: Cambridge University Press.Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141–158.Holland, J. H., Holyoak, K. J., Nisbett, R. E., & Thagard, P. (1986). Induction: Processes of inference, learning, and discovery. Cambridge, MA:

MIT Press.Karpicke, J. D. & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborate studying with concept mapping. Science, 331,

772–775.Kellman, P. J., Massey, C. M., & Son, J. (2010). Perceptual learning modules in mathematics: Enhancing students’ pattern recognition,

structure extraction, and fluency. Topics in Cognitive Science, 2, 285–305.Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of

constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.Koedinger, K., Booth, J., & Klahr, D. (2013). Instructional complexity and the science to constrain it. Science, 342 (6161), 935–937.Lamme, V. A. F., & Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends in

Neuroscience, 23, 571–579.Lee, K. (2000). Cortical areas differentially involved in multiplication and subtraction: A functional magnetic resonance imaging study and

correlation with a case of selective acalculia. Annals of Neurology, 48(4), 657–661.Loewenstein, J., Thompson, L., & Gentner, D. (2003). Analogical learning in negotiation teams: Comparing cases promotes learning and

transfer. Academy of Management Learning and Education, 2(2), 119–127.Lupyan, G. (2008). From chair to “Chair”: a representational shift account of object labeling effects on memory. Journal of Experimental

Psychology: General, 137, 348–369.Mackey, A. P., Miller Singley, A. T., & Bunge, S. A. (2013). Intensive reasoning training alters patterns of brain connectivity at rest. Journal of

Neuroscience, 33(11), 4796–4803.Mayer, R. E. (1987). Educational psychology: A cognitive approach. Boston, MA: Little, Brown.McClelland, J. L., McNaughton, B. L., & O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and

neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3), 419.McCloskey, M., & Kohl, D. (1983). Naive physics: The curvilinear impetus principle and its role in interactions with moving objects. Journal of

Experimental Psychology: Learning, Memory, and Cognition, 9(1), 146.McDermott, R. (1993). Acquisition of a child by a Learning Disability. In S. Chaiklin & J. Lave (Eds.), Understanding practice (pp. 269–305).

London: Cambridge University Press.Medin, D. L., Lynch, E. B., & Solomon, K. O. (2000). Are there kinds of concepts? Annual Review of Psychology, 51, 121–147.Pashler, H., Bain, P., Bottge, B., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing instruction and study to improve

student learning (NCER 2007-2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, and U.S.Department of Education.

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2009). Learning styles. Psychological Science in the Public Interest, 9, 105–119.Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231.Puel, J. L., Bonfils, P., & Pujol, R. (1988). Selective attention modifies the active micromechanical properties of the cochlea. Brain Research,

447, 380–383.Quine, W. V. (1977). Natural kinds. In S. P. Schwartz (Ed.), Naming, necessity, and natural kinds (pp. 155–175). Ithaca, NY: Cornell University

Press.Richland, L. E., Zur, O., & Holyoak, K. J. (2007). Cognitive supports for analogies in the mathematics classroom. Science, 316, 1128–1129.Rivera, S. M., Reiss, A. L., Eckert, M. A., & Menon, V. (2005). Developmental changes in mental arithmetic: Evidence for increased

specialization in the left inferior parietal cortex. Cerebral Cortex, 15(5), 1779–1790.Ross, B. H. (1987). This is like that: The use of earlier problems and the separation of similarity effects. Journal of Experimental Psychology:

Learning, Memory, and Cognition, 13, 629–639.Rumelhart, D. E., McClelland, J. L., & the PDP Research Group (Eds.) (1986). Parallel distributed computing: Explorations in the microstructure

177

of cognition. Volume 1: Foundations. Cambridge, MA: MIT Press.Scholz, J., Klein, M. C., Behrens, T. E., & Johansen-Berg, H. (2009). Training induces changes in white-matter architecture. Nature

Neuroscience, 12(11), 1370–1371.Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition & Instruction, 16, 475–522.Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling

first on learning and transfer. Journal of Educational Psychology, 103(4), 759–775.Skinner, B. F. (1986). Programmed instruction revisited. Phi Delta Kappan, 68, 103–110.Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning

and Memory, 6, 592–604.Smith, L. B., & Sera, M. D. (1992). A developmental analysis of the polar structure of dimensions. Cognitive Psychology, 24, 99–142.Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52(6), 613–629.Stein, B. S., & Bransford, J. D. (1979). Constraints on effective elaboration: Effects of precision and subject generation. Journal of Verbal

Learning and Verbal Behavior, 18(6), 769–777.Stickgold, R. (2005). Sleep dependent memory consolidation. Nature, 437, 1272–1278.Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662.Tobias, S., & Duffy, T. (Eds.) (2009). Constructivist instruction: Success or failure. New York: Routledge, Taylor & Francis.Tsang, J. M., Dougherty, R. F., Deutsch, G. K., Wandell, B. A., & Ben-Shachar, M. (2009). Frontoparietal white matter diffusion properties

predict mental arithmetic skills in children. Proceedings of the National Academy of Science, 106, 22546–22551.Uddin, L. Q., Supekar, K., Amin, H., Rykhlevskaia, E., Nguyen, D. A., Greicius, M. D., & Menon, V. (2010). Dissociable connectivity within

human angular gyrus and intraparietal sulcus: Evidence from functional and structural connectivity. Cerebral Cortex, 20(11), 2636–2646.Varma, S., & Schwartz, D. L. (2011). The mental representation of integers: An abstract-to-concrete shift in the understanding of

mathematical concepts. Cognition, 121, 363–385.Vosniadou, S., & Brewer, W. F. (1992). Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology, 24, 535–

585.Woltz, D. J., Gardner, M. K., & Bell, B. G. (2000). Negative transfer errors in sequential cognitive skills: Strong-but-wrong sequence

application. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(3), 601.

178

6Emotions and Emotion Regulation in Academic Settings

MONIQUE BOEKAERTS

Leiden University, The Netherlands

REINHARD PEKRUN

University of Munich, The Netherlands The classroom is an emotional place. Students frequently experience emotions such as enjoyment of learning,hope for success, pride in accomplishments, anger about task demands, fear of failing an exam, or boredom inacademic settings. Research has shown that both traditional classroom instruction and advanced technology-based learning environments can induce a great variety of such emotions (D’Mello, 2013; Pekrun, Goetz,Titz, & Perry, 2002). Furthermore, the available evidence implies that these emotions are instrumental forachievement and personal growth. Experiencing positive emotions can help a student envision goals, promotecreative problem solving, and support self-regulation (Clore & Huntsinger, 2009; Fredrickson, 2001). On theother hand, experiencing excessive negative emotions about studying and taking exams can impede academicperformance, prompt school dropout, and negatively influence health (Zeidner, 1998, 2014). The far-reachingconsequences of emotional experiences are also likely reflected in the tragic numbers of suicides related toschool or college each year (Westefeld et al., 2005).

Despite the clear relevance of emotions for education and the dramatic increase of attention to emotion inother scientific disciplines, for a long time emotions have been neglected by educational psychology.Exceptions were studies on test anxiety (Zeidner, 1998) and on the role of causal attributions for achievementemotions (Weiner, 1985). Since the beginning of the 2000s, however, there has been growing recognition thatemotions are central to academic achievement strivings as well as students’ and teachers’ personalitydevelopment. Emotions are no longer viewed as incidental phenomena lacking in function or purpose. Rather,in this nascent research, emotions are recognized as being of critical importance for both students’ andteachers’ productivity (see Pekrun & Linnenbrink-Garcia, 2014; Schutz & Pekrun, 2007).

In this chapter, we consider such emotions. While the principles of emotion discussed in the chapterpertain to both students and teachers, the focus is on students’ emotions (for teacher emotions, see Schutz &Zembylas, 2009). To begin, we define emotion and outline concepts of academic emotions. Next, we reviewresearch on the functions and origins of academic emotions. This review highlights the importance ofemotions for students’ learning. In the fourth section, we address emotion regulation (ER) and the role ofemotional intelligence. In conclusion, implications for practice and suggestions for future research arediscussed.

179

180

Concepts of Academic Emotions

Emotions are typically defined as multifaceted phenomena involving sets of coordinated psychologicalprocesses, including affective, cognitive, physiological, motivational, and expressive components (Shuman &Scherer, 2014). For example, a student’s anxiety before an exam can be comprised of nervous, uneasy feelings(affective); worries about failing (cognitive); increased heart rate (physiological); impulses to escape thesituation (motivation); and an anxious facial expression (expressive). Emotion is part of the superordinatecategory of affect that is used for a variety of states, including emotion, moods, and metacognitive feelings. Incomparison to intense emotions, moods are less intense and lack a specific object of reference. Metacognitivefeelings such as feeling of knowing and feeling of confidence are judgments about one’s progress in learning(Efklides, 2006).

Valence, Activation, and Object Focus

Emotions can be grouped according to their valence and to their degree of activation (Table 6.1). In terms ofvalence, positive emotions can be distinguished from negative emotions, such as pleasant enjoyment versusunpleasant anxiety. In terms of activation, physiologically activating emotions can be distinguished fromdeactivating emotions, such as activating excitement versus deactivating relaxation. These two dimensions areused to arrange affective states in a two-dimensional space (“circumplex models” of affect; Barrett & Russell,1998).

As addressed in Pekrun’s (2006; Pekrun et al., 2002) three-dimensional taxonomy of emotions, anotherimportant dimension of emotions is their object focus (Table 6.1). In terms of object focus, the followinggroups of academic emotions can be distinguished.

Achievement emotions. These emotions are tied to achievement activities (e.g., studying) or achievementoutcomes (success and failure), resulting in two groups of achievement emotions: activity emotions andoutcome emotions. Activity emotions include the ongoing emotions students experience while engaging in anachievement activity. Outcome emotions include both prospective emotions related to upcoming success andfailure, and retrospective emotions related to past success and failure. Most emotions pertaining to studying,attending class, and taking tests or exams are considered achievement emotions, because they relate toactivities and outcomes that are judged according to competence-based standards of quality.

Epistemic emotions. Epistemic emotions pertain to the knowledge-generating qualities of cognitive tasks andactivities. These emotions are triggered when students are engaged in novel, non-routine tasks, such asproblem solving and research projects, and are promoted by unexpected information and cognitiveincongruity. Examples are surprise, curiosity, excitement, confusion, wonder, frustration at unsolved problems,and joy of confirmation.

Topic emotions. Topic emotions border on epistemic emotions, yet they are different in the sense that they donot refer to the comprehension process per se, but to the appealing effect that the learning material can have.Topic emotions allude to the themes that are dealt with in the learning material. For example, when the topic

181

Table 6.1

is the Middle Ages, some students may enjoy the stories of the fighting knights and sympathize with theircause, whereas others may feel disgusted by the blood-thirsty stories.

A Three-Dimensional Taxonomy of Academic Achievement Emotions

Positivea Negativeb

Object Focus Activating Deactivating Activating Deactivating

Activity Enjoyment Relaxation Anger Boredom

Outcome Joy Contentment Anxiety Sadness

Hope Relief Shame Hopelessness

Gratitude Pride Anger Disappointment

a Positive = pleasant emotion.b Negative = unpleasant emotion.Adapted from Pekrun (2006).

Social emotions. Educational psychologists have long focused on individual performance rather than oncollaborative learning. This narrow research focus may explain why social emotions in the classroom havebeen neglected. With the advent of social constructivist theories of learning, social emotions have beenupgraded. Social emotions are particularly important in teacher–student and student–student interaction.Examples are love, sympathy, empathy, gratitude, compassion, anger, and social anxiety. Social andachievement emotions overlap in emotions that relate to the success and failure of others, such as admiration,envy, and contempt.

182

Functions for Learning and Performance

In experimental research, moods and emotions have been found to influence a range of cognitive processesthat are relevant to academic learning, such as attention, memory storage and retrieval, and problem solving(Lewis, Haviland-Jones, & Feldman Barrett, 2008). Much of this research, however, has focused on theeffects of positive versus negative mood without drawing distinctions between specific, discrete mood statesand emotions. This implies that it may be difficult and potentially misleading to use the findings to explainstudents’ emotions and learning in real-world academic contexts. Specifically, as argued in Pekrun’s (2006)cognitive/motivational model of emotion effects, it is not sufficient to differentiate positive from negativeaffective states, but imperative to also attend to the degree of activation implied. As such, the minimumnecessary is to distinguish between four groups of emotions: positive activating, positive deactivating, negativeactivating, and negative deactivating (Table 6.1). For example, both anxiety and hopelessness are negativeemotions; however, their effects on students’ engagement can differ dramatically, as anxiety can motivate astudent to invest effort in order to avoid failure, whereas hopelessness likely undermines any kind ofengagement.

In the following sections, we first summarize research on the relation of emotions to different cognitive andmotivational processes that are relevant to learning. We then outline implications for the effects of differentemotions on students’ academic achievement.

Attention and Flow

Research has shown that both positive and negative emotional states consume attentional resources byfocusing attention on the object of emotion (Ellis & Ashbrook, 1988). Consumption of attentional resourcesimplies that fewer resources are available for task completion, thereby negatively impacting performance(Meinhardt & Pekrun, 2003). For example, while preparing for an exam, a student may worry about failure,which in turn may distract her attention away from the task. However, the resource consumption effect likelyis bound to emotions that have task-extraneous objects and produce task-irrelevant thinking. By contrast, inpositive task-related emotions such as curiosity and enjoyment of learning, the task is the object of emotion.In these emotions, attention is focused on the task, working-memory resources can be used for taskcompletion, and experiences of flow are promoted. Corroborating these expectations, empirical studies withK-12 and university students found that negative emotions such as anger, anxiety, shame, boredom, andhopelessness were associated with task-irrelevant thinking and reduced flow, whereas enjoyment relatednegatively to irrelevant thinking and positively to flow (e.g., Pekrun, Goetz, Frenzel, Barchfeld, & Perry,2011; Zeidner, 1998). These findings suggest that students’ emotions have profound effects on theirattentional engagement with academic tasks.

Motivation to Learn

Emotions prepare us for doing something. For example, negative emotions such as fear are warning signalsthat arise in response to events that can have negative consequences. Each of the major negative emotions isthought to be associated with distinct action impulses and serves to prepare the organism for action (or non-action), such as fight, flight, and behavioral passivity in anger, anxiety, and hopelessness, respectively. For

183

positive emotions, motivational consequences are less specific. Likely, one of the functions of positiveemotions such as joy and interest is to motivate exploratory behavior and an enlargement of one’s actionrepertoire, as addressed in Fredrickson’s (2001) broaden-and-build metaphor of positive emotions. Byimplication, emotions can profoundly influence students’ motivational engagement. Supporting this view,positive academic emotions such as enjoyment of learning, hope, and pride have been shown to relatepositively to students’ interest and intrinsic motivation, whereas negative emotions such as anger, anxiety,shame, hopelessness, and boredom related negatively to these motivational variables (Helmke, 1993; Pekrunet al., 2002, 2011; Zeidner, 1998, 2014).

However, as addressed in Pekrun’s (2006) cognitive/motivational model of emotion effects, motivationaleffects may be different for different types of positive and negative emotions. The model posits that activatingpositive emotions (e.g., joy, hope, and pride) strengthen motivation, whereas deactivating negative emotions(e.g., hopelessness, boredom) undermine motivation (Pekrun, Goetz, Daniels, Stupinsky, & Perry, 2010). Bycontrast, effects are posited to be more complex for deactivating positive emotions (e.g., relief, relaxation) andactivating negative emotions (e.g., anger, anxiety). For example, relaxed contentment following success can beexpected to reduce immediate motivation to re-engage with learning contents but strengthen long-termmotivation to do so. Regarding activating negative emotions, anger, anxiety, and shame have been found toreduce intrinsic motivation, but these emotions can strengthen extrinsic motivation to invest effort in order toavoid failure, especially so when expectations to prevent failure and attain success are favorable (Turner &Schallert, 2001). Due to these variable effects, the impact of these emotions on students’ overall motivation tolearn can be variable as well.

Memory Processes

Emotions influence storage and retrieval of information. Two effects that are especially important for theacademic context are mood-congruent memory recall and retrieval-induced forgetting and facilitation. Mood-congruent recall (Parrott & Spackman, 2000) implies that mood facilitates the retrieval of like-valencedmaterial, with positive mood facilitating the retrieval of positive self- and task-related information, andnegative mood facilitating the retrieval of negative information. Mood-congruent recall can impact students’motivation. For example, positive mood can foster positive self-appraisals and thus benefit motivation tolearn; by contrast, negative mood can promote negative-self appraisals and thus hamper motivation.

Retrieval-induced forgetting implies that practicing some learning material impedes later retrieval of relatedmaterial that was not practiced, presumably so because of inhibitory processes in memory networks. Suchforgetting occurs with learning material consisting of disconnected elements. For example, after learning a listof foreign-language words, practicing half of the list can impede students’ memory for the other half. Bycontrast, retrieval-induced facilitation implies that practicing enhances memory for material that was notpracticed (Chan, McDermott, & Roediger, 2006). Facilitation has been found to occur for connectedmaterials consisting of elements that show strong interrelations. For example, after learning coherent textmaterial, practicing half of the material leads to better memory for the non-practiced half.

Negative emotions can undo retrieval-induced forgetting, likely because they can inhibit spreadingactivation in memory networks which underlie such forgetting. Conversely, positive emotions can promoteretrieval-induced facilitation since they promote the relational processing of information underlying such

184

facilitation (Kuhbandner & Pekrun, 2013). This suggests that negative emotions can be helpful for learninglists of unrelated material, such as lists of foreign-language vocabulary, whereas positive emotions shouldpromote learning of coherent material.

Problem Solving, Learning Strategies, and Self-Regulation of Learning

Mood has been shown to influence cognitive problem solving, with positive mood promoting flexible andcreative ways of solving problems, and negative mood promoting more rigid, detail-oriented, and analyticalways of thinking (Clore & Huntsinger, 2009; Fredrickson, 2001). Mood-as-information approaches (Clore &Huntsinger, 2009) explain this finding by assuming that positive affective states signal that “all is well,”implying safety and the discretion to engage in creative exploration, broaden one’s cognitive horizon, andbuild new actions. By contrast, negative states are thought to indicate that something is going wrong, makingit necessary to focus on problems in more cautious, analytical ways.

Judging from the experimental evidence, positive activating emotions such as enjoyment of learning shouldfacilitate use of flexible, holistic learning strategies like elaboration and organization of learning material orcritical thinking. Negative emotions, on the other hand, should sustain more rigid, detail-oriented learning,like simple rehearsal of learning material. Correlational evidence from studies with university studentsgenerally supports this view (Pekrun et al., 2002, 2011). However, for deactivating positive and negativeemotions, these effects may be less pronounced. Deactivating emotions, like relaxation or boredom, mayproduce shallow information processing rather than any more intensive use of strategies (Pekrun et al., 2010).

Furthermore, given that self-regulation of learning requires cognitive flexibility, positive emotions canfoster self-regulation, whereas negative emotions can motivate the individual to rely on external guidance.Correlational evidence is generally in line with these propositions (Linnenbrink & Pintrich, 2002; Pekrun etal., 2002, 2010, 2011). However, the reverse causal direction may also play a role in producing suchcorrelations—self-regulated learning may instigate enjoyment, and external directions for learning may triggeranxiety.

Academic Achievement

Since many different cognitive and motivational mechanisms can contribute to the functional effects ofemotions, the overall effects on students’ academic achievement are inevitably complex and may depend onthe interplay between different mechanisms. Nevertheless, it seems possible to derive inferences from theexisting evidence.

Positive emotions. Traditionally it was assumed that positive emotions, notwithstanding their potential tofoster creativity, are often maladaptive for achievement due to inducing unrealistically positive appraisals,fostering non-analytical information processing, and making effort expenditure seem unnecessary by signalingthat everything is going well. From this perspective, “our primary goal is to feel good, and feeling good makesus lazy thinkers who are oblivious to potentially useful negative information and unresponsive to meaningfulvariations in information and situation” (Aspinwall, 1998, p. 7).

However, as noted, positive mood has typically been regarded as a unitary construct in experimentalresearch. As argued earlier, such a view is inadequate because it fails to distinguish between activating versus

185

deactivating moods and emotions. As detailed in Pekrun’s (2006) cognitive/motivational model, deactivatingpositive emotions, like relief or relaxation, may well have the negative performance effects described forpositive mood, whereas activating positive emotions, such as task-related enjoyment, should have positiveeffects. The evidence cited above suggests that activating enjoyment focuses attention on the task; inducesintrinsic motivation; promotes relational memory processing; and facilitates use of flexible learning strategiesand self-regulation, thus likely exerting positive effects on overall performance under many task conditions. Bycontrast, deactivating positive emotions, such as relief and relaxation, can reduce task attention; can havevariable motivational effects; and can lead to superficial information processing, thus likely making effects onoverall achievement more variable.

The available evidence supports the view that activating positive emotions can enhance achievement.Specifically, enjoyment of learning was found to correlate moderately positively with K-12 and collegestudents’ academic performance (Helmke, 1993; Pekrun et al., 2002, 2011). Furthermore, students’enjoyment, hope, and pride correlated positively with college students’ interest, effort invested in studying,elaboration of learning material, and self-regulation of learning, in line with the view that these activatingpositive emotions can be beneficial for students’ academic agency. Consistent with evidence on discreteemotions, general positive affect has also been found to correlate positively with students’ cognitiveengagement (Linnenbrink, 2007). However, some studies have found null relations between activatingpositive emotions (or affect) and achievement (Linnenbrink, 2007; Pekrun, Elliot, & Maier, 2009). Also,caution should be exercised in interpreting the reported correlations. Linkages between emotions andachievement are likely due not only to performance effects of emotions, but also to effects of performanceattainment on emotions, implying reciprocal rather than unidirectional causation (Pekrun, Hall, Goetz, &Perry, 2014).

Negative activating emotions. Emotions such as anger, anxiety, and shame produce task-irrelevant thinking,thus reducing cognitive resources available for task purposes, and they undermine students’ intrinsicmotivation. On the other hand, these emotions can induce motivation to avoid failure and facilitate the use ofmore rigid learning strategies. By implication, the effects on resulting academic performance depend on taskconditions and may well be variable, similar to the proposed effects of positive deactivating emotions. Theavailable evidence supports this position.

Specifically, it has been shown that anxiety impairs performance on complex or difficult tasks that demandcognitive resources, such as difficult intelligence test items, whereas performance on easy and less complextasks may not suffer or is even enhanced (Zeidner, 1998). In line with experimental findings, field studies haveshown that test anxiety correlates moderately negatively with students’ performance. Typically, 5–10% of thevariance in students’ achievement scores is explained by self-reported anxiety (Hembree, 1988; Zeidner,1998). Again, in explaining the correlational evidence, reciprocal causation of emotion and performance has tobe considered. Linkages between test anxiety and achievement may be caused by effects of success and failureon the development of test anxiety, in addition to effects of anxiety on achievement. The scarce longitudinalevidence available suggests that test anxiety and students’ achievement are in fact linked by reciprocalcausation across school years (Meece, Wigfield, & Eccles, 1990; Pekrun, 1992). Furthermore, zero andpositive correlations have sometimes been found, in line with our view that anxiety can exert ambiguous

186

effects. Anxiety likely has deleterious effects in many students, but it may facilitate overall performance inthose who are more resilient and can productively use the motivational energy provided by anxiety.

Similar to anxiety, shame related to failure showed negative overall correlations with college students’academic achievement and negatively predicted their exam performance (Pekrun et al., 2002, 2011). However,as with anxiety, shame likely exerts variable effects (Turner & Schallert, 2001). Similarly, while achievement-related anger correlated negatively with academic performance in a few studies (Boekaerts, 1993; Pekrun etal., 2011), the underlying mechanisms may imply more than just negative effects. In a study by Lane, Whyte,Terry, and Nevill (2005), depressed mood interacted with anger occurring before an academic exam, such thatanger was related to improved performance in students who reported no depressive mood symptoms—presumably because they were able to maintain motivation and invest effort. In sum, the findings for anxiety,shame, and anger support the notion that performance effects of negative activating emotions are complex,although relationships with overall performance are negative for many task conditions and students.

Negative deactivating emotions. In contrast to negative activating emotions, negative deactivating emotions,such as boredom and hopelessness, are posited to uniformly impair performance by reducing cognitiveresources, undermining both intrinsic and extrinsic motivation, and promoting superficial informationprocessing (Pekrun, 2006). The little evidence available corroborates that boredom and hopelessness relateuniformly negatively to students’ achievement, in line with theoretical expectations (Ahmed, van der Werf,Kuyper, & Minnaert, 2013; Pekrun et al., 2010, 2011, 2014).

In sum, theoretical expectations, the evidence produced by experimental studies, and findings from fieldstudies imply that students’ emotions have profound effects on their engagement and academic achievement.As such, administrators and educators should pay attention to the emotions experienced by students. Mostlikely, the effects of students’ enjoyment of learning are beneficial, whereas hopelessness and boredom aredetrimental for engagement. The effects of emotions like anger, anxiety, or shame are more complex, but forthe average student, these emotions also have negative overall effects.

187

Origins and Development of Emotions

Given that emotions affect students’ learning and achievement, it is important to acquire knowledge abouttheir antecedents. The emotions that students experience in the classroom are affected by a multitude offactors, including genetic dispositions, gender, early socialization, cognitive appraisals, achievement goals,personality traits (e.g., temperament, achievement motives), and the learning environment (Pekrun &Linnenbrink-Garcia, 2014). Herein, we focus on cognitive appraisals and academic environments asantecedents and summarize evidence on the development of students’ emotions over the school years.

Cognitive Appraisals

Test anxiety. In research on test anxiety, appraisals concerning threat of failure have been addressed ascausing anxiety. Using R. S. Lazarus’s transactional stress model (Lazarus & Folkman, 1984) to explain testanxiety, threat in a given achievement setting is evaluated in a primary appraisal related to the likelihood andsubjective importance of failure. If failure is appraised as possible and subjectively important, ways to copewith the situation are evaluated in a secondary appraisal. A student may experience anxiety when his primaryappraisal indicates that failure on an important test is likely, and when his secondary appraisal indicates thatthis threat is not sufficiently controllable. Empirical research confirms that test anxiety is closely related toperceived lack of control over performance. Specifically, numerous studies have shown that K-12 andpostsecondary students’ self-concept of ability, self-efficacy expectations, and academic control beliefscorrelate negatively with their test anxiety (Zeidner, 1998, 2014).

Attributional theory. Extending the perspective beyond test anxiety, B. Weiner (1985; Graham & Taylor,2014) proposed an attributional approach to the appraisal antecedents of emotions related to success andfailure. In Weiner’s theory, causal achievement attributions—explanations about the causes of success andfailure (e.g., ability, effort, task difficulty, luck)—are considered primary determinants of these emotions.More specifically, it is assumed that achievement outcomes are first subjectively evaluated as success or failure.This outcome appraisal immediately leads to cognitively less elaborated, “attribution-independent” emotions,namely, happiness following success, and frustration and sadness following failure. Following the outcomeappraisal and immediate emotional reaction, causal ascriptions are sought that lead to differentiated,attribution-dependent emotions.

Three dimensions of causal attributions are assumed to play key roles in determining attribution-dependentemotions: the perceived locus of causality differentiating internal versus external causes of achievement (e.g.,ability and effort vs. environmental circ*mstances or chance); the perceived controllability of causes (e.g.,subjectively controllable effort vs. uncontrollable ability); and the perceived stability of causes (e.g., stableability vs. unstable chance). Weiner posits that pride should be experienced when success is attributed tointernal causes (e.g., effort or ability); that shame should be experienced when failure is attributed touncontrollable, internal causes (e.g., lack of ability); and that gratitude and anger should be experienced whensuccess or failure, respectively, is attributed to external, other-controlled causes. Consistent with theretrospective nature of causal attributions for success and failure, Weiner’s theory focuses primarily onretrospective emotions following success and failure. However, some predictions for prospective, future-

188

related emotions are also put forward. Specifically, hopefulness and hopelessness are expected to beexperienced when past success and failure are attributed to stable causes (e.g., stable ability). Empiricalresearch has generally supported these propositions (Graham & Taylor, 2014).

Control-value theory. While test anxiety theories and attributional theories have addressed emotionspertaining to success and failure outcomes, they have neglected activity-related achievement emotions. InPekrun’s (2006; Pekrun & Perry, 2014) control-value theory of achievement emotions, propositions of thetransactional stress model (Lazarus & Folkman, 1984), expectancy-value approaches (Pekrun, 1992; Turner &Schaller, 2001), and attributional theories are expanded to explain a broader variety of achievement emotions,including both outcome emotions and activity emotions. The theory posits that achievement emotions areinduced when an individual feels in control of, or out of control of, activities and outcomes that aresubjectively important—implying that appraisals of control (i.e., perceived controllability) and value (i.e.,perceived importance) are the proximal determinants of these emotions.

Different kinds of control and value appraisals are posited to instigate different kinds of achievementemotions (Table 6.1). Prospective, anticipatory joy and hopelessness are expected to be triggered when there ishigh perceived control (joy) or a complete lack of perceived control (hopelessness). For example, a studentwho believes she has the necessary resources to get an A on an important exam may feel joyous about theprospect of seeing this grade becoming reality. Conversely, a student who believes he is incapable ofpreventing failure on a final exam may experience hopelessness. Prospective hope and anxiety are instigatedwhen there is uncertainty about control, the attentional focus being on anticipated success in the case of hopeand on anticipated failure in the case of anxiety. For example, a student who is unsure about being able tosucceed may hope for success, fear failure, or both. Retrospective joy and sadness are considered control-independent emotions that immediately follow success and failure (in line with Weiner’s, 1985, propositions).Disappointment and relief are thought to depend on the perceived match between expectations and the actualoutcome: disappointment arises when anticipated success does not occur, and relief when anticipated failuredoes not occur. Finally, pride, shame, gratitude, and anger are assumed to be instigated by causal attributions ofsuccess and failure to oneself or others, respectively.

Furthermore, the control-value theory proposes that these outcome-related emotions also depend on thesubjective importance of achievement outcomes, implying that they are a joint function of perceived controland value. For instance, a student should feel worried if she judges herself incapable of preparing for an exam(low controllability) that is important (high value). By contrast, if she feels that she is able to preparesuccessfully (high controllability) or is indifferent about the exam (low value), her anxiety should be low.

Regarding activity emotions, enjoyment of achievement activities is proposed to depend on a combination ofpositive competence appraisals and positive appraisals of the intrinsic value of the action (e.g., studying) andits reference object (e.g., learning material). For example, a student is expected to enjoy learning if she feelscompetent to meet the demands of the learning task and values the learning material. If she feels incompetent,or is uninterested in the material, studying is not enjoyable. Anger and frustration are aroused when theintrinsic value of the activity is negative (e.g., when working on a difficult project is perceived as taking toomuch effort which is experienced as aversive). Finally, boredom is experienced when the activity lacks anyintrinsic incentive value (Pekrun et al., 2010).

189

Empirical studies have confirmed that perceived control over achievement relates positively to achievement-related enjoyment, hope, and pride, and negatively to anger, anxiety, shame, hopelessness, and boredom, inuniversity students (e.g., Pekrun et al., 2002, 2011; Turner & Schallert, 2001) and middle- and high-schoolstudents (e.g., Buff, 2014). Similar links have been observed for emotions in online learning environments(Daniels & Stupnisky, 2012). Furthermore, several of these studies have shown that the perceived value ofachievement related positively to both positive and negative achievement emotions except boredom, indicatingthat the importance of success and failure amplifies these emotions. For boredom, negative links withperceived value have been found, suggesting that boredom is reduced when individuals value achievement(Pekrun et al., 2010). Finally, recent research has confirmed that control and value interact in the arousal ofachievement emotions, with positive emotions being especially pronounced when both control and value arehigh, and negative emotions being pronounced when value is high but control is lacking (e.g., Goetz, Frenzel,Stoeger, & Hall, 2010).

The Influence of Academic Environments

The impact of learning environments on students’ emotions is largely unexplored, except research on theantecedents of test anxiety (Zeidner, 1998, 2014). Classroom composition, the design of classroom instructionand exams, as well as goal structures, expectancies, and reactions in students’ social environments have beenfound to play a significant role in students’ anxiety.

Classroom composition. The ability level of the classroom determines the likelihood of performing wellrelative to one’s classmates. All things being equal, chances for performing well in the classroom are higher inlow-ability classrooms, thus students’ self-concepts of ability tend to be higher in low-ability classrooms. Byimplication, it may be preferable to be a “big fish in a little pond” rather than being a member of a classroomof gifted students (Marsh, 1987). Since negative self-evaluations of competence can trigger anxiety of failure,the “big-fish-little-pond” effect of classroom ability level on self-concept can prompt similar effects onstudents’ anxiety. In fact, students’ test anxiety has been found to be higher in high-ability classrooms than inlow-ability classrooms (e.g., Preckel, Zeidner, Goetz, & Schleyer, 2008).

Classroom instruction and exams. Lack of structure and clarity in classroom instruction as well as excessivetask demands are associated with students’ elevated test anxiety (Zeidner, 1998, 2014). These links are likelymediated by students’ expectancies of low control and failure (Pekrun, 1992). This also applies to exams,whereby lack of structure and transparency has been shown to contribute to students’ anxiety (e.g., lack ofinformation regarding demands, materials, and grading practices). Furthermore, the format of test items hasbeen found to be relevant (Zeidner, 1998). For instance, open-ended formats (e.g., essay questions) inducemore anxiety than multiple-choice formats, because open-ended formats require more attentional resources.As noted, these resources may be compromised as a result of anxiety-induced worrying, resulting in moreexperienced threat and debilitating performance in anxious students. The use of multiple-choice formats canreduce these effects. In addition, practices such as permitting students to choose between test items, relaxingtime constraints, and giving second chances in terms of opportunities to retake a test have been found toreduce test anxiety, presumably so because perceived control and achievement expectancies are enhancedunder these conditions.

190

Beyond anxiety, a handful of studies have investigated relationships between classroom instruction andstudents’ positive emotions. For example, teacher-centered instruction that emphasizes rigid drilling andexercise has been found to relate negatively to students’ positive emotional attitudes toward school andenjoyment of task accomplishment (e.g., Valeski & Stipek, 2001). In contrast, the cognitive quality ofinstruction in terms of structure and clarity, and tasks oriented towards creative mental modeling as opposedto algorithmic routine procedures, have been found to correlate positively with students’ enjoyment of learningmathematics (Pekrun et al., 2007). In addition, support for students’ learning-related autonomy correlatedpositively with students’ enjoyment in this study. Finally, teachers’ own enjoyment and enthusiasm duringteaching have been found to relate positively to students’ enjoyment, suggesting transmission of positiveemotions from teachers to students (Frenzel, Goetz, Lüdtke, Pekrun, & Sutton, 2009).

Goal structures and social expectations. Different standards for defining achievement can imply individualistic(mastery), competitive (normative), or cooperative goal structures in the classroom (Johnson & Johnson,1974). The goal structures provided in achievement settings conceivably influence emotions in two ways.First, to the extent that students adopt these structures, they influence individual achievement goals and anyemotions mediated by these goals (e.g., Roeser, Midgley, & Urdan, 1996). Second, goal structures determinerelative opportunities for experiencing success and perceiving control, thus influencing control-dependentemotions. Specifically, competitive goal structures imply that some students have to experience failure, thusinducing negative outcome emotions such as anxiety in these students. In line with this reasoning, empiricalresearch has shown that competition in classrooms is positively related to students’ test anxiety (Zeidner,1998). Similarly, teachers’ and parents’ excessively high expectations for achievement can reduce students’sense of control, thus also contributing to negative emotions such as anxiety, shame, and hopelessness(Pekrun, 1992).

By contrast, a cooperative classroom climate and social support provided by parents and teachers often failto correlate with students’ test anxiety scores (Hembree, 1988). This surprising lack of correlation may resultfrom well-meaning teachers and parents whose attempts to support students actually increase pressure toperform, thus counteracting any beneficial effects of support. Alternatively, there may be negative-feedbackloops between support and anxiety, with social support alleviating anxiety (negative effect of support onanxiety), but anxiety provoking support in the first place (positive effect of anxiety on demanding support),thus negating any correlation between the two variables.

Feedback and consequences of achievement. Cumulative success can strengthen perceived control, andcumulative failure can undermine control. In environments involving frequent assessments, performancefeedback is likely of primary importance for the arousal of achievement emotions (Pekrun, Cusack,Murayama, Elliot, & Thomas, 2014). In addition, the perceived consequences of success and failure areimportant, because these consequences affect the value of achievement outcomes. Positive outcome emotions(e.g., hope for success) can be increased if success produces beneficial long-term outcomes (e.g., acceptance toan esteemed university), provided there is sufficient contingency between one’s own efforts, success, and theseoutcomes. Negative consequences of failure (e.g., unemployment), on the other hand, may increaseachievement-related anxiety and hopelessness (Pekrun, 1992).

191

Development Across the School Years

Emotions related to achievement evolve early and show continuous development across the life span. Between2–3 years of age, children are able to express pride and shame when successfully solving tasks or failing to doso, suggesting that they are able to differentiate internal versus external causation of success and failure.During the early elementary school years, children additionally acquire capabilities to distinguish betweendifferent types of internal and external causes, such as ability and effort, to develop related causal expectancies,and to cognitively combine expectancies, attributions, and value-related information (Heckhausen, 1991). Byimplication, students have developed the cognitive competencies to experience all major types of academicemotions early in their educational career.

Empirical evidence on the development of these emotions in school is scarce. Again, test anxiety studies arean exception. These studies have shown that average scores for test anxiety are low at the beginning ofelementary school, but increase dramatically during the elementary-school years (Hembree, 1988). Thisdevelopment is congruent with the decline in academic self-concepts and intrinsic motivation during thisperiod, and is likely due to increasing realism in academic self-perceptions and to the cumulative failurefeedback students may receive across the school years. After elementary school, average anxiety scores stabilizeand remain at high levels throughout middle school, high school, and college. However, stability at the grouplevel notwithstanding, anxiety can change in individual students, for example, when there are transitionsbetween schools and classrooms (Zeidner, 1998).

Whereas anxiety increases in the average student, positive emotions such as enjoyment of learning seem todecrease across the elementary-school years (Helmke, 1993). The decrease of enjoyment can continue throughthe middle-school years (Pekrun et al., 2007), which is consistent with the decline of average scores for subjectmatter interest and general attitudes toward school (e.g., Fredricks & Eccles, 2002). Important factorsresponsible for this development may be an increase of teacher-centered instruction and academic demands inmiddle school, competition between academic and non-academic interests in adolescence, and the strongerselectivity of subject matter interest that is part of adolescent identity formation.

192

Emotion Regulation and Emotional Intelligence

ER involves goal-directed processes aiming to influence the intensity, duration, and type of emotionexperienced (Jacobs & Gross, 2014). It allows the individual to respond in flexible ways to situationaldemands while taking account of short-term and long-term goals and concerns. Boekaerts (2011) defined ERin the context of the classroom as students’ capacity to use their emotions as a source of energy, yet modifyaspects of the emotional experience when it interferes with the pursuit of important goals. She emphasizedthat the inability to temper the intensity and duration of one’s emotional arousal in the classroom not onlyhinders learning but social functioning as well. Having access to adequate ER strategies helps students to feelself-efficacious and view the learning process as constructive, and the classroom environment as supportive. Assuch, ER is based on individual competencies to manage and use one’s emotions. “Emotional intelligence” is asummary term often used to denote these competencies. In this section, we address ER, the link between ERand self-regulated learning, and emotional intelligence.

Emotion Regulation

Research has indicated that emotions can be regulated in a variety of ways. We will use the distinction ofproblem-focused versus emotion-focused coping as well as Gross’s process model of ER to describe differentstrategies to regulate emotions.

Emotion-focused versus problem-focused coping. Coping researchers (e.g., Lazarus & Folkman, 1984) havedistinguished between problem-focused strategies which involve attempts to alter a stressor, and emotion-focused strategies which attempt to directly change the emotion. Examples of problem-focused coping ineducational settings include situation selection, working hard, and seeking help; examples of emotion-focusedcoping include reappraisal, self-talk, using humor, distraction, relaxation, wishful thinking, self-blame, actingup, distancing, suppression, withdrawal, and self-handicapping (Boekaerts & Röder, 1999).

Boekaerts (1999) reported that students made use of both problem-oriented and emotion-oriented copingin response to interpersonal stressors such as being bullied, whereas they primarily selected problem-focusedstrategies in relation to academic stressors. Compas, Malcarne, and Fondacaro (1988) reported that adolescentgirls used more emotion-focused strategies in response to academic stressors than boys. They showed thatboth problem-focused and emotion-focused coping strategies reduced negative emotions, provided that therewas a match between the students’ perception of control over the situation and the selected coping strategy.Perception of control matched with problem-focused strategies, and perception of low control with emotion-focused strategies. This finding suggests that students need to pick up cues informing them whether anacademic task or an interpersonal conflict is controllable or not. Next, they should select a strategy from theircoping repertoire that is compatible with the degree of control they experience. Pekrun et al. (2002) confirmedthat both coping strategies may successfully diminish emotional stress. Students’ anxiety during an importantexam was positively related to cortisol levels indicating stress-related activation; problem-focused as well asemotion-focused coping during the exam reduced the level of cortisol.

Selecting a coping strategy is not a one-shot process (Boekaerts, 2011). In stressful situations, students mayexperience an urgent need to feel better right away, even though they may also have the intention to resolve

193

the problem. This goal ambivalence will be reflected in the succession of coping strategies that are used. Forexample, Boekaerts (1999) reported that students who experienced intense stress in relation to interpersonalstressors, such as being called names by peers, used a double-focused coping strategy. The students realizedthat it is important to use problem-focused coping (e.g., confronting the aggressor), yet they felt a desire toopt out of the situation to protect their ego (walking away, mental distraction). In other words, problem-oriented and emotion-oriented regulatory goals competed for dominance in the coping process.

Gross’s process model of emotion regulation. Gross (Gross and John, 2002; Jacobs & Gross, 2014) proposed aprocess model of ER that highlights the time course of regulation. The model is based on a sequence ofprocesses that are involved in an emotion episode (situation, attention, appraisal, and response: Figure 6.1).Each of these processes is a potential target for ER. Gross identified five families of ER strategies, namelysituation selection, situation modification, attentional deployment, cognitive change, and responsemodulation. The first four of these categories apply before the individual has appraised the situation, hencebefore the full-blown emotional response occurs, which can make these strategies especially adaptive. Hereinwe address strategies that are particularly relevant for academic settings, including situation selection andmodification, cognitive change, and response modulation (for an extension of Gross’s model also consideringcompetence-oriented strategies, see Pekrun, 2006).

Figure 6.1 Gross’s process model of emotion regulation

Situation selection. Situation selection involves choosing situations that minimize the risk to be confrontedwith an emotional stressor (e.g., failure, threat, loss) and maximize the chances to strengthen one’s positiveemotions. In the classroom, students may not always be able to select learning situations. Yet, if theyunderstand what the antecedents of their emotions are, they may try to organize their learning in such a waythat they have the best chances for mastery and protect their ego at the same time. For example, if teachersallow their students to decide on different types of instruction (e.g., teacher-regulated learning vs. self-regulated learning) or different task assignment (e.g., easy versus difficult exercises), students can makechoices that promote their adaptive emotions.

Situation modification. This group of strategies involves efforts to intervene in the physical, social, andinstructional environment so as to change its emotional impact. Situation modification is an importantstrategy in the classroom because it may allow students to solicit social and instructional support. For example,a student may ask his teacher to change the seating arrangement so that he sits next to a friendlier peer, or to

194

supply extra exercises that will allow him to practice deficient skills.

Students need metacognitive, metaemotional, and metainterpersonal knowledge to select and modifysituations (Boekaerts, 2011). Lack of such knowledge is one barrier to effective selection and modification.Another barrier is weighing short-term benefits of situation selection versus longer-term costs. For example,students may feel better if they can postpone a difficult situation (e.g., preparing for a test) and abandonstrategies that take effort, and may be ignorant that such short-term benefits come at the cost of failing anexam.

Cognitive change. Cognitive change refers to modifying one’s appraisal of the situation so that it changes itsemotional impact (reappraisal). For example, Gross and John (2002) described how athletes and musicalperformers interpret their physiological arousal prior to going on stage. Some performers interpreted thearousal as stage fright, which may have a debilitating effect, whereas others viewed it as getting pumped up,which may have a performance-enhancing effect. Lazarus and Folkman (1984) described reappraisal aschanging perceptions of the significance of the situation or one’s ability to manage situational demands (e.g.,“Is it really such a problem as I originally thought it was?”; “I was able to stand my ground last time, whywould I not be able to do it this time?”). Another way to achieve cognitive change is by comparing one’s ownskills with those of relevant peers (e.g., “If I fail, more than half of the class should fail too”). Gross and John(2002) asserted that reappraisal is positively linked to self-efficacy, positive mood, sharing emotions, andnegatively to negative affect, implying that it can be advantageous for learning.

Response modulation. Emotions can be regulated by influencing the psychological, physiological, andbehavioral responses that are part of emotion. Response modulation occurs late in the emotion episode, afterresponse tendencies have been initiated. For example, students who experience tension and bodily symptomsbefore taking an important test may try to tone down the physiological aspects of anxiety (e.g., increased heartrate, trembling hands) and its psychological symptoms (e.g., worry, feelings of uncertainty) by taking drugs,smoking, drinking, or by practicing relaxation techniques.

Expressive suppression is a response-focused strategy that aims to prevent the emotional response frombeing observed. Gross and Thompson (2007) reported that instructions to suppress one’s emotions whileviewing emotion-arousing videos successfully decreased expressive behavior, but increased rather thandecreased sympathetic arousal. Baumeister (2005) explained that suppression requires continuous monitoring,which taxes cognitive resources. Students who were requested to control their emotions gave up faster on asuccessive task than those who were not requested to do so. Suppression impeded task engagement andpersistence unless the spent energy was restored. Gross and John (2003) compared ER strategies andconfirmed that habitual use of reappraisal is linked to greater positive affect, better interpersonal functioning,and higher well-being, whereas suppression is associated with a less beneficial profile of emotionalfunctioning.

Rather than hiding emotions, students may express them verbally or non-verbally. The advantage ofemotion expression is that attention is called to what one feels, which may contribute to modifying thesituation by changing the behavior of others. For example, peers may become more mindful to what a studentfeels when she shows her disappointment.

195

Explicit versus implicit emotion regulation. Explicit ER requires conscious effort for initiation, demandsmonitoring during implementation, and is associated with some level of insight and awareness, whereasimplicit ER is evoked automatically by the stimulus itself, runs to completion without monitoring, and canhappen without insight and awareness (Gyurak, Gross, & Etkin, 2011). The strategies people use to regulatetheir emotions vary in explicitness over time and across situations. For example, a student who gets upsetwhen the physics teacher makes a cynical remark about his performance may remind himself that on Mondaysthe teacher resents coming to work. This reappraisal may automatically reduce his irritation on Mondays.Gyurak et al. (2011) maintained that, through repeated use, explicit if-then implementation plans for ERstrategies become habitual, involuntary responses that are used with little awareness, similar to theroutinization of cognitive appraisals described earlier.

Gross and Thompson (2007) argued that it is the interplay between explicit and implicit processing thatmakes ER adaptive. Explicit ER can successfully change emotional responses, but it requires extensivemonitoring and hence uses considerable resources. Under conditions of high cognitive load, explicit ER andtask-related information processing may compete for the same limited processing resources. Specifically,down-regulating negative emotions requires resources that may be depleted when the emotions to beregulated are frequent and intense (Baumeister, Zell, & Tice, 2007). By contrast, implicit regulation involvesless load and is often more reliable (Bargh & Williams, 2007).

Emotion Regulation and Self-regulated Learning: Boekaerts’ Dual Processing Model of Self-regulation

Successful ER is an essential aspect of self-regulated learning. Students often experience a dilemma betweendoing what is expected and will lead to positive learning outcomes in the long term and satisfying immediateneeds in order to feel good right away. This dilemma is at the heart of self-regulation: it illustrates that theinteraction between top-down and bottom-up self-regulation does not always run smoothly. Boekaerts andcolleagues (e.g., Boekaerts & Niemivirta, 2000) described top-down self-regulation as behavior that is drivenby the students’ own values and goals. It is contrasted with bottom-up self-regulation, which refers to data-driven processing of environmental cues in the actual situation that may create a mismatch with the students’own goals and trigger emotions.

The adaptable learning model, which later evolved into the dual processing self-regulation model, describesthe dynamic aspects of self-regulated learning and offers a theoretical scaffold for understanding the findingsfrom diverse psychological frameworks, including motivation, emotion, and learning (Figure 6.2). The criticalfeatures of Boekaerts’ self-regulation model are non-stop cognitive appraisals and their concomitant emotions.The model posits that students try to achieve a balance between two main goal priorities, namely masterystrivings (activity in the mastery pathway, symbolized by broken lines in Figure 6.2) and keeping their well-being within reasonable bounds (well-being pathway: dotted lines). The model postulates further thatcognitive appraisals—based on actual perceptions and activated cognitive and affective domain-specificinformation brought into working memory—are triggered when students are first confronted with a learningactivity (see top half of Figure 6.2) and that these cognitions are the proximal determinants of prospective,anticipatory positive and negative emotions, learning intention, and coping intention.

Seegers and Boekaerts (1996) tested these assumptions in the mathematics domain. They provided

196

empirical support that the effects of domain-specific beliefs on learning intention, emotions, and actual taskperformance are mediated by prospective anticipatory appraisals (subjective competence, success expectation,perception of difficulty, personal relevance, and task attraction). Using confirmatory factor analysis,Crombach, Boekaerts, and Voeten (2003) substantiated these findings for mathematics and other subjectmatter domains and showed that the tested model was stable over a half-year period.

Figure 6.2 Different pathways of Boekaerts’ dual processing self-regulation model: activity in the mastery pathway (– –), activity in the well-being pathway ( . . . ) and connecting pathways using emotion regulation strategies ( . . . ) and volitional strategies (–.. –.. –)

Boekaerts (2006) described how appraisals and emotions direct and redirect the focus of attention in theself-regulation system. Attention is directed to the learning activity itself (mastery pathway) when appraisalsand emotions are dominantly positive (implying that working memory capacity as well as (meta)cognitivestrategies are used to improve learning outcomes). By contrast, attention is directed away from the task whenappraisals and emotions are dominantly negative. At that point, students are more concerned with well-beingthan with learning, and task-irrelevant scenarios are activated from long-term memory to regulate emotions(coping).

Boekaerts and Corno (2005) described two types of bottom-up self-regulation strategies that students useto stay on task, namely ER strategies and volition strategies. The former strategies, which are called for whenstudents experience emotions that interfere with the learning task, dampen the emotional arousal and makeswitches between the pathways more probable. The latter strategies, also called good work habits, helpstudents to protect their learning intention when difficult work must be completed, and to re-route activitiesfrom the well-being to the mastery route.

The dual processing self-regulation model posits that students who know how to deal with emotions duringthe learning process will have higher achievement than students who do not have access to ER strategies.Punmongkol (2009) tested this assumption. She asked math teachers to train senior primary-school students

197

in the use of ER strategies and found that these students were better able to reduce their negative emotions.

In addition, their mathematics course grades improved significantly, especially when this training was given inconcert with metacognitive training in mathematics. Boekaerts (2010) argued that accessible ER and volitionstrategies act in tandem and function something like the switching track of a railway system. They turn allother lights to red to keep students on the mastery track or re-route them toward goals that make striving formastery more feasible.

Emotional Intelligence

The generation and regulation of emotions depend on individual competencies to produce, recognize,evaluate, increase or decrease, and make use of one’s own emotions. As such, a broad variety of cognitive andnon-cognitive abilities comprise an individual’s emotional competencies. The term “emotional intelligence” isoften used to denote these diverse abilities (Allen, MacCann, Matthews, & Roberts, 2014). This may implyan overinclusive use of the term “intelligence,” which is commonly used to denote cognitive rather than non-cognitive abilities. Further adding to conceptual ambiguity, authors such as Goleman (1995) and Bar-On(1997) used the term to denote an even broader array of individual dispositions, including various kinds oftrait-like variables that can relate to an individual’s emotional agency, such as assertiveness, self-regard,empathy, social responsibility, happiness, and optimism (for a critique, see Matthews, Zeidner, & Roberts,2002). Conceptions of this kind seem more similar to concepts of personality than to a circ*mscribed set ofcompetencies related specifically to emotion.

Given that emotions affect students’ performance, and that the regulation of these emotions depends onemotional competencies, measures of emotional intelligence should be able to predict academic achievementover and above IQ or prior achievement. The existing evidence only partially supports this expectation (e.g.,Amelang & Steinmayr, 2006; Di Fabio & Palazzeschi, 2009).

198

Recommendations for Practice

What can teachers and parents do to foster students’ adaptive emotions and prevent maladaptive emotions? Asnoted, research on academic emotions is clearly at a nascent stage. In deriving practical recommendations, itshould be kept in mind that cumulative evidence exists for students’ test anxiety, whereas the evidence foracademic emotions other than anxiety is still quite limited. By necessity, evidence-based recommendations todate primarily pertain to influencing students’ anxiety, whereas recommendations for emotions other thananxiety are less firmly based on empirical findings. Nevertheless, a number of implications for psychologicaland educational practice can be derived. Specifically, students’ emotions can be influenced by shaping tasksand learning environments in adequate ways and by using principles of social and emotional learningprograms. Furthermore, in the case of more severe individual problems, therapy can be used to improvestudents’ academic well-being (Zeidner, 1998).

Task Design, Teacher Behavior, and Learning Environments

It is the responsibility of educators and administrators to shape school environments to foster students’academic development and health, including their emotional approaches to learning. The following groups offactors may be especially important to consider.

Cognitive quality of instruction and task assignments. Raising the cognitive quality of instruction in terms ofclarity, structure, and high-quality examples should promote students’ learning and sense of control, thuspositively influencing their emotions. The same applies to task assignments. For example, from an affectiveperspective, cognitively activating material is preferable to less activating material (e.g., mathematicalmodeling tasks versus algorithmic tasks involving technical routine procedures). Furthermore, the level of taskdemands is important as well. Task demands influence students’ chances for mastery and resulting emotions.Moreover, the relative match between demands and individual capabilities can influence students’ valuing ofthe material. If demands are too high or too low, perceived value can be reduced to the extent that boredom isexperienced (Pekrun, 2006).

Cognitive quality and task demands are also critically important to instigate epistemic emotions. Tasksshould imply demands that can ultimately be met by students, yet simultaneously challenge their existingcognitive schemas. This should result in the arousal of surprise, curiosity, and productive confusion, possiblyprompting conceptual change (D’Mello, Lehman, Pekrun, & Graesser, 2014).

Motivational quality of instruction and task assignments. The control-value theory (Pekrun, 2006) proposesthat positive values of academic engagement and achievement should be fostered, and negative values shouldbe prevented, in order to facilitate adaptive emotions. Teachers, parents, and peers deliver direct, verbalmessages about academic values, as well as more indirect messages conveyed by their behavior. Two importantways to foster students’ academic values include the following. First, the development of values can bepromoted by shaping instructional material, assigned tasks, and classroom interaction such that they meetstudents’ needs (Deci & Ryan, 1987). Examples are authentic learning tasks and classroom discourse thatengage all students such that needs for social relatedness are met. Second, by way of observational learningand emotional contagion, teachers’ own enthusiasm in dealing with academic material likely facilitates

199

students’ absorption of values and positive emotions (Frenzel et al., 2009; Turner, Meyer, Midgley, & Patrick,2003). However, to be effective, it likely is important that enthusiasm is openly displayed, and that displayedenthusiasm is congruent with experienced emotion rather than just being enacted in superficial ways. Apartfrom value appraisals, competence appraisals that coincide with positive emotions will promote effortinvestment and positive outcome appraisals (Boekaerts, 2010).

Autonomy support and self-regulation. Learning environments that create demands to engage in self-regulatedlearning can also promote positive emotions. Self-regulation can take place at the individual or group level(individual vs. cooperative learning). As argued earlier, when the learning environment affords opportunitiesfor self-regulated learning, and when students perceive themselves as capable of regulating their learning,positive emotions (e.g., enjoyment) should be increased for at least two reasons. First, students can fulfill theirneed for autonomy and increase their sense of personal control (Deci & Ryan, 1987). As compared toindividual learning, cooperative self-regulated learning has the additional advantage of serving students’ socialneeds, which may also contribute to their appreciation of academic engagement. Second, students can selectand organize learning material to meet their individual interests, thus increasing the subjective value of thecourse and course material.

However, an important requirement for self-regulated learning to be emotionally effective is that studentsare in fact competent to self-direct their learning. If students are unable to regulate their learning, negativeemotions may be promoted due to perceived loss of control, thus highlighting the need to fine-tune theaffordances and constraints of these learning environments to students’ regulatory capabilities.

Goal structures, grading practices, and achievement expectations. The goal structures of the classroom definestudents’ opportunities to attain success and avoid failure (Johnson & Johnson, 1974). In individualistic goalstructures, achievement is defined by individual competence gain (individual standard of evaluation) or bymastery of the learning material (absolute standard). Under such structures, individual achievement isindependent of other students’ attainment, meaning that all students can attain success provided thatsufficient progress is made. By contrast, in competitive goal structures, achievement is defined by normativesocial comparison standards, making individual achievement dependent on the relative attainment of others.In competitive structures, the achievement of different students is negatively linked, because success of somestudents implies failure of others. By implication, only some students may expect success, while others mustexpect failure. Cooperative goal structures imply that achievement is defined by the performance of the group,meaning that attainment is positively linked across individuals.

Different goal structures give students different opportunities to experience success. For the averagestudent, opportunities for perceived control may be higher under individualistic and cooperative goalstructures, as compared with competitive structures. By implication, although competitive structures can beenjoyable for high-achieving students (Frenzel, Pekrun, & Goetz, 2007), their average emotional effects arelikely less beneficial. Accordingly, teachers should refrain from using social comparison standards to assessstudent achievement. Specifically, this applies to high-stakes assessments that can have dramatic emotionalconsequences due to making important consequences contingent on achievement. Grading based on socialcomparison may be necessary for purposes of placement and selection, implying that the goals of fosteringstudents’ emotions and gathering information on their performance may conflict. However, to the extent that

200

assessments aim to serve teaching and learning rather than selection purposes, criterion-oriented gradingpertaining to mastery of the learning material is likely more recommendable.

Teachers’, parents’, and peers’ individual achievement expectations can operate in similar ways as the overallgoal structures applied to the classroom. These expectations provide definitions of success and failure, thusinfluencing students’ perceived control and their emotions. To promote perceptions of control and positivelyimpact the resulting emotions, expectations should not exceed students’ capabilities.

Design of tests and exams. Drawing on test anxiety research, measures that increase perceived control anddecrease the importance of failure can be beneficial. These measures include: (a) providing structure andtransparency regarding task demands, materials, exam procedures, and grading practices; (b) avoidingexcessively high task demands; (c) relaxing time constraints; (d) giving students a choice between tasks; (e)giving students second chances in terms of retaking tests; (f) providing external aids, such as access to lecturenotes, text books, or computers; and (g) using closed-item formats to ease working memory load (Zeidner,1998, 2014). Naturally, some of these measures also have disadvantages. For example, using only multiple-choice items may reduce anxiety, but may preclude the use of item formats better suited to assess deep-levelthinking and creative problem solving. As such, educational measures to reduce students’ anxiety should becounter balanced in the context of multiple educational goals.

Consequences of performance. Regarding the value of academic success and failure for future outcomes (e.g.,career opportunities), it should prove helpful to highlight connections between students’ academic effort andthe attainment of future prospects. Effort–outcome associations of this type can increase students’ perceivedcontrol and interest (Hulleman & Harackiewicz, 2009), thus strengthening positive and reducing negativefuture-related emotions (Pekrun, 2006). To the contrary, should future desired outcomes not be contingentupon students’ effort, then students may experience reduced perceived subjective control and increasednegative prospective emotions like anxiety or hopelessness.

Emotional scaffolding, social and emotional learning, and treatment. Students may experience negativeemotions or a lack of positive emotions in class and may find it difficult to regulate them. At such a pointsupport from their teacher in terms of external ER and emotional scaffolding can be helpful. Yet, Hyson,Copple, and Jones (2006) observed that only one-third of preschool teachers spent time talking about feelings.As argued by Boekaerts (2010), it may be difficult for the teacher to provide appropriate emotional support.Students may differ in the type of emotional support they expect, and teachers may lack knowledge of how toprovide such support. For improving the situation, using principles of SEL programs may be helpful. Theseprograms have been developed to support students’ ER skills more generally but could be used to also improvetheir emotional situation at school (see Brackett & Rivers, 2014). Teaching students how to anticipateemotionally challenging situations and how to prepare for regulating these situations may be especiallyimportant in this respect (Punmongkol, 2009).

201

Directions for Future Research

As outlined in this chapter, research on students’ emotions in academic settings has been slow to emerge.During the past 15 years, however, there has been an increase of studies examining the nature of students’emotions (Pekrun & Linnenbrink-Garcia, 2014). These studies have produced new insights demonstratingthat emotions profoundly affect students’ engagement, achievement, and identity, which also implies that theyare of critical importance for the agency of educational institutions and of society at large. At the same time,however, the studies conducted thus far seem to pose more new, challenging questions than they can answer.Theories, strategies, and measures for analyzing these emotions have yet to be fully developed. Also, to datestudies are too scarce to allow any meta-analytic synthesis based on cumulative evidence, or any firmconclusions informing practitioners in validated ways how to deal with emotions, evidence on test anxietybeing an exception. The progress made so far is promising, but much more has to be done if research onacademic emotions is to evolve over the next years in ways benefiting education and society. Importantchallenges include the need for more cross-cultural research, the need to integrate perspectives from affectiveneuroscience into emotion research in educational psychology, and the need to set up intervention studies thatexamine how students can acquire ER strategies. We also need a more fine-grained analysis of the interactionsbetween discrete emotions in the classroom as well as longitudinal studies to document the development ofstudents’ theory of emotion in connection with their evolving theory of ER and the impact of these theorieson student’s engagement and learning.

202

Acknowledgment

The writing of this chapter was supported by a Ludwig Maximilian University Research Chair grant awardedto Reinhard Pekrun by the University of Munich.

203

References

Ahmed, W., van der Werf, G., Kuyper, H., & Minnaert, A. (2013). Emotions, self-regulated learning, and achievement in mathematics: Agrowth curve analysis. Journal of Educational Psychology, 105, 150–161.

Allen, V., MacCann, C., Matthews, G., & Roberts, R. D. (2014). Emotional intelligence in education: From pop to emerging science. In R.Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 162–182). New York: Taylor & Francis.

Amelang, M., & Steinmayr, R. (2006). Is there a validity increment for tests of emotional intelligence in explaining the variance of performancecriteria? Intelligence, 34, 459–468.

Aspinwall, L. (1998). Rethinking the role of positive affect in self-regulation. Motivation and Emotion, 22, 1–32.Bargh, J. A., & Williams, L. E. (2007). The nonconscious regulation of emotion. In J. J. Gross (Ed.), Handbook of emotion regulation (pp. 429–

445). New York: Guilford Press.Bar-On, R. (1997). The Emotional Intelligence Inventory (EQ-I): Technical manual. Toronto, Canada: Multi-Health Systems.Barrett, L. F., & Russell, J. A. (1998). Independence and bipolarity in the structure of current affect. Journal of Personality and Social Psychology,

74, 967-984.Baumeister, R. F. (2005). The cultural animal: Human nature, meaning, and social life. New York: Oxford University Press.Baumeister, R. F., Zell, A. L., & Tice, D. M. (2007). How emotions facilitate and impair self-regulation. In J. J. Gross (Ed.), Handbook of

emotion regulation (pp. 408–427). New York: Guilford Press.Boekaerts, M. (1993). Anger in relation to school learning. Learning and Instruction, 3, 269–280.Boekaerts, M. (1999). Coping in context: Goal frustration and goal ambivalence in relation to academic and interpersonal goals. In E.

Frydenberg (Ed.). Learning to cope: Developing as a person in complex societies (pp. 175–197). Oxford: Oxford University Press.Boekaerts, M. (2006). Self-regulation and effort investment. In E. Sigel & K. A. Renninger (Vol. Eds), Handbook of child psychology (Vol. 4, pp.

345–377). New York: Wiley.Boekaerts, M. (2010). Motivation and self-regulation: Two close friends. In T. C. Urdan & S. A. Karabenick (Eds.), The decade ahead:

Applications of motivation and achievement. Advances in motivation and achievement (Vol. 16b, pp. 69–108). Bingley, UK: Emerald.Boekaerts, M. (2011). Emotions, emotion regulation, and self-regulation of learning. In B. Zimmerman & D. Schunk (Eds.), Handbook of self-

regulation of learning and performance (pp. 408–425). London: Routledge.Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and intervention. Applied Psychology: An

International Review, 54, 199–231.Boekaerts, M., & Niemivirta, M. (2000). Self-regulated learning: Finding a balance between learning goals and ego-protective goals. In M.

Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 417–450). San Diego, CA: Academic Press.Boekaerts, M., & Röder, I. (1999). Stress, coping, and adjustment in children with a chronic disease: A review of the literature. Disability and

Rehabilitation, 21, 311–337.Brackett, M. A., & Rivers, S. E. (2014). Transforming students’ lives with social and emotional learning. In R. Pekrun & L. Linnenbrink-

Garcia (Eds.), International handbook of emotions in education (pp. 368–388). New York: Taylor & Francis.Buff, A. (2014). Enjoyment of learning and its personal antecedents: Testing the change–change assumption of the control-value theory of

achievement emotions. Learning and Individual Differences, 31, 21–29.Chan, C. K., McDermott, K. B., & Roediger, H. L. (2006). Retrieval induced facilitation: Initially nontested material can benefit from prior

testing. Journal of Experimental Psychology: General, 135, 533–571.Clore, G. L., & Huntsinger, J. R. (2009). How the object of affect guides its impact. Emotion Review, 1, 39–54.Compas, B. E., Malcarne, V. L., & Fondacaro, K. (1988). Coping with stressful events in older children and adolescents. In L. A. Bond & B.

E. Compas (Eds.), Primary prevention and promotion in the schools (pp. 319–340). Newbury Park: Sage.Crombach, M. J., Boekaerts, M., & Voeten, M. J. M. (2003). Online measurement of appraisals of students faced with curricular tasks.

Educational and Psychological Measurement, 63(1), 96–111.Daniels, L. M., & Stupnisky, R. H. (2012). Not that different in theory: Discussing the control-value theory of emotions in online learning

environments. Internet and Higher Education, 15, 222–226.Deci, E. L., & Ryan, R. M. (1987). The support of autonomy and the control of behavior. Journal of Personality and Social Psychology, 53, 1024–

1037.Di Fabio, A., & Palazzeschi, L. (2009). An in-depth look at scholastic success: Fluid intelligence, personality traits or emotional intelligence?

Personality and Individual Differences, 46, 581–585.D’Mello, S. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of

Educational Psychology, 105, 1082–1099.D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153–170.

204

Efklides, A. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Educational ResearchReview, 1, 3–14.

Ellis, H. C., & Ashbrook, P. W. (1988). Resource allocation model of the effect of depressed mood states on memory. In K. Fiedler & J. Forgas(Eds.), Affect, cognition, and social behavior (pp. 25–43). Toronto: Hogrefe International.

Fredricks, J. A., & Eccles, J. (2002). Children’s competence and value beliefs from childhood through adolescence: Growth trajectories in twomale-sex-typed domains. Developmental Psychology, 38, 519–533.

Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. AmericanPsychologist, 56, 218–226.

Frenzel, A. C., Goetz, T., Lüdtke, O., Pekrun, R., & Sutton, R. E. (2009). Emotional transmission in the classroom: Exploring therelationship between teacher and student enjoyment. Journal of Educational Psychology, 101, 705–716.

Frenzel, A. C., Pekrun, R., & Goetz, T. (2007). Perceived learning environment and students’ emotional experiences: A multi-level analysis ofmathematics classrooms. Learning and Instruction, 17, 478–493.

Goetz, T., Frenzel, A. C., Stoeger, H., & Hall, N. C. (2010). Antecedents of everyday positive emotions: An experience sampling analysis.Motivation and Emotion, 34, 49–62.

Goleman, D. (1995). Emotional intelligence. New York: Bantam.Graham, S., & Taylor, A. Z. (2014). An attributional approach to emotional life in the classroom. In R. Pekrun & L. Linnenbrink-Garcia

(Eds.), International handbook of emotions in education (pp. 96–119). New York: Routledge.Gross, J. J., & John, O. P. (2002). Wise emotion regulation. In L. F. Barrett & P. Salovey (Eds.), The wisdom of feelings: Psychological processes in

emotional intelligence (pp. 297–318). New York, NY: Guilford Press.Gross, J. J., & John, O. P. (2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-

being. Journal of Personality and Social Psychology, 85, 348–362.Gross, J. J., & Thompson, R. A. (2007). Emotion regulation: Conceptual foundations. In J. J. Gross (Ed.), Handbook of emotion regulation (pp.

3–26). New York, NY: Guilford Press.Gyurak, A., Gross, J. J., and Etkin, A. (2011). Explicit and implicit emotion regulation: A dual-process framework. Cognition and Emotion, 25,

400–412.Heckhausen, H. (1991). Motivation and action. New York: Springer.Helmke, A. (1993). Die Entwicklung der Lernfreude vom Kindergarten bis zur 5. Klassenstufe [Development of enjoyment of learning from

kindergarten to grade 5]. Zeitschrift für Pädagogische Psychologie, 7, 77–86.Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 47–77.Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326(5958), 1410–

1412.Hyson, M., Copple, C., & Jones, J. (2006). Early child development and education. In K. A. Renninger & I. E. Sigel (Eds.), Handbook of child

psychology (6th ed., pp. 3–47). New York: Wiley.Jacobs, S. E., & Gross, J. J. (2014). Emotion regulation in education: Conceptual foundations, current applications, and future directions. In R.

Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 183–201). New York: Taylor & Francis.Johnson, D. W., & Johnson, R. T. (1974). Instructional goal structure: Cooperative, competitive or individualistic. Review of Educational

Research, 4, 213–240.Kuhbandner, C., & Pekrun, R. (2013). Affective state influences retrieval-induced forgetting for integrated knowledge. PloS ONE, 8(2),

e56617.Lane, A. M., Whyte, G. P., Terry, P. C., & Nevill, A. M. (2005). Mood, self-set goals and examination performance: The moderating effect of

depressed mood. Personality and Individual Differences, 39, 143–153.Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping. New York: Springer.Lewis, M., Haviland-Jones, J. M., & Feldman Barrett, L. (Eds.). (2008). Handbook of emotions (3rd ed.). New York: Guilford.Linnenbrink, E. A. (2007). The role of affect in student learning: A multi-dimensional approach to considering the interaction of affect,

motivation, and engagement. In P. A. Schutz & R. Pekrun (Eds.), Emotion in education (pp. 107–124). Amsterdam: Academic Press.Linnenbrink, E. A., & Pintrich, P. R. (2002). Achievement goal theory and affect: An asymmetrical bidirectional model. Educational

Psychologist, 37, 69–78.Marsh, H. W. (1987). The big-fish-little-pond effect on academic self-concept. Journal of Educational Psychology, 79, 280–295.Matthews, G., Zeidner, M., & Roberts, R. D. (2002). Emotional intelligence: Science and myth. Cambridge, MA: MIT Press.Meece, J.L., Wigfield, A., & Eccles, J. S. (1990). Predictors of math anxiety and its influence on young adolescents course enrollment

intentions and performance in mathematics. Journal of Educational Psychology, 82, 60–70.Meinhardt, J., & Pekrun, R. (2003). Attentional resource allocation to emotional events: An ERP study. Cognition and Emotion, 17, 477–500.

205

Parrott, W. G., & Spackman, M. P. (2000). Emotion and memory. In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of emotions (2nd ed.,pp. 476–490). New York: Guilford Press.

Pekrun, R. (1992). Expectancy-value theory of anxiety. In D. G. Forgays, T. Sosnowski, & K. Wrzesniewski (Eds.), Anxiety: Recentdevelopments in self-appraisal, psychophysiological and health research (pp. 23–41). Washington, DC: Hemisphere.

Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research andpractice. Educational Psychology Review, 18, 315–341.

Pekrun, R., Cusack, A., Murayama, K., Elliot, A. J., & Thomas, K. (2014). The power of anticipated feedback: Effects on students’achievement goals and achievement emotions. Learning and Instruction, 29, 115–124.

Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: Testing a model of their joint relations withacademic performance. Journal of Educational Psychology, 101, 115–135.

Pekrun, R., Goetz, T., Daniels, L. M., Stupnisky, R. H., & Perry, R. P. (2010). Boredom in achievement settings: Control-value antecedentsand performance outcomes of a neglected emotion. Journal of Educational Psychology, 102, 531–549.

Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and performance: TheAchievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology, 36, 36–48.

Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program ofquantitative and qualitative research. Educational Psychologist, 37, 91–106.

Pekrun, R., Hall, N. C., Goetz, T., & Perry, R. P. (2014). Boredom and academic achievement: Testing a model of reciprocal causation.Journal of Educational Psychology 106, 696–710.

Pekrun, R., & Linnenbrink-Garcia, L. (Eds.). (2014). International handbook of emotions in education. New York: Taylor & Francis.Pekrun, R., & Perry, R. P. (2014). Control-value theory of achievement emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International

handbook of emotions in education (pp. 120–141). New York: Taylor & Francis.Pekrun, R., vom Hofe, R., Blum, W., Frenzel, A. C., Goetz, T., & Wartha, S. (2007). Development of mathematical competencies in

adolescence. In M. Prenzel (Ed.), Studies on the educational quality of schools (pp. 17–37). Münster: Waxmann.Preckel, F., Zeidner, M., Goetz, T., & Schleyer, E. (2008). Female ‘big fish’ swimming against the tide: The ‘big-fish-little-pond effect’ and

gender ratio in special gifted classes. Contemporary Educational Psychology, 33, 78–96.Punmongkol, P. (2009). The regulation of academic emotions (unpublished doctoral dissertation). Sydney, Australia: University of Sydney.Roeser, R. W., Midgley, C., & Urdan, T. C. (1996). Perceptions of the school psychological environment and early adolescents’ psychological

and behavioral functioning in school: The mediating role of goals and belonging. Journal of Educational Psychology, 88, 408–422.Schutz, P. A., & Pekrun, R. (Eds.). (2007). Emotion in education. San Diego, CA: Academic Press.Schutz, P. A., & Zembylas, M. (Eds.). (2009). Advances in teacher emotion research. The impact on teachers’ lives. New York: Springer.Seegers, G., & Boekaerts, M. (1996). Gender-related differences in self-referenced cognitions in relation to mathematics. Journal for Research in

Mathematics Education, 27(2), 215–240.Shuman, V., & Scherer, K. R. (2014). Concepts and structures of emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International

handbook of emotions in education (pp. 13–35). New York: Taylor & Francis.Turner, J. C., Meyer, D. K., Midgley, C., & Patrick, H. (2003). Teacher discourse and sixth graders’ reported affect and achievement behaviors

in two high-mastery/high-performance mathematics classrooms. Elementary School Journal, 103, 357.Turner, J. E., & Schallert, D. L. (2001). Expectancy-value relationships of shame reactions and shame resiliency. Journal of Educational

Psychology, 93, 320–329.Valeski, T. N., & Stipek, D. J. (2001). Young children’s feelings about school. Child Development, 72, 1198–1213.Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548–573.Westefeld, J. S., Homaifar, B., Spotts, J., Furr, S., Range, L., & Werth, J. L. (2005). Perceptions concerning college student suicide: Data from

four universities. Suicide & Life-Threatening Behavior, 35, 640–645.Zeidner, M. (1998). Test anxiety: The state of the art. New York: Plenum.Zeidner, M. (2014). Anxiety in education. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp.

265–288). New York: Taylor & Francis.

206

7Motivation

LISA LINNENBRINK-GARCIA

Michigan State University

ERIKA A. PATALL

University of TExas, Austin What is motivation? As you read this chapter, you may have your own ideas about what constitutesmotivation and motivated behavior in the classroom. While definitions vary, a working definition consistentwith the theoretical frameworks described in this chapter is that motivation refers to the processes of bothinitiating and sustaining behavior (Schunk, Meece, & Pintrich, 2014). Moreover, the study of motivation ineducational psychology goes beyond thinking of students as motivated or unmotivated to examine how theirself-related beliefs, cognitions, goals, and experiences shape engagement and learning. Importantly, these self-related motivational beliefs are thought to be “cognitive, conscious, affective, and often under control of theindividual” (Wigfield, Eccles, Schiefele, Roeser, & Davis-Kean, 2006, p. 933).

We begin this chapter with a focus on six major theoretical frameworks from which much of the motivationresearch in education is currently conducted. Our aim in describing these theories is to provide a shortintroduction and description of how motivation relates to learning and engagement within each theory, notingrecent theoretical and empirical advances since the publication of the last Handbook of Educational Psychology.Next, we consider recent empirical and theoretical work that integrates across theoretical perspectives. Weclose by suggesting several avenues for future research.

207

Major Theoretical Approaches to the Study of Motivation in Education

Social Cognitive Theory

Theoretical overview. Bandura’s (1986) social cognitive theory is one of the major modern theories ofmotivation, both because it has contributed broad ideas about social cognition and for its theorizing regardingacademic self-efficacy. Indeed, the major motivational theories reviewed in this chapter all emphasizereciprocal determinism or the interplay among the person, behavior, and the environment, a key conceptwithin social cognitive theory. Moreover, the concept of agency, that individuals are “self-organizing,proactive, self-regulating, and self-reflecting” (Bandura, 2006, p. 164), underlies modern research onmotivation.

Within social cognitive theory, the construct of self-efficacy is most relevant to the current chapter. Self-efficacy refers to individuals’ beliefs about their capacity to execute behaviors at particular levels (Bandura,1997). Applied to education, academic self-efficacy refers to students’ beliefs about their ability to learn,develop skills, or master material. Self-efficacy is distinct from outcome expectations (e.g., belief that a givenbehavior will lead to a certain outcome) and self-concept (e.g., cognitive evaluations of ability: Bong &Skaalvik, 2003; Schunk & Pajares, 2005). Below, we briefly describe the construct of self-efficacy and currenttrends regarding its relations with academic outcomes (see Chapter 11, this volume, for more on self-efficacy).

Research related to engagement, learning, and achievement. Self-efficacy beliefs are related to students’ courseand career choices, putting forth greater effort and task persistence even in the face of failure, increased use ofadaptive self-regulatory strategies, more positive and less negative emotions, and enhanced academicachievement (Bandura, 1986, 1997; Pajares, 1996; see also Klassen & Usher, 2010; Schunk & Pajares, 2005;and Chapter 11, this volume). In the past decade, many studies have further documented how students’ self-efficacy beliefs shape affective, behavioral, and cognitive engagement as well as noted group differences basedon gender and ethnicity (see Klassen & Usher, 2010; and Chapter 11, this volume). At the intersection ofself-regulation and self-efficacy, researchers have also considered calibration (e.g., the congruency betweenefficacy judgments and actual performance), although this remains understudied. For instance, Chen (2003)found that both calibration and self-efficacy had independent, positive effects on adolescents’ mathperformance. Klassen (2007) found that learning-disabled students had lower self-efficacy, as expected, butthat they were also less calibrated (more overconfident) than non-learning-disabled students, which maypartially explain achievement differences between these groups.

There is also a growing body of research focused on collective self-efficacy beliefs. Research on thecollective efficacy of teachers for supporting students’ learning suggests that this construct is predictive ofschool-wide achievement and student behavior (see Chapter 30, this volume; Klassen, Tze, Betts, & Gordon,2011). There is also emerging research on students’ collective efficacy in small groups, indicating that it toopredicts performance (Klassen & Krawchuk, 2009). Researchers have also sought to clarify how self-efficacy issupported in educational settings (see Usher & Pajares, 2008). For instance, using latent profile analysis, Chenand Usher (2013) found that adolescents who drew from multiple sources of self-efficacy (mastery experience,vicarious experience, social persuasion, and affective/physiological states) simultaneously had the highestscience self-efficacy and achievement, which was significantly higher than students who derived their self-

208

efficacy primarily from mastery experiences alone. Those most at risk appeared particularly sensitive to

physiological arousal information.

Expectancy-Value Theory

Theoretical overview. Expectancy-value conceptualizations of behavior have a long history in psychology (e.g.,Atkinson, 1964). Similar to predecessors, modern expectancy-value theory (Eccles et al., 1983) assumes thatindividuals’ expectations for success and subjective value for tasks are the most proximal predictors of theiracademic choices, achievement-related behaviors, and ultimately, learning and achievement, and arethemselves predicted by a variety of psychological, social, and cultural influences (for recent reviews, seeEccles, 2005; Wigfield, Tonks, & Klauda, 2009). Expectancies refer to individuals’ beliefs about how well theywill do on upcoming tasks and are distinct conceptually, but not empirically, from beliefs about ability(evaluations of competence, Eccles & Wigfield, 1995; see Chapter 11, this volume). Four categories of taskvalue exist: utility value (task perceived as useful to other aspects of the person’s life), attainment value(personal importance or self-relevance of doing well on a task), intrinsic value (perceiving the task asinteresting, enjoyable, or fun), and cost (negative aspects of engaging in the task). Research indicates thatstudents can distinguish between competence and value beliefs in early elementary school and can differentiateamong types of value by fifth grade (e.g., Eccles & Wigfield, 1995).

Research related to engagement, learning, and achievement. Much research within the United States and acrosscountries suggests that expectancies and value1 predict achievement outcomes, including persistence,performance, and choice of activities (e.g., Chow, Eccles, & Salmela-Aro, 2012; Durik, Vida, & Eccles, 2006;Nagengast et al., 2011) among children as young as first grade, and the relations gain strength with age(Denissen, Zarrett, & Eccles, 2007; Eccles et al., 1983). Expectancies for success most strongly predictperformance, even when previous performance is controlled, and generally precede and predict students’ values(e.g., Jacobs, Lanza, Osgood, Eccles, & Wigfield, 2002), though possibly less so for females (Denissen et al.,2007). In contrast, students’ task values most strongly predict activity choices and enrollment decisions, evenhaving long-term consequences. For example, values in elementary years predict activity choice and courseenrollment in high school (e.g., Durik et al., 2006). Interestingly, expectancies and values may not simplywork to additively and independently predict academic outcomes. Rather, recent research conducted withadolescents suggests that the effect of expectancies on academic choices and achievement is stronger whenvalue is higher and vice versa, but neither high expectancies nor values can compensate for when the other islow (Nagengast et al., 2011; Trautwein et al., 2012).

Given the clear links of expectancies and task value to important academic outcomes, researchdemonstrating that students experience age-related declines in expectancies (see Chapter 11, this volume) andvalues in the United States (e.g., Archambault, Eccles, & Vida, 2010; Jacobs et al., 2002) and other countries(e.g., Henderson, Marx, & Kim, 1999; Watt, 2004) is of great concern to educators. Recent advances in thisarea include the use of growth mixture modeling and other sophisticated techniques to investigateheterogeneity in developmental trajectories across individuals and domains over time (e.g., Archambault et al.,2010). Declines seem to occur particularly for language arts in early elementary years and for math duringhigh school. Likewise, gender-stereotypic differences in competence beliefs and task values are of particular

209

practical interest, given links between these beliefs and career trajectories. For instance, girls express higherexpectancies and task value for language arts domains and boys express higher beliefs for math, sports, science,and engineering (e.g., Durik et al., 2006; Jacobs et al., 2002; Watt, 2004; see Chapter 11, this volume;Wigfield, Battle, Keller, & Eccles, 2002 for reviews).

One area of research with particular growth in the last decade is the cost component of task value. Thoughmost empirical examinations of task values have overlooked cost, recent research suggests that cost may helpto differentiate levels of academic success. For example, cost negatively predicted undergraduates’ intentions toenter graduate school, controlling for other forms of value (Battle & Wigfield, 2003). Similarly, costdifferentiated individuals in terms of their motivational profiles, affect, and achievement outcomes, withstudents in high-cost profiles experiencing less adaptive outcomes overall (Conley, 2012). Recent researchfurther suggests that cost, like other forms of value, is multidimensional. Perez, Cromley, and Kaplan (2014)reported varying results based on type of cost, with effort cost followed by opportunity cost as the strongestpredictors of students’ intentions to leave science, technology, engineering, and mathematics majors;psychological cost was unrelated to intentions. Finally, there is some limited evidence that costs may be aparticularly powerful predictor of women’s occupational choices. For example, concerns about job flexibilityand high time demands in the context of balancing work and family life, along with lower intrinsic value ofphysical science, were the best predictors of women changing their occupational aspirations out of male-dominated fields (Frome, Alfeld, Eccles, & Barber, 2006).

Interest

Theoretical overview. Psychologists have been studying interest for more than a century (e.g., Dewey, 1913;James, 1890). Despite such strong roots, it has been relatively neglected, but has benefited from a surge ofresearch in the past few decades. While there are varying views on interest, much of the current researchdifferentiates between two forms: individual and situational (see Renninger, Hidi, & Krapp, 1992; Renninger& Hidi, 2011). Individual interest (a.k.a. personal interest) is relatively stable and resides within theindividual; it includes a deep personal connection to the domain and a willingness to re-engage in the domainover time (Schiefele, 2009). Individual interest is characterized by positive feelings (e.g., enjoyment) as well asvalue for and personal importance of the domain. Additionally, Renninger and colleagues (Hidi & Renninger,2006) propose that knowledge is a key component and that individual interest can be differentiated intoemerging and well-developed forms, with deeper levels of stored knowledge serving as a catalyst for shiftsfrom emerging to well-developed individual interest. Situational interest refers to interest that emerges fromand is supported by the context (Schiefele, 2009). As with individual interest, there are several different viewsof situational interest (see Hidi & Renninger, 2006; Krapp, 2002, Krapp & Prenzel, 2011; Mitchell, 1993;Schiefele, 2009; Silvia, 2005), but most of them include at least two primary forms. One form, triggeredsituational interest,2 is a relatively short, heightened affective state that is initiated by contextual supports. Theother form, maintained situational interest,3 refers to situational support of more focused involvement,attention, and persistence in a domain, including finding meaning and personal connections to the domaincontent. With maintained situational interest, students are likely to experience positive feelings (e.g.,enjoyment), but are also developing deeper value for and knowledge of the content.

210

Research related to engagement, learning, and achievement. Much research focused on text-based interestsuggests that situational interest or actualized individual interest supports increased attention, cognitiveprocessing, and persistence on reading tasks (Hidi, 2001; Schiefele, 2009; Schraw & Lehman, 2001).Recently, researchers have extended beyond text-based work to examine the role of interest in predictingengagement and learning more broadly. For instance, using multiple assessments within a single daylongproblem-based learning session for undergraduates, Rotgans and Schmidt (2011) found that situationalinterest predicted academic engagement and in turn achievement. In several classroom and lab-basedcorrelational and experimental studies conducted by Harackiewicz and colleagues, situational interest waspositively related to task involvement (Durik & Harackiewicz, 2007) and course grades (e.g., Harackiewicz,Durik, Barron, Linnenbrink-Garcia, & Tauer, 2008; Hulleman & Harackiewicz, 2009), but was a strongerpredictor of course choice than achievement several years later (e.g., Harackiewicz et al., 2008). Notably, theobserved effects are not always straightforward and at times vary based on perceived competence, initialinterest, prior achievement, or type of situational interest. For instance, Durik and Harackiewicz (2007) foundthat triggered situational interest supported task involvement for undergraduates with low individual interest,but undermined involvement for those with high interest; maintained situational interest was related to highertask involvement for undergraduates with high initial individual interest only.

In the past decade, interest researchers have also developed more detailed theoretical accounts regarding thedevelopment of interest (Hidi & Renninger, 2006) and provided empirical evidence documenting shifts fromsituational to individual interest (e.g., Harackiewicz et al., 2008; Linnenbrink-Garcia, Patall, & Messersmith,2013; Renninger & Hidi, 2002). Relatedly, there is a growing body of research aimed at understandingcontextual supports for situational interest. This work suggests that several contextual factors, includingautonomy support, instructor approachability and friendliness, opportunities for involvement, and relevance ofcourse material, support situational interest and may in turn support individual interest (e.g., Hulleman &Harackiewicz, 2009; Linnenbrink-Garcia et al., 2013; Palmer, 2009; Rotgans & Schmidt, 2011).

Self-determination Theory

Theoretical overview. Self-determination theory (Deci & Ryan, 1985; Ryan & Deci, 2000) is a macro theoryof motivation and development that has particular relevance to education. Self-determination theorydistinguishes among types of motivation based on reasons for action. In line with a long history in psychology(Berlyne, 1960; White, 1959), the most basic distinction is between intrinsic motivation, doing something forthe inherent satisfaction that engaging in the activity provides, and extrinsic motivation, doing somethingbecause it leads to a separable outcome (e.g., praise or money; Ryan & Deci, 2000). Further, extrinsicmotivation may itself vary in the degree to which it is internalized and experienced as autonomous versuscontrolled (Ryan & Connell, 1989). In addition to fully extrinsic versus intrinsic forms, motivation for actionmay emerge from feelings of obligation, guilt, or pride (introjected), because a behavior is perceived to haveutility or importance for accomplishing personal goals (identified), or because it is fully internalized andrepresentative of one’s central values (integrated).

Self-determination theory proposes that innate psychological needs for autonomy (e.g., feeling that actionsemanate from the self), competence, and relatedness underlie people’s natural growth tendencies, optimalpsychological functioning, and productivity (e.g., Jang, Reeve, Ryan, & Kim, 2009; Ryan & Deci, 2000).

211

Satisfaction of or support for these needs enhances intrinsic motivation as well as internalization (e.g., movingfrom more external to internal forms of regulation: Ryan & Deci, 2000). Further, a growing body of evidencehas demonstrated the importance of these basic needs for supporting psychological functioning acrossindividualistic (Western) and collectivistic (Eastern) cultures (e.g., Ferguson, Kasser, & Jahng, 2010; Jang etal., 2009).

Research related to engagement, learning, and achievement. Extensive research relates students’ intrinsicmotivation and other autonomous forms of motivation to adaptive academic outcomes, including creativity,academic engagement, deep conceptual learning strategies, and academic achievement (e.g., Corpus,McClintic-Gilbert, & Hayenga, 2009; Lepper, Corpus, & Iyengar, 2005; Otis, Grouzet, & Pelletier, 2005;Walker, Greene, & Mansell, 2006). In contrast, many studies suggest that more extrinsic forms of motivationpredict negative outcomes, such as maladaptive learning strategies and attitudes, anxiety, poorer ability to copewith challenges, poor academic achievement, and even school dropout (e.g., Lepper et al., 2005; Ryan &Connell, 1989; Vansteenkiste, Zhou, Lens, & Soenens, 2005; Walker et al., 2006), though a few studiessuggest that extrinsic motivation may at times be beneficial for outcomes such as self-regulation and academicadjustment (Miller, Greene, Montalvo, Ravindran, & Nichols, 1996; Otis et al., 2005). Given these patterns,documented declines in both intrinsic and extrinsic motivation within and across school years (e.g. Corpus etal., 2009; Lepper et al., 2005) are a concern. For example, in a longitudinal study with third- through eighth-graders, intrinsic motivation and classroom grades mutually influenced one another positively and reciprocallyover the academic year (Corpus et al., 2009). In contrast, extrinsic motivation was unrelated to grades, butpoor academic performance predicted higher extrinsic motivation. Both extrinsic and intrinsic motivationdeclined within the school year and across grade levels.

Person-centered approaches exploring profiles of intrinsic and extrinsic motivation have led to similarconclusions, with some studies suggesting that high autonomous motivation and low controlled motivationare most adaptive (e.g., Hayenga & Corpus, 2010; Ratelle, Guay, Vallerand, Larose, & Senecal, 2007;Vansteenkiste, Sierens, Soenens, Luyckx, & Lens, 2009), and other studies suggesting that high levels of bothforms of motivation can also be beneficial (Wormington, Corpus, & Anderson, 2012). Some of this debatemay be resolved by assessing the differential effects of various forms of motivation across outcomes. Forexample, in one study with elementary-school students, intrinsic motivation was most strongly linked withpsychological well-being, while identified motivation was most strongly linked with academic achievement(e.g., Burton, Lydon, D’Allessandro, & Koestner, 2006).

Within the past decade, extensive research suggests that psychological needs, and in turn engagement andachievement, can be supported by the environment through teaching practices such as providing meaningfulchoices, emphasizing personal relevance, using non-controlling informational language, allowing students toexpress opinions and negative affect, and providing feedback and structure (see Reeve, 2009; Reeve & Jang,2006; Stroet, Opdenakker, & Minnaert, 2013 for reviews). Conversely, directly controlling teacher behaviors(e.g., intentional suppression of perspectives, commands, and surveillance) may have maladaptiveconsequences for motivation, engagement, and learning (e.g., Assor, Kaplan, Kanat-Maymon, & Roth, 2005;Reeve & Jang, 2006). However, the effects of these practices are complex; they seem to interact with oneanother and with a variety of personal and situational factors to shape students’ outcomes. For example, the

212

strategy of providing choices in the classroom may be optimally effective when choices are administeredwithout pressure and allow students to self-regulate, are not overwhelming in number or complexity, and areadministered to individuals who feel competent or interested in the task or who ascribe to an upper-classWestern cultural sensibility (Patall, 2013; Patall, Dacy, & Han, 2014; see Patall, 2012 for review). Alongsimilar lines, research suggests that competence and autonomy work synergistically such that motivation andlearning are optimized when both are supported, despite lay views that autonomy-support and competence-support (i.e., structure) are incompatible (e.g., Jang, Reeve & Deci, 2010; Reeve, 2009; Vansteenkiste et al.,2012). Finally, recent research focuses on the previously neglected need of relatedness, showing that non-controlling relatedness support and satisfaction are linked with engagement, learning, and achievement (e.g.,Furrer & Skinner, 2003; Roorda, Koomen, Spilt, & Ooart, 2011; Ryan & Patrick, 2001).

Achievement Goal Theory

Theoretical overview. Achievement goal theory remains one of the most prominent motivation theories ineducational psychology. Achievement goal theory proposes that there are two primary reasons or underlyingpurposes related to individuals’ engagement in achievement-related activities: mastery, with a focus ondeveloping competence, and performance, with a focus on demonstrating competence (Ames, 1992; Dweck &Leggett, 1988; Maehr & Midgley, 1991; Nicholls, 1984). The trichotomous model (Elliot, 1999) furtherdifferentiates performance goals into approach goals, with a focus on appearing competent, and avoidancegoals, with a focus on avoiding appearing incompetent. The 2 × 2 model (Elliot & McGregor, 2001) extendsthe approach-avoidance distinction to mastery, such that one can approach the goal to develop competence(mastery-approach) or avoid declining competence or not fulfilling one’s potential (mastery-avoidance),although mastery-avoidance goals have not been widely studied.

Goal orientations represent a general framework through which students interpret and react to achievementsettings, resulting in varying patterns of affect, cognitions, and behaviors (Dweck & Leggett, 1988). Goalorientations are shaped both by the context, such as underlying goal structure of the classroom or school(Ames, 1992; Maehr & Midgley, 1991; see Urdan, 2010 for a review), as well as by personal antecedents, suchas motives (Elliot, 1999) and theories of intelligence (Cury, Elliot, Da Fonseca, & Moller, 2006; Dweck,1999).

Research related to engagement, learning, and achievement. Research has established the benefits of mastery-approach goals and detriments of performance-avoidance goals across educational outcomes, whereas thefindings for performance-approach goals remain mixed and controversial (see Anderman & Wolters, 2006).While still relatively understudied, research on mastery-avoidance has increased. Mastery-avoidance goals arerelated to negative outcomes such as negative affect, poor study strategies, avoidant behaviors, and lowerachievement (e.g., Bong, 2009; Elliot & McGregor, 2001; Lovejoy & Durik, 2010; see Huang, 2011, 2012;Hulleman, Schrager, Bodmann, & Harackiewicz, 2010 for meta-analyses), though some studies suggest nullor adaptive links to learning strategies (Bong, 2009; Elliot & McGregor, 2001; Madjar, Kaplan, &Weinstock, 2011). Meta-analyses indicate that mastery-avoidance goals are unrelated to positive affect andinterest (Huang, 2011; Hulleman et al., 2010).

An ongoing controversy remains regarding how achievement goals relate to achievement, especially

213

mastery-approach and performance-approach goals. Several recent reviews and meta-analyses sought to clarifythis pattern (Huang, 2012; Hulleman et al., 2010; Linnenbrink-Garcia, Tyson, & Patall, 2008). For example,Hulleman and colleagues (2010) found small significant positive correlations with achievement for mastery-approach (^r = 0.11) and performance-approach (^r = 0.06) goals and negative correlations for mastery-avoidance (^r = –0.12) and performance-avoidance (^r = –0.13) goals. These results were qualified by severalsignificant moderators. For instance, when primarily normative items (e.g., outperforming others) were usedto measure performance-approach goals, the correlation was larger and positive (^r = 0.14); however, whenthe majority of items focused on appearance (e.g., looking smart) and/or evaluation (e.g., demonstratingability), the correlation was negative (^r = –0.14). This suggests that framing performance goals as normativestandards (see Elliot, Murayama, & Pekrun, 2011; Elliot & Thrash, 2001; Senko, Hulleman, &Harackiewicz, 2011) versus broader orientations related to demonstrating competence (Dweck & Leggett,1988; Kaplan & Maehr, 2007) may yield very different patterns, especially for performance goals. Indeed, thisdistinction between standards and orientations is part of a larger ongoing discussion of what constitutes aperformance goal, how it should be measured, whether approach and avoidance forms are distinct, and howlikely performance goals are to emerge in classrooms (see Brophy, 2005; Linnenbrink-Garcia et al., 2012;Senko et al., 2011; Urdan & Mestas, 2006).

Moving beyond variable-centered approaches, recent research has utilized a person-centered approach (e.g.,Daniels et al., 2008; Luo, Paris, Hogan, & Luo, 2011; Tuominen-Soini, Salmela-Aro, & Niemivirta, 2012).This extends earlier work examining interactions among multiple goals (e.g., Pintrich, 2000) to identifynaturally occurring combinations of goals and their relations to achievement. For instance, Luo and colleagues(2011) found that a profile with at least moderate mastery, high performance-approach, but low performance-avoidance goals was most beneficial across a variety of outcomes. This research suggests that it is critical toconsider the relative levels of multiple goals, as doing so may help to further clarify the observed complexity infindings, especially for performance-approach goals.

There is also research examining how the educational context shapes students’ goals (see Urdan, 2010, for areview). Recent work seeks to clarify the interplay between goal structures and social-relational components(e.g., Patrick, Kaplan, & Ryan, 2011; Turner, Gray, Anderman, Dawson, & Anderman, 2013), interactionsbetween goal structures and personal goals in shaping academic outcomes (Lau & Nie, 2008; Linnenbrink,2005; Murayama & Elliot, 2009; Wolters, 2004), the role of teachers’ motivation in shaping the goalstructures they create (Butler, 2012), the relative influence of individual versus shared perceptions of goalstructures on academic outcomes (Karabenick, 2004; Urdan, 2004), as well as goal stability and change bothwithin a single context (Fryer & Elliot, 2007; Muis & Edwards, 2009; Senko & Harackiewicz, 2005) andacross contexts, particularly school transitions (e.g., Paulick, Watermann, & Nückles, 2013; Tuominen-Soiniet al., 2012). This research highlights the importance of considering personal goal orientations embedded inclassroom contexts and the need to examine how personal goal orientations change based on both objectivefeatures of the classroom as well as students’ perceptions of these features. For instance, while personal goalsare generally stable over time, they may shift as a function of feedback or exam performance (Muis &Edwards, 2009; Senko & Harackiewicz, 2005). Moreover, there is some evidence that the goal context maymagnify the relations between personal goals and academic outcomes (Lau & Nie, 2008; Murayama & Elliot,2009), although this is not consistent across all studies (Linnenbrink, 2005; Wolters, 2004).

214

Finally, researchers have expanded achievement goals to focus on the social domain, examining how socialgoal orientations relate to both social and academic outcomes (e.g., Rodkin, Ryan, Jamieson & Wilson, 2013;Ryan & Shim, 2006, 2008). For instance, Ryan and Shim (2006) developed and validated measures of socialgoals focused on developing versus demonstrating social competence and found that social development goalswere associated with social adjustment while social demonstration-avoidance goals were associated withmaladjustment. Social demonstration-approach goals were generally unrelated to social adjustment, althoughmore recent research suggests that there may be both benefits and detriments of these goals (e.g., Rodkin etal., 2013). This expansion to the social realm is one potential way to more fully capture the nature andfunction of goals in school settings.

Attribution Theory

Theoretical overview. Building on early expectancy-value theories and conceptualizations of attributions (e.g.,Atkinson, 1964; Heider, 1958; Rotter, 1966), Weiner’s attribution theory of achievement (e.g., Weiner, 1985,2011; for a review see Graham & Williams, 2009) assumes that people are motivated to understand outcomesthey experience, especially when outcomes are unexpected or negative. In their search to explain an outcome,students may arrive at many possible causal attributions (i.e., ability, effort, luck, or task difficulty) that arethemselves influenced by a variety of factors. These attributions are organized along three underlyingdimensions: locus or the extent to which the cause is internal to the individual (e.g., ability, effort) or external(e.g., luck, task difficulty); stability or the extent that the cause will persist in the future (e.g., aptitude) or istransient (e.g., effort); and controllability or the extent of perceived influence an individual has on the cause(e.g., effort is controllable, luck is uncontrollable). These dimensions are theorized to have differentialimplications for expectancies, values and emotions, and subsequent achievement behavior, with stabilityrelating most directly to expectancies for success and failure, a more internal locus to affective reactions tosuccess and failure (i.e., pride or self-esteem), and controllability to hopefulness, social emotions (i.e., shame,guilt) and help giving (Weiner, 2011).

Research related to engagement, learning, and achievement. Research is generally consistent with hypothesizedpatterns such that attributions are associated with varying emotions, expectancies, and academic functioning(e.g., Liu, Cheng, Chen, & Wu, 2009; Perry, Stupnisky, Daniels, & Haynes, 2008; Shell & Husman, 2008;Wolters, Fan, & Daugherty, 2013), though effort attributions are not consistently more beneficial relative toability attributions, as attribution theorists often predict (e.g., Hsieh & Schallert, 2008; Vispoel & Austin,1995). Current research integrates causal attributions, particularly ability and effort attributions, into varioustheoretical explanations of motivation and achievement. Attributions have an important place in achievementgoal theory and implicit theories about the nature of intelligence (e.g., Blackwell, Trzesniewski, & Dweck,2007; Haynes, Daniels, Stupnisky, Perry, & Hladkyj, 2008; Shell & Husman, 2008; Wolters et al., 2013).This research generally suggests a bidirectional relation between adopting performance goals or a fixed view ofintelligence and using more helpless attributions (i.e., more internal, stable, and uncontrollable causes afterfailure), versus endorsing mastery goals or a malleable view of intelligence and using more adaptiveattributions (i.e., controllable causes of success and unstable causes for failure). Current research also focuseson the links between causal attributions and constructs prominent in other motivation theories such as interest

215

and self-regulation (e.g., Fulmer & Fritjers, 2011; Soric & Palekčic, 2009; Wolters et al., 2013). For example,Fulmer and Fritjers (2011) found that high topic interest for a reading passage buffered adolescents from thenegative effects of excessive challenge, sustaining engagement and preventing attributions regarding the sourceof difficulty.

An additional focus of current attribution research is on how attribution training interventions may be usedto promote a variety of student outcomes, with many intervention studies demonstrating that trainingstudents to think about academic successes as controllable and academic failures as unstable has beneficialeffects (e.g., Good, Aronson, & Inzlicht, 2003; Haynes et al., 2008; Perry, Stupnisky, Hall, Chipperfield, &Weiner, 2010). Given links between implicit theories of intelligence and attributions, interventions that trainstudents to adopt a mindset emphasizing the malleable or controllable nature of intelligence seem to havesimilar beneficial effects (e.g., Blackwell et al., 2007; Good et al., 2003).

Finally, research on attributions has taken several new directions. Current research explores multipleattributions (e.g., Perry et al., 2008, 2010), attributions with a social focus (e.g., Liu et al., 2009; McClure etal., 2011), and the correlates of attributions in new educational contexts or previously unexamined domains(e.g., Hsieh & Schallert, 2008; Perry et al., 2008, 2010), including interpersonal interactions (e.g., Natale,Viljaranta, Lerkkanen, Poikkeus, & Nurmi, 2009; Peterson & Schreiber, 2006, 2012). For example, Petersonand Schreiber (2006) found that college students’ outcome expectations and emotions were more stronglyrelated to effort than ability attributions in the context of a collaborative project. Using a person-centeredapproach, Perry and colleagues (2008) found that compared to students who used other combinations ofattributions for poor performance in college, students using a combination of modifiable internal controllableattributions (low effort, bad strategy) and external uncontrollable attributions that protect self-worth (testdifficulty, poor teaching) demonstrated the most adaptive motivation and goal striving when transitioningfrom high school to college.

Other Theoretical Perspectives

We end this overview of key theories by noting that there are a number of important and influential constructsand theories that are not given adequate attention in our review. While we could not possibly review thetenets and findings of every fruitful motivation theory applicable to achievement contexts, there are manyadditional theories that make meaningful contributions to our understanding of students’ motivation,engagement, and achievement. Many of these theories share and extend ideas and constructs central to thosetheories summarized above. For example, a number of theories related to personal control (e.g., Connell &Wellborn, 1991; Perry, 2003; Skinner, 1995) share commonalities with social cognitive theory, self-determination theory, and attribution theory. Control-value theory (Pekrun, 2006) focuses on the role ofperceived control and value appraisals in achievement emotions and performance, highlighting constructs andideas that overlap with expectancy-value theory, attribution theory, and achievement goal theory.

Many researchers study self-processes, including possible selves (e.g., Oyserman, Bybee, & Terry, 2006),self-concept (e.g., Harter, 1998; Marsh & O’Mara, 2008), self-worth (e.g., Covington, 1992), and beliefsabout ability and intelligence (e.g., Dweck, 2006), that guide, direct, and motivate behaviors in achievementcontexts and overlap with competence, control, and goal-related concepts central to many of the theoriesdescribed in this chapter. In particular, and as alluded to previously, a great deal of research on self-theories or

216

mindsets (see Dweck & Grant, 2008 for a review) suggests that self-theories about the malleability of personalattributes, including personality and intelligence, may underlie several of the motivation constructs previouslydiscussed. Research suggests that those with a more malleable versus fixed theory of self-attributes are morelikely to adopt mastery goals, maintain intrinsic interest, persist in the face of challenge, demonstrate higheracademic performance, make more adaptive attributions for outcomes, and have greater overall psychologicalwell-being.

Some theorists have studied similar constructs from different time perspectives. For example, flow theory(Shernoff & Csikszentmihalyi, 2009) focuses on in-the-moment or transient subjective flow experiences thatare much like the intrinsic motivation or interest constructs. Research on future time perspectives (e.g.,Husman & Lens, 1999) focuses on value for educational activities in the future rather than in the present.

Finally, while the theories we reviewed assume people consciously engage in goal-directed action and thenevaluate subsequent affective, cognitive, and performance consequences, there is substantial evidence thatmuch motivation is unconscious and people are often unaware of what guides their moods, thoughts, andbehavior (see Aarts & Custers, 2012; Chartrand & Bargh, 2002). For example, Bargh, Gollwitzer, Lee Chai,Barndollar, and Trotschel (2001) found participants who were primed with an achievement goal by beingunobtrusively exposed to words such as “strive” and “succeed” outperformed and persisted longer on ananagram task compared to those not primed with achievement words. While educational research has nottypically included unconscious motivation, the last two decades of psychological research suggest it mayprovide powerful explanations in educational contexts.

217

Integrating Across Theoretical Perspectives

A clear strength of the theory-driven research conducted in the second half of the twentieth century is that itprovided the groundwork for many advances in our understanding of motivational functioning in theclassroom. However, with this emphasis on theory building came a tendency to conduct research based withina single theoretical tradition. For more than the past decade, however, this trend has been changing.Researchers are now considering, both empirically and theoretically, how multiple forms of motivation frommultiple theories combine to shape engagement and learning. This shift is quite important and reflects thecommon understanding that educational outcomes, including achievement, may be multiply determined.Below, we highlight three complementary approaches to theory integration.

First, researchers have examined the contribution of multiple motivational constructs to students’engagement and learning using a traditional variable-centered approach (e.g., Ciani, Sheldon, Hilpert, &Easter, 2011; Hulleman, Durik, Schweigert, & Harackiewicz, 2008; Liem, Lau, & Nie, 2008; Wolters et al.,2013). In these studies, researchers often explore the relative or unique contribution of each variable, aftercontrolling for other forms of motivation, and consider interactions among personal and contextualmotivational variables in relation to educational outcomes. For example, Ciani and colleagues (2011) exploredconcepts from self-determination and achievement goal theories; they found that psychological needsatisfaction in life was linked with adopting academic mastery goals via autonomous motivation in class, andautonomy support slowed the decline in mastery goals over the course of a semester among undergraduatestudents. In an integration of expectancy-value, interest, and goal orientation theories, Hulleman andcolleagues (2008) found that college and high-school students’ intrinsic and utility value for a course oractivity mediated the effects of mastery-approach goals on both subsequent interest in the course and finalgrade; performance-approach goals and utility value also predicted final grades, though values did not mediatethese effects.

Going further, researchers also consider the joint and interactive effects of motivation, emotion, andcognitive variables in a multidisciplinary fashion to create more complete models of learning and engagement.For example, in their model of domain learning, Alexander and colleagues (e.g., Alexander, Murphy, Woods,Duhon, & Parker, 1997; Murphy & Alexander, 2002) explored the interplay between knowledge, interest,and strategic processing in students’ paths to developing domain expertise. Likewise, research conducted byPekrun and others (see Pekrun & Perry, 2014) has examined the links among emotion, motivation, self-regulation and learning strategies, cognitive resources, and academic achievement. In general, this variable-centered approach is useful for providing a more complete picture of the function of motivation in school,sometimes in the context of non-motivational factors.

Second, in the past decade, researchers have attempted to integrate motivation theories by examiningmotivational profiles of individuals across multiple forms of motivation (e.g., Braten & Olaussen, 2005;Conley, 2012; Lau & Roeser, 2008; Shell & Husman, 2008). Rather than focusing on how one particularvariable functions, presumably across all individuals and in isolation, this person-centered approach allowsresearchers to identify particularly adaptive (or maladaptive) combinations of motivation and explore howthese profiles function in the classroom. For instance, Conley (2012) created motivational profiles usingconstructs from achievement goal and expectancy-value theories. She found that combining variables from

218

multiple theories in seven profiles was critical for predicting affect and achievement; neither achievement goalsnor subjective task value explained the pattern of findings alone.

In one of the most comprehensive approaches to date, Shell and Husman (2008) examined constructs fromsocial cognitive, expectancy-value, attribution, and achievement goal theories, as well as affect and self-regulatory behaviors, to identify five distinct groupings of variables along three canonical dimensions. Forinstance, they identified a highly motivated and strategic learner dimension. The motivation coefficientsincluded high competence beliefs, positive affect, attributions to effort, and mastery and performance-approach goals and were linked with highly strategic self-regulatory behavior coefficients. Another dimensionreflected more of an intrinsically motivated, mastery, high-competency focus tied with the use of knowledge-building strategies, but not general cognitive and metacognitive strategies or high study effort. Thesedimensions, along with others they identified, suggest that there may be multiple adaptive and maladaptivegroupings of motivational and self-regulatory behaviors.

Third, motivation researchers are integrating across theories to consider how multiple forms of motivationemerge from and are supported by the educational context (e.g., Guthrie, Klauda, & Ho, 2013; Nolen, 2007;Turner, Warzon, & Christensen, 2011). For instance, Turner and colleagues (2011) synthesized motivationtheories to identify and implement “best practices” designed to support multiple forms of adaptive motivation.The concept-oriented reading program developed by Guthrie and Wigfield goes further by examining how anintervention based on multiple motivational theories shapes patterns of motivation and engagement (e.g.,Guthrie et al., 2013). Other research takes a more situated approach to investigate how multiple forms ofmotivation develop. For example, Nolen (2007) used a grounded-theory approach to examine thedevelopment of elementary students’ motivation to read and write in a mixed-methods longitudinal study.While Nolen’s primary focus was interest development, she identified shifts in multiple forms of reading andwriting motivation (e.g., interest, mastery, ego concerns, reading to learn) and considered how the educationalcontext related to varying motivational patterns. Integrated approaches such as these are critical in thetranslation of motivation research into coherent and useful recommendations for classroom practice.

219

Future Directions

We close this chapter by highlighting several important or promising avenues for future research. First, ouroverview of the major theoretical models highlights the breadth of motivation constructs; however there is alsoa great deal of similarity among constructs such as those related to value (e.g., intrinsic motivation, task value,interest) and competence (e.g., self-concept, self-efficacy, expectancies), and the field suffers from the varietyof terms used for seemingly similar constructs (Murphy & Alexander, 2000). Thus, it is critical thatresearchers not only carefully align conceptual definitions with measurement instruments, but also assesswhere constructs can be combined and where nuanced differences are needed. Likewise, readers should payspecial attention to conceptual and operational definitions when interpreting results.

Second, current research integrating motivation perspectives is particularly promising, both forunderstanding how motivation relates to engagement and learning and for supporting multiple adaptive formsof motivation in school settings (see also Chapter 12, this volume). In pursuing this work, we urge researchersto reflect upon what it means theoretically to integrate constructs and whether a more unified, cohesivetheoretical approach is possible (see Ford, 1992). However, in doing so, there is a need for parsimony, ascomplex theories will be unlikely to be widely adopted and have limited utility for informing practice.

Third, we urge researchers to continue to investigate underlying psychological processes and mechanismsand to more carefully consider how and why classroom contexts shape motivation. For instance, whileresearch on situational interest has progressed in understanding contextual supports for situational interest andin using situational interest to predict individual interest, we know very little about the processes throughwhich situational interest develops into individual interest. Similarly, within achievement goal theory, a clearerunderstanding of the psychological processes that shift as a function of goal structures is needed (see O’Keefe,Ben-Eliyahu, & Linnenbrink-Garcia, 2013). Similar suggestions can be made regarding the role ofpsychological needs in research on the links between external events and academic outcomes or themechanisms by which attribution retraining influences achievement.

Fourth, motivation researchers may need to look beyond social cognitive theories to consider socioculturaland situated approaches, which place a greater emphasis on understanding the person in the context (forreviews, see Nolen & Ward, 2008; Perry, Turner, & Meyer, 2006). Sociocultural and situated approaches maybe especially useful in understanding how motivation develops and functions in educational settings (seeNolen, 2007) and for investigating the role of culture in motivation (Zusho & Clayton, 2011). Moreover,these approaches may help motivational researchers work with educators to provide guidelines that are morerealistic and useful for supporting motivation in classrooms by acknowledging the complexities of classroomsand better representing how multiple contextual components synergistically support motivation.

Fifth, we urge researchers to more carefully consider culture in the study of motivation. While this call isnot new (Graham, 1994), it remains understudied. There are a number of recent theoretical reviews thatthoughtfully discuss issues of culture, race, and ethnicity within the context of motivation research (e.g.,Graham & Hudley, 2005; Kumar & Maehr, 2010; Zusho & Clayton, 2011). As these authors articulate,research on culture must move beyond simply identifying racial/ethnic or country-level differences to examinehow meaningful conceptualizations of culture shape the nature of and variations in motivational phenomena.

Finally, while a variety of methods are used to study motivation, there is a heavy reliance on self-report

220

instruments. Self-reports are useful for gaining access to social cognitive constructs like motivational self-beliefs; however, they have a number of drawbacks. Indeed, several studies suggest that students may not beinterpreting items as intended (Karabenick et al., 2007; Koskey, Karabenick, Woolley, Bonney, & Dever,2010; Urdan & Mestas, 2006). Thus, we urge researchers to continue to refine self-report measures while alsoemploying other possible assessment techniques. Behavioral, observational, neuroimaging, facial recognition,and implicit techniques are among many methods that could be used to study motivation. For instance, Zhouand Winne (2012) employed goal traces (behavioral indicators operationalized as tags participants applied toselections of text and hyperlinks they clicked in an article) to collect in-the-moment goal orientations whilestudents studied a passage. The goal traces and self-reported goals were correlated, but only goal tracessignificantly predicted test performance. Classroom observations are effectively employed to examine teachers’motivating practices and students’ engagement (e.g., Jang et al., 2010; Turner et al., 1998), but are stillrelatively underutilized. Finally, recent efforts demonstrate that complex human motivational constructs canbe understood through neuroscience methods (see Reeve & Lee, 2012, for a review).

221

Conclusion

Research on motivation in educational settings continues to be a vibrant and productive area of study. As wehave noted in this chapter, a great deal of progress in the study of motivation has occurred within the lastdecade. Researchers have continued to refine and advance our theoretical understanding of motivation,examined how motivation relates to engagement and learning, and explored how classroom contexts supportit. One noteworthy advance we observed is the move beyond variable-centered analyses to consider howmultiple forms of motivation function synergistically within individuals. We see this as quite fruitful,especially as researchers attempt to develop integrative approaches that move beyond the major theoriesoutlined in this chapter. Relatedly, the use of situated or sociocultural perspectives may be particularly usefulfor understanding how motivation develops and is supported by educational contexts. Moreover, the increaseduse of diverse methods such as behavioral indicators of motivation and experience sampling designs mayfurther clarify the processes by which motivation shapes academic outcomes. We are encouraged that theresearch on motivation continues to evolve and look forward to many more decades of productive motivationalresearch.

222

2.3.

Notes

We refer to task value or value rather than individual components when various components of value were examined across similar studies orthe primary research assessed the construct as a whole.Also known as “catch” (Mitchell, 1993) and “emerging situational interest” (Krapp & Prenzel, 2011).Also known as “hold” (Mitchell, 1993) or “stabilized situational interest” (Krapp & Prenzel, 2011).

223

References

Aarts, H., & Custers, R. (2012). Unconscious goal pursuit: Nonconscious goal regulation and motivation. In R. Ryan (Ed.), Oxford handbook ofhuman motivation (pp. 232–247). New York, NY: Oxford University Press.

Alexander, P. A., Murphy, P. K., Woods, B. S., Duhon, K. E., & Parker, D. (1997). College instruction and concomitant changes in students’knowledge, interest, and strategy use: A study of domain learning. Contemporary Educational Psychology, 22, 125–146.

Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84, 261–271.Anderman, E. M., & Wolters, C. A. (2006). Goals, values, and affect: Influences on student motivation. In P. A. Alexander & P. H. Winne

(Eds.), Handbook of educational psychology (pp. 369–389). Mahwah, NJ: Lawrence Erlbaum.Archambault, I., Eccles, J. S., & Vida, M. N. (2010). Ability self-concepts and subjective value in literacy: Joint trajectories from grades 1

through 12. Journal of Educational Psychology, 102, 804–816.Assor, A., Kaplan, H., Kanat-Maymon, Y., & Roth, G. (2005). Directly controlling teachers’ behaviors as predictors of poor motivation and

engagement in girls and boys: The role of anger and anxiety. Learning and Instruction, 15, 397–413.Atkinson, J. W. (1964). An introduction to motivation. Princeton, NJ: Van Nostrand.Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.Bandura, A. (2006). Toward a psychology of human agency. Perspectives on Psychological Science, 1, 164–180.Bargh, J. A., Gollwitzer, P. M., Lee Chai, A., Barndollar, K., & Trotschel, R. (2001). The automated will: Nonconscious activation and pursuit

of behavioral goals. Journal of Personality and Social Psychology, 81, 1014–1027.Battle, A., & Wigfield, A. (2003). College women’s value orientations toward family, career, and graduate school. Journal of Vocational Behavior,

62, 56–75.Berlyne, D. E. (1960). Conflict, arousal, and curiosity. McGraw-Hill Series in Psychology. New York, NY: McGraw-Hill.Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent

transition: A longitudinal study and an intervention. Child Development, 78, 246–263.Bong, M. (2009). Age-related differences in achievement goal differentiation. Journal of Educational Psychology, 101, 879–896.Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15,

1–40.Braten, I., & Olaussen, B. S. (2005). Profiling individual differences in student motivation: A longitudinal cluster-analytic study in different

academic contexts. Contemporary Educational Psychology, 30, 359–396.Brophy, J. (2005). Goal theorists should move on from performance goals. Educational Psychologist, 40, 167–176.Burton, K. D., Lydon, J. E., D’Alessandro, D. U., & Koestner, R. (2006). The differential effects of intrinsic and identified motivation on well-

being and performance: Prospective, experimental, and implicit approaches to self-determination theory. Journal of Personality and SocialPsychology, 91, 750–762.

Butler, R. (2012). Striving to connect: Extending an achievement goal approach to teacher motivation to include relational goals for teaching.Journal of Educational Psychology, 104, 726–742.

Chartrand, T. L., & Bargh, J. A. (2002). Nonconscious motivations: Their activation, operation, and consequences. In A. Tesser, D. A. Stapel,& J. V. Wood (Eds.), Self and motivation: Emerging psychological perspectives (pp. 13–41). Washington, DC: American PsychologicalAssociation.

Chen, J. A., & Usher, E. (2013). Profiles of the sources of science self-efficacy. Learning and Individual Differences, 24, 11–21.Chen, P. P. (2003). Exploring the accuracy and predictability of the self-efficacy beliefs of seventh-grade mathematics students. Learning and

Individual Differences, 14, 79–92.Chow, A., Eccles, J. S., & Salmela-Aro, K. (2012). Task value profiles across subjects and aspirations to physical and IT-related sciences in the

United States and Finland. Developmental Psychology, 48, 1612–1628.Ciani, K. D., Sheldon, K. M., Hilpert, J. C., & Easter, M. A. (2011). Antecedents and trajectories of achievement goals: A self-determination

theory perspective. British Journal of Educational Psychology, 81, 223–243.Conley, A. M. (2012). Patterns of motivation beliefs: Combining achievement goal and expectancy-value perspectives. Journal of Educational

Psychology, 104, 32–47.Connell, J. P., & Wellborn, J. G. (1991). Competence, autonomy, and relatedness: A motivational analysis of self-system processes. In R.

Gunnar & L. A. Sroufe (Eds.), Minnesota symposia on child psychology (Vol. 23, pp. 43–77). Hillsdale, NJ: Lawrence Erlbaum Associates.Corpus, J. H., McClintic-Gilbert, M. S., & Hayenga, A. O. (2009). Within-year changes in children’s intrinsic and extrinsic motivational

orientations: Contextual predictors and academic outcomes. Contemporary Educational Psychology, 34, 154–166.Covington, M. V. (1992). Making the grade: A self-worth perspective on motivation and school reform. New York, NY: Cambridge University Press.

224

Cury, F., Elliot, A. J., Da Fonseca, D., & Moller, A. C. (2006). The social-cognitive model of achievement motivation and the 2 × 2achievement goal framework. Journal of Personality and Social Psychology, 90, 666–679.

Daniels, L. M., Haynes, T. L., Stupnisky, R. H., Perry, R. P., Newall, N. E., & Pekrun, R. (2008). Individual differences in achievement goals:A longitudinal study of cognitive, emotional, and achievement outcomes. Contemporary Educational Psychology, 33, 584–608.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum.Denissen, J. J., Zarrett, N. R., & Eccles, J. S. (2007). I like to do it, I’m able, and I know I am: Longitudinal couplings between domain-specific

achievement, self-concept, and interest. Child Development, 78, 430–447.Dewey, J. (1913). Interest and effort in education. Boston, MA: Riverside.Durik, A. M., & Harackiewicz, J. M. (2007). Different strokes for different folks: How individual interest moderates the effects of situational

factors on task interest. Journal of Educational Psychology, 99, 597–610.Durik, A. M., Vida, M., & Eccles, J. S. (2006). Task values and ability beliefs as predictors of high school literacy choices: A developmental

analysis. Journal of Educational Psychology, 98, 382–393.Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. Philadelphia: Taylor & Francis.Dweck, C. S. (2006). Mindset. New York: Random House.Dweck, C. S., & Grant, H. (2008). Self-theories, goals, and meanings. In J. Y. Shah & W. L. Gardner (Eds.), Handbook of motivation science

(pp. 405–416). New York, NY: Guilford.Dweck, C. S., & Leggett, E. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256–273.Eccles, J. (2005). Subjective task value and the Eccles et al. model of achievement-related choices. In A. J. Elliot & C. S. Dweck (Eds.),

Handbook of competence and motivation (pp. 105–121). New York, NY: Guilford Publications.Eccles, J. S., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., . (1983). Expectancies, values, and academic behaviors. In J.

T. Spence (Ed.), Achievement and achievement motives (pp. 75–146). San Francisco: Freeman.Eccles, J. S., & Wigfield, A. (1995). In the mind of the actor: The structure of adolescents’ achievement task values and expectancy-related

beliefs. Journal of Personality and Social Psychology, 21, 215–225.Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals. Educational Psychologist, 34, 169–189.Elliot, A. J., & McGregor, H. A. (2001). A 2 × 2 achievement goal framework. Journal of Personality and Social Psychology, 80, 501–519.Elliot, A. J., Murayama, K., & Pekrun, R. (2011). A 3 × 2 achievement goal model. Journal of Educational Psychology, 103, 632–648.Elliot, A. J., & Thrash, T. M. (2001). Achievement goals and the hierarchical model of achievement motivation. Educational Psychology Review,

13, 139–156.Ferguson, Y. L., Kasser, T., & Jahng, S. (2010). Differences in life satisfaction and school satisfaction among adolescents from three nations:

The role of perceived autonomy support. Journal of Research on Adolescence, 21, 649–661.Ford, M. (1992). Motivating humans: Goals, emotions, and personal agency beliefs. Newbury Park, CA: Sage.Frome, P. M., Alfeld, C. J., Eccles, J. S., & Barber, B. L. (2006). Why don’t they want a male-dominated job? An investigation of young

women who changed their occupational aspirations. Educational Research and Evaluation, 12, 359–372.Fryer, J. W., & Elliot, A. J. (2007). Stability and change in achievement goals. Journal of Educational Psychology, 99, 700–714.Fulmer, S. M., & Frijters, J. C. (2011). Motivation during an excessively challenging reading task: The buffering role of relative topic interest.

Journal of Experimental Education, 79, 185–208.Furrer, C., & Skinner, E. (2003). Sense of relatedness as a factor in children’s academic engagement and performance. Journal of Educational

Psychology, 95, 148–162.Good, C., Aronson, J., & Inzlicht, M. (2003). Improving adolescents’ standardized test performance: An intervention to reduce the effects of

stereotype threat. Journal of Applied Developmental Psychology, 24, 645–662.Graham, S. (1994). Motivation in African Americans. Review of Educational Research, 64, 55–117.Graham, S., & Hudley, C. (2005). Race and ethnicity in the study of motivation and competence. In A. J. Elliot & C. S. Dweck (Eds.),

Handbook of competence and motivation (pp. 392–413). New York, NY: Guilford Publications.Graham, S., & Williams, C. (2009). An attributional approach to motivation in school. In K. R. Wentzel & A. Wigfield (Eds.), Handbook of

motivation at school (pp. 11–33). New York, NY: Routledge.Guthrie, J. T., Klauda, S. L., & Ho, A. N. (2013). Modeling the relationships among reading instruction, motivation, engagement, and

achievement for adolescents. Reading Research Quarterly, 48, 9–26.Harackiewicz, J. M., Durik, A. M., Barron, K. E., Linnenbrink-Garcia, L., & Tauer, J. M. (2008). The role of achievement goals in the

development of interest: Reciprocal relations between achievement goals, interest, and performance. Journal of Educational Psychology, 100,105–122.

Harter, S. (1998). The development of self-representations. In W. Damon (Series Ed.) & N. Eisenberg (Vol. Ed.), Handbook of child psychology(Vol. 3, 5th ed., pp. 553–617). New York: John Wiley.

225

Hayenga, A. O., & Corpus, J. H. (2010). Profiles of intrinsic and extrinsic motivations: A person-centered approach to motivation andachievement in middle school. Motivation and Emotion, 34, 371–383.

Haynes, T. L., Daniels, L. M., Stupnisky, R. H., Perry, R. P., & Hladkyj, S. (2008). The effect of attributional retraining on mastery andperformance motivation among first-year college students. Basic and Applied Social Psychology, 30, 198–207.

Heider, F. (1958). The psychology of interpersonal relations. Hillsdale, NJ: Lawrence Erlbaum Associates.Henderson, B. B., Marx, M. H., & Kim, Y. C. (1999). Academic interests and perceived competence in American, Japanese, and Korean

children. Journal of Cross-Cultural Psychology, 30, 32–50.Hidi, S. (2001). Interest, reading, and learning: Theoretical and practical considerations. Educational Psychology Review, 13, 191–209.Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41, 111–127.Hsieh, P. H. P., & Schallert, D. L. (2008). Implications from self-efficacy and attribution theories for an understanding of undergraduates’

motivation in a foreign language course. Contemporary Educational Psychology, 33, 513–532.Huang, C. (2011). Achievement goals and achievement emotions: A meta-analysis. Educational Psychology Review, 23, 359–388.Huang, C. (2012). Discriminant and criterion-related validity of achievement goals in predicting academic achievement: A meta-analysis.

Journal of Educational Psychology, 104, 48–73.Hulleman, C. S., Durik, A. M., Schweigert, S. B., & Harackiewicz, J. M. (2008). Task values, achievement goals, and interest: An integrative

analysis. Journal of Educational Psychology, 100, 398–416.Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326, 1410–1412.Hulleman, C. S., Schrager, S. M., Bodmann, S. M., & Harackiewicz, J. M. (2010). A meta-analytic review of achievement goal measures:

Different labels for the same constructs or different constructs with similar labels? Psychological Bulletin, 136, 422–449.Husman, J., & Lens, W. (1999). The role of the future in the study of motivation. Educational Psychologist, 34, 113–125.Jacobs, J. E., Lanza, S., Osgood, D. W., Eccles, J. S., & Wigfield, A. (2002). Changes in children’s self-competence and values: Gender and

domain differences across grades one through twelve. Child Development, 73, 509–527.James, W. (1890). The principles of psychology. London: Macmillan.Jang, H., Reeve, J. & Deci, E. L. (2010). Engaging students in learning activities: It is not autonomy support or structure but autonomy support

and structure. Journal of Educational Psychology, 102, 588–600.Jang, H., Reeve, J., Ryan, R. M., & Kim, A. (2009). Can self-determination theory explain what underlies the productive, satisfying learning

experiences of collectivistically oriented Korean students? Journal of Educational Psychology, 101, 644–661.Kaplan, A., & Maehr, M. L. (2007). The contributions and prospects of goal orientation theory. Educational Psychology Review, 19, 141–184.Karabenick, S. A. (2004). Perceived achievement goal structure and college student help seeking. Journal of Educational Psychology, 96, 569–581.Karabenick, S. A., Woolley, M. E., Friedel, J. M., Ammon, B. V., Blazevski, J., Bonney, C. R., De Groot, E., Gilbert, M., Musu, L., Kempler,

T., & Kelly, K. L. (2007). Cognitive processing of self-report items in educational research: Do they think what we mean? EducationalPsychologist, 42, 139–151.

Klassen, R. (2007). Using predictions to learn about the self-efficacy of early adolescents with and without learning disabilities. ContemporaryEducational Psychology, 32, 173–187.

Klassen, R., & Krawchuk, L. (2009). Collective motivation beliefs of early adolescents working in small groups. Journal of School Psychology, 47,101–120.

Klassen, R. M., Tze, V. M. C., Betts, S. M., & Gordon, K. A. (2011). Teacher efficacy research 1998-2009: Signs of progress or unfulfilledpromise? Educational Psychology Review, 23, 21–43.

Klassen, R. M., & Usher, E. L. (2010). Self-efficacy in educational settings: Recent research and emerging directions. In T. C. Urdan & S. A.Karabenick (Eds.), Advances in motivation and achievement: Vol. 16A. The decade ahead: Theoretical perspectives on motivation and achievement(pp. 1–33). Bingley, UK: Emerald Publishing Group.

Koskey, K. L., Karabenick, S. A., Woolley, M. E., Bonney, C. R., & Dever, B. V. (2010). Cognitive validity of students’ self-reports ofclassroom mastery goal structure: What students are thinking and why it matters. Contemporary Educational Psychology, 35, 254–263.

Krapp, A. (2002). Structural and dynamic aspects of interest development: Theoretical considerations from an ontogenetic perspective. Learningand Instruction, 12, 383–409.

Krapp, A., & Prenzel, M. (2011). Research on interest in science: Theories, methods, and findings. International Journal of Science Education, 33,27–50.

Kumar, R., & Maehr, M. L. (2010). Schooling, cultural diversity, and student motivation. In J. Meece, & J. Eccles (Eds.) Handbook of researchon schools, schooling, and human development (pp. 308–324). New York, NY: Routledge.

Lau, S., & Nie, Y. (2008). Interplay between personal goals and classroom goal structures in predicting student outcomes: A multilevel analysisof person–context interactions. Journal of Educational Psychology, 100, 15–29.

Lau, S., & Roeser, R. W. (2008). Cognitive abilities and motivational processes in science achievement and engagement: A person-centered

226

analysis. Learning and Individual Differences, 18, 497–504.Lepper, M. R., Corpus, J. H., & Iyengar, S. S. (2005). Intrinsic and extrinsic motivational orientations in the classroom: Age differences and

academic correlates. Journal of Educational Psychology, 97, 184–196.Liem, A. D., Lau, S., & Nie, Y. (2008). The role of self-efficacy, task value, and achievement goals in predicting learning strategies, task

disengagement, peer relationship, and achievement outcome. Contemporary Educational Psychology, 33, 486–512.Linnenbrink, E. A. (2005). The dilemma of performance-approach goals: The use of multiple goal contexts to promote students’ motivation

and learning. Journal of Educational Psychology, 97, 197–213.Linnenbrink-Garcia, L., Middleton, M. J., Ciani, K. D., Easter, M. A., O’Keefe, P. A., & Zusho, A. (2012). The strength of the relation

between performance-approach and performance-avoidance goal orientations: Theoretical, methodological, and instructional implications.Educational Psychologist, 47, 281–301.

Linnenbrink-Garcia, L., Patall, E. A., & Messersmith, E. E. (2013). Antecedents and consequences of situational interest. British Journal ofEducational Psychology, 83, 591–614.

Linnenbrink-Garcia, L., Tyson, D. F., & Patall, E. A. (2008). When are achievement goal orientations beneficial for academic achievement? Acloser look at moderating factors. International Review of Social Psychology, 21, 19–70.

Liu, K. S., Cheng, Y. Y., Chen, Y. L., & Wu, Y. Y. (2009). Longitudinal effects of educational expectations and achievement attributions onadolescents’ academic achievements. Adolescence, 44, 911–924.

Lovejoy, C. M., & Durik, A. M. (2010). Self-handicapping: The interplay between self-set and assigned achievement goals. Motivation andEmotion, 34, 242–252.

Luo, W., Paris, S. G., Hogan, D., & Luo, Z. (2011). Do performance goals promote learning? A pattern analysis of Singapore students’achievement goals. Contemporary Educational Psychology, 36, 165–176.

Madjar, N., Kaplan, A., & Weinstock, M. (2011). Clarifying mastery-avoidance goals in high school: Distinguishing between intrapersonal andtask-based standards of competence. Contemporary Educational Psychology, 36, 268–279.

Maehr, M. L., & Midgley, C. (1991). Enhancing student motivation: A schoolwide approach. Educational Psychologist, 26, 399–427.Marsh, H. W., & O’Mara, A. J. (2008). Self-concept is as multidisciplinary as it is multidimensional: A review of theory, measurement, and

practice in self-concept research. In H. W. Marsh, R. G. Craven, & D. M. McInerney (Eds.). Self-processes, learning, and enabling humanpotential: Dynamic new approaches (Vol. 3, pp. 87–118). Charlotte, NC: Information Age Publishing.

McClure, J., Meyer, L. H., Garisch, J., Fischer, R., Weir, K. F., & Walkey, F. H. (2011). Students’ attributions for their best and worst marks:Do they relate to achievement? Contemporary Educational Psychology, 36, 71–81.

Miller, R. B., Greene, B. A., Montalvo, G. P., Ravindran, B., & Nichols, J. D. (1996). Engagement in academic work: The role of learninggoals, future consequences, pleasing others, and perceived ability. Contemporary Educational Psychology, 21, 388–422.

Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school mathematics classroom. Journal of EducationalPsychology, 85, 424–436.

Muis, K. R., & Edwards, O. (2009). Examining the stability of achievement goal orientation. Contemporary Educational Psychology, 34, 265–277.

Murayama, K., & Elliot, A. J. (2009). The joint influence of personal achievement goals and classroom goal structures on achievement-relevantoutcomes. Journal of Educational Psychology, 101, 432–447.

Murphy, K., & Alexander, P. (2000) A motivated exploration of motivation terminology. Contemporary Educational Psychology, 25, 3–53.Murphy, P. K., & Alexander, P. A. (2002). What counts? The predictive powers of subject-matter knowledge, strategic processing, and interest

in domain-specific performance. Journal of Experimental Education, 70, 197–214.Nagengast, B., Marsh, H. W., Scalas, L. F., Xu, M. K., Hau, K. T., & Trautwein, U. (2011). Who took the “×” out of expectancy-value theory?

A psychological mystery, a substantive-methodological synergy, and a cross-national generalization. Psychological Science, 22, 1058–1066.Natale, K., Viljaranta, J., Lerkkanen, M. K., Poikkeus, A. M., & Nurmi, J. E. (2009). Cross-lagged associations between kindergarten teachers’

causal attributions and children’s task motivation and performance in reading. Educational Psychology, 29, 603–619.Nicholls, J. G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological

Review, 91, 328–346.Nolen, S. B. (2007). Young children’s motivation to read and write: Development in social contexts. Cognition & Instruction, 25, 219–270.Nolen, S. B., & Ward, C. J. (2008). Sociocultural and situative research on motivation. In M. Maehr, S. Karabenick, & T. Urdan (Eds.), Social

psychological perspective on motivation and achievement (pp. 428–460). London, UK: Emerald Group.O’Keefe, P., Ben-Eliyahu, A., & Linnenbrink-Garcia, L. (2013). Shaping achievement goal orientations in a mastery-structured environment

and concomitant changes in related contingencies of self-worth. Motivation and Emotion, 37, 50–64.Otis, N., Grouzet, F. M. E., & Pelletier, L. G. (2005). Latent motivational change in an academic setting: A 3-year longitudinal study. Journal

of Educational Psychology, 97, 170–183.

227

Oyserman, D., Bybee, D., & Terry, K. (2006). Possible selves and academic outcomes: How and when possible selves impel action. Journal ofPersonality and Social Psychology, 91, 188–204.

Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66, 543–578.Palmer, D. H. (2009). Student interest generated during an inquiry skills lesson. Journal of Research in Science Teaching, 46, 147–165.Patall, E. A. (2012). The motivational complexity of choosing: A review of theory and research. In R. Ryan (Ed.), Oxford handbook of human

motivation (pp. 249–279). New York, NY: Oxford University Press.Patall, E. A. (2013). Constructing motivation through choice, interest, and interestingness. Journal of Educational Psychology, 105, 522–534.Patall, E. A., Dacy, B. S., & Han, C. (2014). The role of competence in the effects of choice on motivation. Journal of Experimental Social

Psychology, 50, 27–44.Patrick, H., Kaplan, A., & Ryan, A. M. (2011). Positive classroom motivational environments: Convergence between mastery goal structure

and classroom social climate. Journal of Educational Psychology, 103, 367–382.Paulick, I., Watermann, R., & Nückles, M. (2013). Achievement goals and school achievement: The transition to different school tracks in

secondary school. Contemporary Educational Psychology, 38, 75–86.Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and

practice. Educational Psychology Review, 18, 315–341.Pekrun, R., & Perry, R. P. (2014). Control-value theory of achievement emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), Handbook of

emotions in education (pp. 120–141). New York: Routledge.Perez, T., Cromley, J. G., & Kaplan, A. (2014). The role of identity development, values, and costs in college STEM retention. Journal of

Educational Psychology, 106, 315–329.Perry, N. E., Turner, J. C., & Meyer, D. K. (2006). Classrooms as contexts for motivating learning. In P. A. Alexander & P. H. Winne (Eds.),

Handbook of educational psychology (2nd ed., pp. 327–348). Mahwah, NJ: Lawrence Erlbaum.Perry, R. P. (2003). Perceived (academic) control and causal thinking in achievement settings. Canadian Psychology, 44, 312–331.Perry, R. P., Stupnisky, R. H., Daniels, L. M., & Haynes, T. L. (2008). Attributional (explanatory) thinking about failure in new achievement

settings. European Journal of Psychology of Education, 23, 459–475.Perry, R. P., Stupnisky, R. H., Hall, N. C., Chipperfield, J. G., & Weiner, B. (2010). Bad starts and better finishes: Attributional retraining

and initial performance in competitive achievement settings. Journal of Social and Clinical Psychology, 29, 668–700.Peterson, S. E., & Schreiber, J. B. (2006). An attributional analysis of personal and interpersonal motivation for collaborative projects. Journal of

Educational Psychology, 98, 777–787.Peterson, S. E., & Schreiber, J. B. (2012). Personal and interpersonal motivation for group projects: Replications of an attributional analysis.

Educational Psychology Review, 24, 287–311.Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational

Psychology, 92, 544–555.Ratelle, C. F., Guay, F., Vallerand, R. J., Larose, S., & Senecal, C. (2007). Autonomous, controlled, and amotivated types of academic

motivation: A person-oriented analysis. Journal of Educational Psychology, 99, 734–746.Reeve, J. (2009). Why teachers adopt a controlling motivating style toward students and how they can become more autonomy supportive.

Educational Psychologist, 44, 159–175.Reeve, J., & Jang, H. (2006). What teachers say and do to support students’ autonomy during a learning activity. Journal of Educational

Psychology, 98, 209–218.Reeve, J. & Lee, W. (2012). Neuroscience and human motivation. In R. Ryan (Ed.), Oxford handbook of human motivation (pp. 365–380). New

York, NY: Oxford University Press.Renninger, K. A., & Hidi, S. (2002). Student interest and achievement: Developmental issues raised by a case study. In A. Wigfield & J. S.

Eccles (Eds.), Development of achievement motivation (pp. 173–195). San Diego, CA: Academic Press.Renninger, K. A., & Hidi, S. (2011). Revisiting the conceptualization, measurement, and generation of interest. Educational Psychologist, 46,

168–184.Renninger, K., Hidi, S. E., & Krapp, A. E. (Eds.) (1992). The role of interest in learning and development. Hillsdale, NJ: Lawrence Erlbaum

Associates.Rodkin, P. C., Ryan, A. M., Jamison, R., & Wilson, T. (2013). Social goals, social behavior, and social status in middle childhood.

Developmental Psychology, 49, 1139–1150.Roorda, D. L., Koomen, H. M. Y., Spilt, J. L., & Ooart, F. J. (2011). The influence of affective teacher–student relationships on students’

school engagement and achievement: A meta-analytic approach. Review of Educational Research, 81, 493–529.Rotgans, J. I., & Schmidt, H. G. (2011). Situational interest and academic achievement in the active-learning classroom. Learning and

Instruction, 21, 58–67.

228

Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs: General andApplied, 80, 1–28.

Ryan, A. M., & Patrick, H. (2001). The classroom social environment and changes in adolescents’ motivation and engagement during middleschool. American Educational Research Journal, 38, 437–460.

Ryan, A. M., & Shim, S. S. (2006). Social achievement goals: The nature and consequences of different orientations toward social competence.Personality and Social Psychology Bulletin, 32, 1246–1263.

Ryan, A. M., & Shim, S. S. (2008). An exploration of young adolescents’ social achievement goals and social adjustment in middle school.Journal of Educational Psychology, 100, 672–687.

Ryan, R. M., & Connell, J. P. (1989). Perceived locus of causality and internalization: Examining reasons for acting in two domains. Journal ofPersonality and Social Psychology, 57, 749–761.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.American Psychologist, 55, 68–78.

Schiefele, U. (2009). Situational and individual interest. In K. R. Wentzel & A. Wigfield (Eds.), Handbook of motivation at school (pp. 197–222).New York: Routledge.

Schraw, G., & Lehman, S. (2001). Situational interest: A review of the literature and directions for future research. Educational PsychologyReview, 13, 23–52.

Schunk, D. H., Meece, J. L., & Pintrich, P. R. (2014). Motivation in education: Theory, research, and applications (4th ed.). Upper Saddle River,NJ: Merrill Prentice Hall.

Schunk, D. H., & Pajares, F. (2005). Competence perceptions and academic functioning. In A. J. Elliot & C. S. Dweck (Eds.), Handbook ofcompetence and motivation (pp. 85–104). New York: Guilford Press.

Senko, C., & Harackiewicz, J. M. (2005). Regulation of achievement goals: The role of competence feedback. Journal of Educational Psychology,97, 320–336.

Senko, C., Hulleman, C. S., & Harackiewicz, J. M. (2011). Achievement goal theory at the crossroads: Old controversies, current challenges,and new directions. Educational Psychologist, 46, 26–47.

Shell, D. F., & Husman, J. (2008). Control, motivation, affect, and strategic self-regulation in the college classroom: A multidimensionalphenomenon. Journal of Educational Psychology, 100, 443–459.

Shernoff, D. J., & Csikszentmihalyi, M. (2009). Flow in schools: Cultivating engaged learners and optimal learning environments. In R.Gilman, E. S. Huebner, & M. Furlong (Eds.), Handbook of positive psychology in schools (pp. 131–145). New York: Routledge.

Silvia, P. J. (2005). What is interesting? Exploring the appraisal structure of interest. Emotion, 5, 89–102.Skinner, E. A. (1995). Perceived control, motivation, and coping. Thousand Oaks, CA: Sage Publications.Soric, I., & Palekčic, M. (2009). The role of students’ interests in self-regulated learning: The relationship between students’ interests, learning

strategies and causal attributions. European Journal of Psychology of Education, 24, 545–565.Stroet, K., Opdenakker, M., & Minnaert, A. (2013). Effects of need supportive teaching on early adolescents’ motivation and engagement: A

review of the literature. Educational Psychology Review, 9, 65–87.Trautwein, U., Marsh, H. W., Nagengast, B., Lüdtke, O., Nagy, G., & Jonkmann, K. (2012). Probing for the multiplicative term in modern

expectancy–value theory: A latent interaction modeling study. Journal of Educational Psychology, 104, 763–777.Tuominen-Soini, H., Salmela-Aro, K., & Niemivirta, M. (2012). Achievement goal orientations and academic well-being across the transition

to upper secondary education. Learning and Individual Differences, 22, 290–305.Turner, J. C., Gray, D. L., Anderman, L. H., Dawson, H. S., & Anderman, E. M. (2013). Getting to know my teacher: Does the relation

between perceived mastery goal structures and perceived teacher support change across the school year? Contemporary Educational Psychology,38, 316–317.

Turner, J. C., Meyer, D. K., Cox, K. E., Logan, C., DiCintio, M., & Thomas, C. (1998). Creating contexts for involvement in mathematics.Journal of Educational Psychology, 90, 730–745.

Turner, J. C., Warzon, K. B., & Christensen, A. (2011). Motivating mathematics learning: Changes in teachers’ practices and beliefs during anine-month collaboration. American Educational Research Journal, 48, 718–762.

Urdan, T. (2004). Using multiple methods to assess students’ perceptions of classroom goal structures. European Psychologist, 9, 222–231.Urdan, T. (2010). The challenges and promise of research on classroom goal structures. In J. L. Meece & J. S. Eccles (Eds.), Handbook of

research on schools, schooling, and human development (pp. 92–108). New York: Routledge.Urdan, T., & Mestas, M. (2006). The goals behind performance goals. Journal of Educational Psychology, 98, 354–365.Usher, E. L., & Pajares, F. (2008). Sources of self-efficacy in school: Critical review of the literature and future directions. Review of Educational

Research, 78, 751–796.Vansteenkiste, M., Sierens, E., Goossens, L., Soenens, B., Dochy, F., Mouratidis, A., Aelterman, N., Haerens, L., & Beyers, M. (2012).

229

Identifying configurations of perceived teacher autonomy support and structure: Associations with self-regulated learning, motivation andproblem behavior. Learning and Instruction, 22, 431–439.

Vansteenkiste, M., Sierens, E., Soenens, B., Luyckx, K., & Lens, W. (2009). Motivational profiles from a self-determination perspective: Thequality of motivation matters. Journal of Educational Psychology, 101, 671–688.

Vansteenkiste, M., Zhou, M., Lens, W., & Soenens, B. (2005). Experiences of autonomy and control among Chinese learners: Vitalizing orimmobilizing? Journal of Educational Psychology, 97, 468–483.

Vispoel, W. P., & Austin, J. R. (1995). Success and failure in junior high school: A critical incident approach to understanding students’attributional beliefs. American Educational Research Journal, 32, 377–412.

Walker, C. O., Greene, B. A., & Mansell, R. A. (2006). Identification with academics, intrinsic/extrinsic motivation, and self-efficacy aspredictors of cognitive engagement. Learning and Individual Differences, 16, 1–12.

Watt, H. M. (2004). Development of adolescents’ self-perceptions, values, and task perceptions according to gender and domain in 7th through11th grade Australian students. Child Development, 75, 1556–1574.

Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548–573.Weiner, B. (2011). An attribution theory of motivation. In A. W. Kruglanski, P. A. M. Van Lange, & E. T. Higgins (Eds.) Handbook of

theories in social psychology (Vol. 1, pp. 135–155). London, UK: Sage Publications.White, R. W. (1959). Motivation reconsidered: The concept of competence. Psychological Review, 66, 297–333.Wigfield, A., Battle, A., Keller, L. B., & Eccles, J. S. (2002). Sex differences in motivation, self-concept, career aspiration, and career choice:

Implications for cognitive development. In A. McGillicuddy-De Lisi & R. De-Lisi (Eds.), Biology, society, and behavior: The development ofsex differences in cognition (pp. 93–124). Westport, CT: Ablex Publishing.

Wigfield, A., Eccles, J. S., Schiefele, U., Roeser, R. W., & Davis-Kean, P. (2006). Development of achievement motivation. In N. Eisenberg,W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology: Vol. 3, Social, emotional, and personality development (6th ed., pp. 933–1002).Hoboken, NJ: John Wiley.

Wigfield, A., Tonks, S., & Klauda, S. L. (2009). Expectancy-value theory. In K. R. Wentzel & A. Wigfield (Eds.), Handbook of motivation atschool (pp. 55–75). New York: Routledge.

Wolters, C. A. (2004). Advancing achievement goal theory: Using goal structures and goal orientations to predict students’ motivation,cognition, and achievement. Journal of Educational Psychology, 96, 236–250.

Wolters, C. A., Fan, W., & Daugherty, S. G. (2013). Examining achievement goals and causal attributions together as predictors of academicfunctioning. Journal of Experimental Education, 81, 295–321.

Wormington, S. V., Corpus, J. H., & Anderson, K. G. (2012). A person-centered investigation of academic motivation and its correlates inhigh school. Learning and Individual Differences, 22, 429–438.

Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction,22, 413–419.

Zusho, A., & Clayton, K. (2011). Culturalizing achievement goal theory and research. Educational Psychologist, 46, 239–260.

230

8Volition

GABRIELE OETTINGEN

New York University

JANA SCHRAGE

University of Hamburg, Germany

PETER M. GOLLWITZER

New York University Consider a teacher who is expressing concern about a student’s academic performance. The teacher says thestudent lags behind the rest of the class, and needs to do well on an important, upcoming test. The studentlistens to the teacher’s feedback: To prepare for the test, he decides to study an extra hour every day during thenext few weeks. The incentive value of regularly studying an extra hour is high as the student wants to excelon the test. Also, he knows from past performance that he actually can study every day for an extra hour.Given a high incentive value and high expectations of successfully putting in extra work, the student ismotivated and begins to add regular study time starting the next day. However, after a week has passed, thestudent has managed to add the extra hour just once. Even worse, he did not sleep well last night and is nowoverly tired. The student still intends to sit down and open his book that evening, but just then a friend callsand asks him over to watch an award-winning movie. In light of these difficulties and temptations, it is nowvolition that determines whether or not the student will give in and see the movie or go forward with hisintention to stay home and study.

231

Motivation and Volition

In this chapter, we explore how volitional processes affect behavior change. In contrast to motivationalprocesses such as those affecting expectations and incentive values, volitional processes are needed when thereis resistance to or conflict with attaining a desired future. In education, volitional processes support students,teachers, and administrators in mastering resistance or conflict (e.g., obstacles or temptations) on the way toreaching a desired future.

We conceptualize motivation as the energy to pursue a desired future and the direction that helps tochannel this energy. Our definition builds upon that of Hull (1943), who proposed that variation in behavioris a function of intensity and direction. The intensity of a behavior is defined by the aroused energy (Duffy,1934), whereas the direction of action is defined by whether the behavior is aimed at approaching or avoidinga certain stimulus (Atkinson, 1957; McClelland, 1985; see also Oettingen et al., 2009). The sources ofintensity and direction are specified as motive disposition, expectation, and incentive value (Atkinson, 1957;Hull, 1943; Tolman, 1932).

Regarding the determinants of motivation, Gollwitzer (1990, 2012) coined the summary terms ofdesirability and feasibility. Desirability is defined as the expected value of a certain desired future (i.e., theperceived attractiveness of the expected short- and long-term consequences, within and outside the person, ofhaving reached the desired future), while feasibility relates to expectations of attaining the desired future.Expectations are beliefs or judgments of perceived probabilities that are based on experiences in the past (e.g.,Bandura, 1977; Mischel, 1973). Expectations come in different forms. There are: (a) expectations of whetheror not one is capable of performing a certain behavior that is necessary to achieve a desired outcome (self-efficacy expectations: Bandura, 1977); (b) expectations of whether or not the performed behavior will lead tothe desired outcome (outcome expectations: Bandura, 1977; Mischel, 1973); and (c) general expectations ofwhether or not one will reach the desired outcome (general expectations: Oettingen & Mayer, 2002).

Theory holds and research shows that beliefs pertaining to expectations and incentive values are the keydeterminants of motivation. Thus motivational strategies can be defined as those that are tailored to changeperceived incentive values and expectations to attain a desired future. In educational contexts the aimed-forchange of beliefs focuses on increasing the incentive value of a normative behavior (e.g., studying) anddecreasing the incentive value of a non-normative behavior (e.g., attending class unprepared). At the sametime, motivational procedures pertain to increasing expectations of performing the normative behavior.Chapter 7 (this volume) discusses the history of research on motivation, and Chapter 12 (this volume)discusses motivation interventions in education.

In contrast, the concept of volition comprises self-regulation strategies that target resistance and conflict(e.g., conflicts that may block or delay goal striving). Therefore, volitional strategies often help people toclarify goals when goals are ambiguous or equivocal; they prepare for potential obstacles standing in the way ofattaining the desired future; and they enable individuals to stay on track and pursue their desired future evenin the face of impediments, difficulties, and temptations. In line with this definition of volition, WilliamJames (1890) pointed out that volition is needed when a person faces resistance or conflict. James stated that:

volition is a psychic or moral fact pure and simple, and is absolutely completed when the stable state of the idea is there. . . . The essentialachievement of the will, in short, when it is most “voluntary”, is to attend to a difficult object and hold it fast before the mind. (James,

232

1890, p. 446)

Accordingly, the use of volitional strategies supports individuals as they act upon the pre-existing incentivevalue of their desired futures and the expectations of attaining them. Put differently, using volitional strategiesaims at translating high incentive value and expectations into respective behavior.

In the next sections, we provide an overview of the history and recent research on volitional processes andstrategies that are relevant to educational settings. In particular, we discuss two volitional strategies and theircombination to illustrate the role that volition plays in learning and performance. These strategies are mentalcontrasting and forming implementation intentions; combining mental contrasting with implementationintentions (MCII) forms a third kind of strategy.

233

Volitional Processes and Interventions

As noted above, volition is required whenever people who have a desired future in mind face resistances orconflict (James, 1890; Oettingen, 2000, 2012; Oettingen, Wittchen, & Gollwitzer, 2013). In the context ofeducation theory, volition plays a role in the translation of dispositions and processes of motivation intooutcomes of learning and performance (e.g., Corno, 1993, 2004; Corno & Kanfer, 1993). Contemporaryapproaches to research on volition distinguish between top-down and bottom-up processes of volition(Boekaerts & Corno, 2005). In top-down processes the volition needed is determined by the goals thatstudents pursue. In bottom-up processes the students react to stimuli in their environment (e.g., stressors) andadjust their volition to the situation. We will now briefly describe three prominent examples for the interplayof top-down and bottom-up volitional processes: goal-shielding, the goal–subgoal hierarchy, and theregulation of conflicts between growth and well-being.

Volitional Processes

Goal shielding. One serious challenge in goal pursuit is shielding the adopted goal from interfering goals andbehaviors (Gollwitzer, Bayer, & McCulloch, 2005). Conducive for goal shielding are, for example, high goalcommitment (Shah, Friedman, & Kruglanski, 2002) and an action orientation in contrast to a stateorientation (Kuhl & Beckmann, 1994). Some assumed mechanisms that drive goal shielding are:environmental control, cognitive control, and emotion control (Kuhl & Beckmann, 1994). Furthermore, theinterplay between a person’s present emotions and the proximity of the goal seems to drive goal-shieldingprocesses. If the goal is perceived as distal, positive emotions increase goal shielding because they signal highgoal commitment. However, if the goal is perceived as proximal, positive emotions decrease goal shieldingbecause they signal goal attainment; then negative emotions increase goal shielding (Louro, Pieters, &Zeelenberg, 2007).

Goals and subgoals. Recent research also takes into account that more than one goal is activated at any giventime, and that every superordinate goal can be broken down into several subgoals (Fishbach, Shah, &Kruglanski, 2004). If the superordinate goal is activated, initial success regarding a subgoal signals highcommitment to the superordinate goal; in contrast, initial failure on the subgoal indicates low commitment tothe superordinate goal. If the superordinate goal is not activated, however, initial success on a subgoal signalsgoal attainment, whereas initial failure on a subgoal leaves the goal incomplete (Fishbach, Dhar, & Zhang,2006). According to this approach, goal pursuit can only be understood in the context of the goal structurethat characterizes the individual.

Dual processing self-regulation model. This model is specific to classroom learning and distinguishes betweentwo pathways of self-regulation: the growth pathway and the well-being pathway (Boekaerts & Cascallar,2006; Boekaerts & Corno, 2005). Assuming that on the growth pathway volition works top-down, studentsregulate their cognition, emotion, and behavior to pursue the respective goal (e.g., learning a new language).Assuming that on the well-being pathway volition works bottom-up, students regulate their cognition,emotion, and behavior to maintain their well-being in the face of hindrances (e.g., avoiding harm orprotecting one’s self-esteem). Students’ self-regulation efforts in the well-being path are cue-driven as they

234

react to hindrances and setbacks in their environment, trying to avoid further misery and instead stabilizewell-being.

Volitional Interventions

Various interventions attest to volitional processes in the field of education. Corno (1994) provides anoverview of such interventions, differentiating them by three categories: (1) volitional interventions directed atparticular students or content areas; (2) volitional interventions that focus on improving homework; and (3)interventions that aim at collaborative efforts with teachers to design classroom activities that promotevolitional control.

Volitional interventions directed at particular students and content areas. Interventions in educational settingsoften focus only on subjects students learn in school or are directed at students who display particularproblems (e.g., impulse control). However, domain-specific instructions may provide insufficient context forretention and transfer (see e.g., Hattie, Biggs, & Purdie, 1996), and not all students need instruction involitional control. One study by Perels, Dignath, and Schmitz (2009) used a pretest/post-test–control-groupdesign to test a self-regulation intervention in sixth-grade mathematics students in Germany. They observed ateacher instructing one class using the regular math curriculum offering strategies for solving math problems(e.g., segmentation of complex problems into components; control group), and then observed the sameteacher instructing another class in using self-regulation techniques when solving the math problems (e.g.,dealing with distractions; intervention group). In the post-test, the intervention group reported more self-regulated behavior than the control group, while there was no difference in the pretest. In addition, only inthe intervention group did scores of mathematical competences improve over time. Other interventions havetargeted processes of volition and motivation in reading and writing for students who are particularly in needof help (Schunk & Zimmerman, 2007).

Interventions for volitional enhancement during homework. Homework can be considered a reference task forstudying processes of volitional control. Specifically, it can be used to observe how volitional control is appliedby students and how such control is taught by parents and teachers (Corno, 2011). In a recent example, Xu,Yuan, Xu, and Xu (2014) studied variables that predict time management in the context of mathematicshomework in a large sample of Chinese secondary students. The more students reported to engage involitional control (e.g., turn off the TV), the better they reported to manage their time (e.g., I set prioritiesand plan ahead), even though other important factors (e.g., prior math achievements) were statisticallyadjusted for. These results have implications for creating volition-enhancing interventions for parents andcaregivers. Educators can teach students strategies to improve their homework routines, and students canshare their effective strategies for doing homework with others in a class or via social media. Anotherreference task for studying volitional control is strategic reading (see Pressley et al., 1990, for interventionsenhancing reading comprehension and fluency). This area of research is particularly important now that somuch studying is done online or using a computer (for examples, see Corno, 2011).

Collaborative interventions with teachers on curriculum development. Randi (2005) collaborated with pre-serviceteachers to develop the teachers’ volitional control skills, by teaching them the theoretical foundations of self-

235

regulation as described in Boekaerts and Corno (2005), as well as knowledge about opportunities to use self-regulation strategies (e.g., emotion control strategies related to teaching effectively). The teachers were alsoencouraged to recall curricular experiences that allowed them to model specific volitional control strategies.They were taught to focus on negotiating opportunities to teach the curricula they had developed, onevaluating their own teaching practices, and on seeking feedback from mentors.

Interventions deploying conscious and non-conscious processes. Next to classifying interventions according to theneeds in the classroom, as discussed above, volitional interventions may be grouped according to whether theydeploy conscious processes versus processes that occur outside of awareness (non-conscious processes), orwhether they use both conscious and non-conscious processes. Some approaches, including our own lines ofinvestigation, focus on conscious volitional strategies that trigger volitional processes outside of awareness.

Mental Contrasting

Fantasy realization theory (review by Oettingen, 2012) specifies a powerful volitional strategy of behaviorchange, referred to as mental contrasting. Mental contrasting involves engaging in fantasies about a desiredfuture, and alongside reflecting realities that might impede attaining that future. Mental contrasting producesa wise use of energy: Heightened energy when people perceive their chances of success as being high, andreduced energy when people perceive their chances of success as being low (Oettingen, 2000; Oettingen, Pak,& Schnetter, 2001; Oettingen et al., 2009; summary by Oettingen, 2012).

In educational settings, when students mentally contrast, they first imagine a desired future (e.g., to get agood grade on an upcoming math test), and then imagine the reality that stands in the way of attaining thisdesired future (e.g., being distracted). Mental contrasting activates expectations of successfully overcoming thereality towards attaining the desired future: If these expectations are high, students will actively pursue(commit to and strive for) reaching the desired future of attaining a good grade. If expectations of success arelow, students will refrain from realizing the desired future and will curb their efforts to reach this future, or letgo of pursuing this future to save their resources for more promising endeavors (Oettingen et al., 2001). Inthis way, mental contrasting helps people differentiate between their pursuits, allowing them to invest theirresources into futures that warrant success and to refrain from investing in futures they deem futile. Mentalcontrasting thus qualifies as a strategy that conserves energy and resources, both in the short and the longterm.

Beyond mental contrasting, fantasy realization theory specifies three more modes of thought: (a) indulging,which means imagining the desired future without considering the reality; (b) dwelling, focusing on the realitywithout the desired future in mind; and (c) reverse contrasting, first focusing on the present reality andthereafter elaborating on the desired future. In contrast to mental contrasting, indulging does not juxtaposethe resisting reality to the positive future, and dwelling does not incorporate the desired future into thoughtsabout the reality. Such one-sided thoughts and images do not signal that resistances need to be overcome toattain the desired future (indulging) and they do not suggest in which direction to act (dwelling). Finally, withrespect to reverse contrasting, it is important to keep in mind that the effects of mental contrasting depend onpeople perceiving the present reality as impeding the desired future. In mental contrasting, individuals firstimagine the desired future, and thus the future works as the reference point. Only then do they elaborate the

236

present reality. Thus the reality can change its meaning and become an obstacle to attain the desired future(Kappes, Wendt, Reinelt, & Oettingen, 2013). Reversing this order (i.e., reverse contrasting), by firstimagining the present reality, and then the desired future, does not present the reality as impeding or standingin the way of the desired future. Therefore, reverse contrasting does not promote goal pursuit and behaviorchange in line with expectations of success (e.g., Kappes et al., 2013; Oettingen et al., 2001, Study 3; Sevincer& Oettingen, 2013). In sum, indulging, dwelling, or reverse contrasting do not activate expectations of successand none of them leads to prudent pursuit of the future that is in line with one’s chances to attain the desiredfuture (Oettingen et al., 2001; Oettingen, 2012).

Think about the classroom context, and an elementary-school student who wants to improve her readingskills during the next term. The student’s incentive value is high; she loves science fiction and is keen onlearning how to read the books on her own. She has high expectations of successfully improving her readingskills; thus far, she has been a good student. Using mental contrasting, the student first vividly imagines howtruly wonderful it would be to read her favorite science fiction book all by herself, independently and withoutany assistance from her parents (desired future). Then she identifies what it is in herself that holds her backfrom practicing her reading skills. What is her main obstacle? The student discovers that her main obstacle isthat she is constantly distracted by social media, with all the tempting news of her friends (obstacle of thepresent reality). She now imagines these feelings of temptation, how she is tempted to look at her friends’ coolpictures and getting the latest news. After this short imagery-based exercise of mental contrasting, the studentrecognizes that constantly looking at her social media outlets prevents her from becoming an independent andself-reliant reader. Now she will shut off her media applications, at least for a while, and practice reading.

Effects of mental contrasting. Mental contrasting has been shown to effectively change behavior in manydifferent educational settings and with diverse student samples (see summary by Oettingen, 2012). Forexample, one experimental study investigated first-year students in a vocational school for computerprogramming. For these students, mathematics was the most critical subject and they viewed improving theirmath skills as highly desirable. Oettingen et al. (2001, Study 4) instructed the participants to identify factorsthey associated with excelling in mathematics (participants named e.g., better chances to get a good job, tosimply be happy), and to identify aspects of their present reality that may stand in their way of excelling(participants named e.g., not enough sleep and partying). Three modes of thought were then experimentallyinduced. Participants were directed to imagine and write about two aspects of their desired future and twoaspects of their present reality, in an alternating order, beginning with the desired future (mental contrastingcondition). Alternatively, they had to mentally elaborate four aspects of the desired future (indulgingcondition) or four aspects of the present reality (dwelling condition). Participants were then asked, directlyfollowing the experimental procedure, to rate (on five-point scales) how energized they felt with respect toexcelling in mathematics (e.g., how active, eventful, energetic). Two weeks later, participants’ teachersreported how much effort each student had invested in schoolwork during the past 2 weeks. In addition, theteachers provided course grades for each student during that time period.

For students in the mental contrasting condition, the link between expectations of success and beingenergized was significantly stronger than in the indulging and dwelling conditions. In addition, mentalcontrasting students were found to have exerted significantly more effort, and earned grades in line with their

237

expectations of success: Those with high expectations of success felt most energized, exerted most effort, andwere awarded with the highest grades. The reverse was true for those with low expectations of success.Students in the indulging and dwelling conditions ranged in between, regardless of their expectations ofsuccess.

A series of further experimental studies replicated these results. Those pertinent to education involved:studying abroad in university students (Oettingen et al., 2001, Study 2), acquiring English as a foreignlanguage in middle-school students (Oettingen, Hönig, & Gollwitzer, 2000, Study 1), excelling in giving anad hoc presentation in university students (Oettingen et al., 2009, Study 2), seeking help from academicexperts in university students (Oettingen, Stephens, Mayer, & Brinkmann, 2010, Study 1), increasingtolerance towards members of minorities in high-school students (Oettingen, Mayer, Thorpe, Janetzke, &Lorenz, 2005, Study 2), and successfully combining work and family life while raising a child as a graduatestudent (Oettingen, 2000, Study 2). Further, strength of goal pursuit was assessed in these studies by cognitive(e.g., making plans), affective (e.g., feelings of responsibility to attain the desired future), motivational (e.g.,feelings of disappointment), and behavioral indicators (e.g., exerted effort and spent resources). Indicatorswere measured subjectively (e.g., self-report) and objectively (e.g., content analysis, independent observations),directly after the experiment or weeks and months later. Across experiments, the described pattern of findingswas observed: Participants with high expectations in the mental contrasting condition vigorously pursued theirdesired future, while participants with low expectations decreased their efforts or let go altogether.Participants in the indulging or dwelling conditions pursued their future with moderate effort and successregardless of whether their expectations of success were high or low. To summarize, only mental contrastingparticipants regulated their goal pursuit so that their resources were protected. They showed high investmentwhen the attainment of the future was likely and low or no investment when attainment was unlikely.

It was hypothesized and found that mental contrasting does not change expectations of success, butactivates pre-existing expectations of success, translating them into goal pursuit and behavior change(Oettingen, 2012). In two studies, Oettingen, Marquardt, and Gollwitzer (2012) investigated whether mentalcontrasting translates expectations into heightened effort and performance even if they are induced in situ viapositive situational feedback. The authors used a creativity task to provide positive or moderate bogusfeedback to college students. Thereafter participants engaged in mental contrasting, indulging, dwelling, or incontrasting irrelevant content. Mental contrasting increased creative performance after positive feedbackcompared with moderate feedback. Indulging, dwelling, and irrelevant contrasting did not change creativeperformance, regardless of feedback. Importantly, by manipulating expectations through bogus feedback, theOettingen et al. (2012) studies showed that mental contrasting indeed translates expectations of success intobehavior change, rather than affecting a third variable that may underlie both expectations of success andbehavior. Further, these studies suggest that if the prerequisite of high expectations of success is not met, thensuch expectations can be induced on the spot through the provision of positive performance feedback. This isan important finding for teachers who wish to increase energy and study efforts in their students. By providingstudents with doable challenges (e.g., in math) and giving them respective positive feedback, teachers can takeadvantage of the students’ heightened expectations: mental contrasting will then effectively increase students’efforts and successful performance, even in areas and tasks that they had not been strong in originally (e.g., toexcel in math tests).

238

Processes of mental contrasting. The effects of mental contrasting on behavior change are mediated by cognitiveand motivational processes. As for cognitive changes, mental contrasting paired with high expectationsstrengthens the mental associations between future and the obstacle of reality as well as between the obstacleand the instrumental means to overcome it. It also changes the meaning of reality, in that the reality nowbecomes interpreted as an obstacle (Kappes & Oettingen, 2014; Kappes, Oettingen, & Pak, 2012; Kappes etal., 2013). Regarding motivational changes, mental contrasting catalyzes energy (measured by systolic bloodpressure). That is, when prospects are good, it heightens energy, when they are bad it relaxes, so that the savedenergy can be used for alternative projects. Importantly, changes in energy mediate the relation betweenexpectations and goal pursuit (Oettingen et al., 2009; Sevincer, Busatta, & Oettingen, 2014). Finally,regarding responses to negative feedback, mental contrasting changes the ways students deal with negativefeedback. When the desired future seems reachable, negative feedback is processed as valuable information forreaching the desired future. It is processed without impairing a student’s subjective competence, and it bolstersbeneficial attributions (Kappes et al., 2012).

Taken together, mental contrasting will help students to attain success (e.g., excelling in a test, beingfriendly to the teacher) without consciously exerting effort. That is, the described processes mediating theeffects of mental contrasting happen outside of awareness. Specifically, the building of mental associationsbetween the desired future (e.g., good job opportunities) and obstacles of the present reality (e.g., poorlanguage skills), and between the obstacles and instrumental means that deal with these obstacles (e.g., askingthe teacher for help with language homework) will lead the student to actually go ahead and realize thedesired future (e.g., ask the teacher for support; e.g., Oettingen et al., 2010c). Again, without awareness,mental contrasting will also provide the necessary energy and effort to reach the desired future (e.g., seek theteacher’s help).

It comes as no surprise, then, that objective measures of effort and performance show the effects of mentalcontrasting more clearly than self-report measures. In other words, it may be hard for students to report onthe exerted effort, as this effort is triggered outside of awareness. Finally, mental contrasting prepares them toeffectively respond to critical feedback, by allowing them to non-consciously process immanently usefulinformation entailed in the negative feedback. Mental contrasting is beneficial also because it shelters studentsfrom taking negative feedback from their teacher personally. Reducing students’ defensiveness should aidstudent–teacher interactions when negative feedback is impending. In their entirety the reported processesinstigated by mental contrasting support students to master some of the most difficult tasks in the educationalcontext: initiating appropriate behavior change and carrying on in light of difficulties and setbacks.

Mental contrasting as a metacognitive intervention. So far, mental contrasting has been shown to be a volitionalself-regulation strategy that helps people initiate and sustain behavior change across time and in the face ofdifficulties. The question arises whether mental contrasting could be taught as a metacognitive strategy, thatis, as a strategy that implies thinking about one’s own thinking (Flavell, 1979). Can students learn mentalcontrasting as a skill that enables them to wisely select and prudently pursue their own idiosyncratic wishes?Can teachers and administrators learn and adopt the strategy in everyday life? Such use of mental contrastingmay support individuals as they study, teach, or provide other services of schooling that call for effective timemanagement and prioritizing goals.

239

Effective time management and decision making. Corno (2001) made the case that time and resourcemanagement as well as prioritizing goals is an important volitional skill for students in school and in everydaylife. Mental contrasting, which promotes selective goal pursuit, should benefit students, teachers, andadministrators by improving their time management and decision making. The effectiveness of mentalcontrasting for time management and decision making was shown in a study with middle-level health careadministrators who had to work on many projects simultaneously and constantly adjust their time schedules(Oettingen, Mayer, & Brinkmann, 2010). The administrators were taught mental contrasting as ametacognitive strategy. That is, participants learned how to apply mental contrasting to a host of wishes orconcerns in their everyday lives.

The administrators were randomly assigned to two conditions. In one condition, participants were taughtto use mental contrasting regarding important everyday concerns, while participants in the other conditionwere taught to indulge in respective future fantasies. Participants generated concerns such as solving a conflictwith an employee, writing a report, or organizing a dinner party, all of which they then either practiced usingmental contrasting or indulging. The selected problems had to be controllable and participants needed to beable to act upon them. However, participants also had to feel somewhat uneasy about how to solve them.Each participant practiced the respective strategy (mental contrasting vs. indulging) using at least six suchproblems, and were then told to apply it to as many problems as possible during the upcoming weeks(Oettingen et al., 2010a). Two weeks later, compared to those in the indulging condition, participants in themental contrasting condition reported to have managed their time more effectively and to have made bettereveryday life decisions.

As outlined above, mental contrasting with low expectations of success leads to relatively weak goal pursuitor even goal disengagement. However, sometimes goal disengagement from certain focal goals is unwantedfor ethical or practical reasons. For example, it is not desirable for students to disengage from the goal ofattending school or learning basic skills such as reading, writing, or math. In these cases, mental contrastingcan still strengthen goal pursuit, if expectations of success are high. There are three ways to ensure that allparticipants who use mental contrasting hold high expectations of success. As described above, one way toinstill positive expectations in situ is by giving positive performance feedback (Oettingen et al., 2012). Anotherway is to assign participants a new task, for which they have no pre-existing performance experiences and toassure them that it is feasible for them to succeed (A. Gollwitzer, Oettingen, Kirby, Duckworth, & Mayer,2011). And finally, one can ask participants to generate an idiosyncratic (academic or well-being) wish orconcern that is challenging yet feasible (Oettingen, 2012).

Learning a foreign language. Applying the second of the three options, A. Gollwitzer et al. (2011) showed intwo studies that mental contrasting managed to heighten academic performance for elementary and middle-school children. The intervention was directed at second- and third-graders in Germany and fifth-graders inthe United States. The children had to either learn vocabulary in a foreign language (English for the Germanparticipants) or they had to learn to say thank you in ten different languages (participants were fifth-graders inthe United States). To guarantee high expectations of success, participants were not given the opportunity togain prior experience with the task and it was ensured that it was possible for all students to succeed (A.Gollwitzer et al., 2011). Across studies, participants in the mental contrasting condition were more successful

240

in learning the new vocabulary than students in the indulging (control) condition.

Increasing well-being: Eating healthier and becoming more active. Applying the third option mentioned above,college students who were interested in improving their well-being named respective idiosyncratic wishes forthe next 2 weeks (e.g., eating healthier, losing weight). Thereafter, they were either instructed to mentalcontrast or indulge in fulfilling these wishes (Johannessen, Oettingen, & Mayer, 2012); a third group receivedno treatment. Two weeks later, compared to those in the indulging or no treatment condition, students in themental contrasting condition reported an overall lower calorie intake, as they consumed less high-calorie foodsand more low-calorie foods (Johannessen et al., 2012). Interestingly, the effects of mental contrastingtransferred across domains. Students in the mental contrasting also reported more physical activity comparedto participants in the other two conditions (Johannessen et al., 2012).

Summary. Mental contrasting is a volitional strategy that allows for both engagement to, and disengagementfrom, desired futures—depending on the feasibility of realizing the envisioned future. Specifically, mentalcontrasting produces cognitive changes (e.g., mental associations, changes in the meaning of reality), energy(e.g., systolic blood pressure), and constructive mastery of negative feedback (e.g., careful processing ofinformation) that in turn predicts behavior change in line with how feasible the desired future is perceived.Thus, mental contrasting is a conscious strategy that produces changes in cognition outside of awareness,which in turn predicts the observed behavior change. Engaging in promising and disengaging from futilefutures guarantees that a person who uses mental contrasting saves resources for successfully managingeveryday life and long-term development. Mental contrasting is easy to apply and can be taught as ametacognitive strategy, unfolding its effects in such diverse life domains as excelling in academic performance,promoting one’s health and well-being, and managing time and other resources.

Implementation Intentions

When pursuing academic goals, students are often confronted with the following challenges: they need to getstarted and take the first steps toward pursuing their goals; they must stay on track when goal striving hasstarted; they should not overextend when striving for a given goal; and finally, they should disengage from anunattainable goal or futile means of attaining that goal (Gollwitzer & Sheeran, 2006). Planning in advancehow one wants to deal with these challenges is an effective remedy. Gollwitzer (1993, 1999, 2014) highlightedthe importance of forming implementation intentions that specify plans with the format of “If situation X isencountered, then I will perform the goal-directed response Y!” Thus, implementation intentions definewhen, where, and how one wants to act. For instance, a student who wants to make more constructivecontributions in class might form the following if-then plan: “And if another student is desperately trying toanswer a difficult question, then I’ll immediately jump to his rescue!” Empirical data support the assumptionthat implementation intentions help raise the rate of goal attainment. A meta-analysis based on close to ahundred studies shows a medium to large effect on increased rate of goal attainment (d = 0.61; Gollwitzer &Sheeran, 2006).

Underlying processes of implementation intention effects. Research on the underlying processes ofimplementation intention effects has revealed that implementation intentions facilitate goal attainment on the

241

basis of psychological mechanisms that relate to both the anticipated situation (specified in the if-part of theplan) and the association created between the if-part and the then-part of the plan (Gollwitzer, 1999).Because forming an implementation intention implies the selection of a critical future situation, the mentalrepresentation of this situation becomes highly activated and hence more accessible. For instance, Achtziger,Bayer, and Gollwitzer (2012) observed in a cued recall experiment that participants more effectively recalledthe available situational opportunities to attain a set goal, given that these opportunities had been specified inif-then links (i.e., in implementation intentions); this effect showed up no matter whether the cued recall wasrequested 15 minutes or 24 hours later. Furthermore, a study by Parks-Stamm, Gollwitzer, and Oettingen(2007), using a lexical decision task paradigm, showed that implementation intentions not only increased theactivation level of the specified critical cues, they also diminished the activation level of non-specifiedcompeting situational cues.

Forming implementation intentions creates strong associations between the specified critical situations andgoal-directed responses. Thus, the execution of the goal-directed response, once the critical situational cue isencountered, can be expected to exhibit features of strong associations, such as automaticity in terms ofimmediacy, efficiency, and no need for conscious intent. Indeed, there is vast empirical evidence that if-thenplanners act more quickly (e.g., Gollwitzer & Brandstätter, 1997, Experiment 3), deal more effectively withcognitive demands (e.g., speed-up effects still emerge under high cognitive load and thus qualify as efficient;e.g., Brandstätter, Lengfelder, & Gollwitzer, 2001), and do not need to consciously intend to act in thecritical moment (e.g., Bayer, Achtziger, Gollwitzer, & Moskowitz, 2009).

Further support for the hypothesis that action control by implementation intentions qualifies as automaticis also obtained in studies assessing brain data. In a functional magnetic resonance imaging study reported byGilbert, Gollwitzer, Cohen, Oettingen, and Burgess (2009), acting on the basis of goal intentions wasassociated with brain activity in the lateral rostral prefrontal cortex, whereas acting on the basis ofimplementation intentions was associated with brain activity in the medial rostral prefrontal cortex. Brainactivity in the latter area is known to be associated with bottom-up (stimulus) control of action, whereas brainactivity in the former area is known to be related to top-down (goal) control of action (Burgess, Dumontheil,& Gilbert, 2007). Moreover, the automaticity of implementation intentions effects has also been supported bystudies that collected brain data employing electroencephalography (e.g., Gallo, Keil, McCulloch, Rockstroh,& Gollwitzer, 2009, Study 3).

But do these postulated processes actually mediate implementation intention effects on goal attainment?There is supportive evidence for this assumption. In the Gilbert et al. (2009) study, the increased brain activityin the medial rostral prefrontal cortex matched the increase in prospective memory performance inparticipants who had formed implementation intentions. Moreover, studies by Webb and Sheeran (2007,2008) found that the effects of if-then plans on goal attainment were mediated simultaneously by theaccessibility of the specified situational cues and by the strength of the association between these cues and theintended response. The search for further mediating variables has shown that neither an increase in goalcommitment nor an increase in self-efficacy qualifies as a potential alternative mediator of implementationintention effects.

Implementation intentions as a means to overcome typical challenges of goal striving. The effects of

242

implementation intentions have been demonstrated in the educational, interpersonal, health, andenvironmental domains, with respect to each of the four challenges to effective goal striving: getting started,staying on track, and disengaging from futile and inappropriate goals, as well as avoiding resource depletion.With respect to the first problem, implementation intentions were found to help individuals get started withgoal striving in terms of remembering to act and overcoming an initial reluctance to act (e.g., see summary byGollwitzer & Oettingen, 2011). Accordingly, it seems safe to assume that if-then plans can be used effectivelyto help students and teachers fight procrastination (e.g., getting started with homework or getting started withgrading students’ homework: Wieber & Gollwitzer, 2010).

However, many goals cannot be accomplished by a simple, discrete, one-shot action because they requirepeople to keep striving over an extended period of time. Staying on track may then become very difficult whencertain internal stimuli (e.g., being nervous) or external stimuli (e.g., distractions) interfere with the ongoinggoal pursuit (e.g., going to bed that guarantees a satisfying sleep: Loft & Cameron, 2013). With respect toshielding an ongoing goal pursuit from inside stimuli, implementation intentions were demonstrated to beeffective with respect to performance anxiety (Achtziger, Gollwitzer, & Sheeran, 2008), test anxiety (Parks-Stamm, Gollwitzer, & Oettingen, 2010), social anxiety (Webb, Onanaiye, Sheeran, Reidy, & Lavda, 2010),as well as general anxiety (Varley, Webb, & Sheeran, 2011). Implementation intentions have also beendemonstrated to be effective in shielding goal pursuit from outside stimuli. For instance, they helped collegestudents who were trying to solve math problems to shield themselves from distractive video clips (“If I seemoving pictures or hear some noise, then I’ll ignore them!”: Gollwitzer & Schaal, 1998). Analogous findingswere obtained with children of 6–8 years of age (Wieber, Suchodoletz, Heikamp, Trommsdorff, &Gollwitzer, 2011). Ignore-implementation intentions were highly effective in a classification task(categorizing vehicles vs. animals, presented on a computer screen), even when the distractions were highlyattractive (i.e., cartoon movie sequences), and no matter whether these distractions appeared inside or outsidethe children’s sight.

Implementation intentions may use different formats. For instance, if a student wants to keep studying eventhough the students next to her start a loud conversation, she can form suppression-oriented implementationintentions, such as “And if the students around me get noisy, then I will not get upset!” The then-componentof such suppression-oriented implementation intentions negated the critical behavior (in the present example“then I will not get upset”). However, it may also specify a replacement behavior (“..., then I will stay calm andask them in a friendly manner to be more quiet!”) or focus on ignoring the critical cue (“..., then I will justignore the noise!”). Recent research (Adriaanse, van Oosten, de Ridder, de Wit, & Evers, 2011) suggests that“negation” implementation intentions are less effective than the latter two types (i.e., replacement and ignoreimplementation intentions). Implementation intentions specifically geared towards stabilizing the ongoinggoal striving are particularly effective (e.g., using if-then plans that explicate in detail what needs to be done toreach the goal; Bayer et al.,2009). In fact, it even blocked the disruptive effects created by inappropriatemoods, ego depletion, or feelings of insecurity.

Goals or means that are no longer feasible and/or desirable in their current form may require individuals toadjust goal striving and to disengage from a goal or a chosen means to achieve that goal. Such disengagementfrom unattainable goals or dysfunctional means can free up resources and minimize negative affect resultingfrom repeated failure feedback (Carver & Scheier, 1998; Locke & Latham, 1990, 2006). Implementation

243

intentions help to master this third challenge of effective goal pursuit (i.e., functional disengagement) by: (a)specifying negative feedback as a critical cue; and (b) linking this cue to switching to an alternative goal ormeans (e.g., a different way of studying for an academic test; Henderson, Gollwitzer, & Oettingen, 2007).

Finally, regarding the fourth challenge of effective goal pursuit, not overextending oneself, formingimplementation intentions prevents resource depletion. Specifically, it enables individuals to engage inautomated goal striving. As a consequence, the self should not become depleted (Muraven & Baumeister,2000) when goal striving is regulated by implementation intentions. Indeed, in studies using different ego-depletion paradigms, research participants who used implementation intentions to self-regulate in one taskdid not show reduced self-regulatory capacity in a subsequent task (e.g., switching from one academic task tothe next; Webb & Sheeran, 2003).

When effective goal striving gets particularly hard. The following three situations ask for more powerful self-regulation: (a) situations in which a person’s knowledge and skills constrain performance, such as having tosolve difficult math problems; (b) situations in which a competitor limits one’s performance, such ascompetitive sports; and (c) situations in which the wanted behavior (e.g., paying attention in class) conflictswith established habits favoring an antagonistic response (e.g., chatting with one’s classmate). For all three ofthese situations, implementation intentions turned out to be beneficial.

Implementation intentions were found to enhance participants’ performance on the Raven intelligence test,which consists of a series of problems to be solved (Bayer & Gollwitzer, 2007). The implementation intention“If I start a new problem, then I will tell myself: I can do it!” was more effective than the respective goalintention “I will tell myself: I can do it!” Tennis players participating in competitive tennis tournaments usingimplementation intentions effectively coped with critical situations during the game (e.g., “If I’m fallingbehind, then I’ll tell myself: Stay concentrated!”; Achtziger et al., 2008).

Finally, assuming that action control by implementation intentions is immediate and efficient, a horseracemodel of action control suggests that implementation intentions can be used to deal with antagonistic habitualresponses (Adriaanse, Gollwitzer, de Ridder, de Wit, & Kroese, 2011). Implementation intentions thatspecify responses contrary to the habitual responses (Cohen, Bayer, Jaudas, & Gollwitzer, 2008), have beenshown to effectively reduce habitual responses, such as stereotyping (e.g., “When I see the face of someonewho looks different from me, then I will think ‘safe’!”; e.g., Gollwitzer & Schaal, 1998; Mendoza, Gollwitzer,& Amodio, 2010; Stewart & Payne, 2008).

Still, forming implementation intentions may not always succeed in blocking habitual responses. Whetherthe habitual response or the if-then guided response will “win the race” depends on the relative strength of thetwo behavioral orientations (Webb, Sheeran, & Luszczynska, 2009). This implies that controlling stronghabits requires the formation of strong implementation intentions (e.g., trying to break the bad habit ofwatching TV when one gets home from school by an if-then plan to first do one’s homework). Formingstrong implementation intentions can be achieved by various measures. One pertains to creating particularlystrong links between situational cues (if-component) and goal-directed responses (then-component) by askingparticipants to use mental imagery (e.g., Knäuper, Roseman, Johnson, & Krantz, 2009). Also, certain formatsof implementation intentions (i.e., replacement and ignore implementation intentions) seem to be moreeffective in fighting habits than others (i.e., negation implementation intentions), and some formats seem to

244

work better for some people than others (e.g., test-anxious individuals particularly benefit from ignoreimplementation intentions; Parks-Stamm et al., 2010). Finally, one has to keep in mind that behavior changecannot only be achieved by breaking old habits; one can also form new habits in new situational contexts (e.g.,doing one’s homework in the library before one goes home).

Moderators of implementation intention effects. Recent research has identified a number of moderators ofimplementation intention effects on goal attainment. First, implementation intentions only benefit goalattainment when commitment to both the goal is high (Sheeran, Webb, & Gollwitzer, 2005) and to executingthe implementation intention is high (Achtziger et al., 2012, Study 2). Second, person attributes play a role.In undergraduate students (Webb, Christian, & Armitage, 2007), attendance in class was studied as afunction of conscientiousness, openness to experience, goal intentions, and implementation intentions.Increased class attendance due to planning occurred only for low/moderately conscientious students as highconscientious students showed a perfect class attendance to begin with. This finding is in line with therepeated observation (Gollwitzer & Sheeran, 2006) that implementation intention effects are stronger whenused for difficult rather than easy goals.

Moreover, implementation intention effects do not seem to depend on a person’s lack of self-regulatorycapacity (i.e., executive control resources; Hall, Zehr, Ng, & Zanna, 2012). It comes as no surprise then, thatimplementation intentions have been found to benefit children with attention deficit hyperactivity disorder(ADHD). According to the dual-pathway model (Sonuga-Barke, 2002), ADHD impairs behavioral controlin two ways: (a) through an inhibitory dysfunction leading to poor task engagement and inattentiveness; and(b) through a deregulation of reward mechanisms leading to a higher preference for immediate rewards.Children with ADHD benefit from forming implementation intentions by improving both functions (e.g.,Gawrilow & Gollwitzer, 2008; Gawrilow, Gollwitzer, & Oettingen, 2011a) as well as their ability to delaygratification (Gawrilow, Gollwitzer, & Oettingen, 2011b).

Summary. Forming implementation intentions is a volitional strategy that links cognitive, affective, orbehavioral responses that are instrumental to reaching desired outcomes to critical situational cues. As aconsequence, when the critical situation is encountered, the specified response is executed immediately,effortlessly, and without conscious intent. If-then planning can thus be understood as a self-regulation toolthat allows for strategically delegating one’s action control to critical situational cues.

There are two new lines of implementation intention research (see Gollwitzer, 2014) that are of relevanceto improving the cooperation, communication, and interaction between students, teachers, andadministrators. The first pertains to the use of implementation intentions in groups. This research askswhether individual group members can use implementation intentions to promote collaboration and thusimprove group performance, and whether groups can also use we-implementation intentions (“If weencounter . . . , then we will . . . !”) to promote group performance. The second new line of implementationintention research explores whether if-then plans can be used to benefit communication and social interaction.For instance, one question is whether implementation intentions can boost interest in sustained contact andclose interpersonal distance in anxiety-provoking interactions (e.g., interracial interactions).

Mental Contrasting with Implementation Intentions as a Metacognitive Intervention

245

The two volitional strategies of mental contrasting and implementation intentions have been combined toform a strategy called mental contrasting with implementation intentions (MCII). MCII is found to be moreeffective in changing behavior than each of the two alone, as the two strategies support each other. As mentalcontrasting of feasible wishes strengthens the non-conscious association between reality and instrumentalmeans (Kappes et al., 2012; Oettingen, 2012), explicitly forming implementation intentions strengthens thisassociation even further. Mental contrasting in turn benefits the effects of implementation intentions.Specifically, it prepares the application of implementation intentions in two ways: (a) Mental contrasting offeasible wishes fosters goal commitment and energization, and goal commitment is a necessary prerequisite forimplementation intentions to be effective (Sheeran et al., 2005); and (b) in mental contrasting theidiosyncratic obstacles and means to pursue the desired future are specified, so that the obstacle can work asthe if-component of a given implementation intention, and the instrumental means as the then-component.In sum, if-then plans as part of MCII may look like: “If . . . (obstacle), then I will . . . (respond to overcome orcircumvent the obstacle).”

MCII is more effective than MC and II alone. MCII has been found to be more effective than mentalcontrasting and forming implementation intentions alone (Adriaanse et al., 2010; Kirk, Oettingen, &Gollwitzer, 2013; see summary by Oettingen, 2012). For example, MCII helped college students more inbreaking snacking habits than mental contrasting only and forming implementation intention only.Importantly, mental contrasting did increase perceived clarity about personal obstacles towards reducingunhealthy snacking. These findings suggest that MCII may also be a valid strategy for fighting bad habits ineducational settings (e.g., procrastination).

What underlying processes make MCII so effective for behavior change? Mental contrasting creates clarityabout one’s personal obstacles which can then be used as critical cues in the formation of implementationintentions (e.g., if my friends call, then I will tell them that I need to do my homework). Indeed, whenAdriaanse, de Ridder, and de Wit (2009) compared the effectiveness of if-then plans that were personalizedvs. kept general (i.e., specifically referred to each participant’s unique action control problem vs. a generalaction control problem), it was the personalized if-then plans that turned out to be more effective.

MCII improves academic performance in schoolchildren. Duckworth, Grant, Loew, Oettingen, and Gollwitzer(2011) conducted an intervention study with university-bound high-school students preparing for thePreliminary SAT (PSAT) over the summer. Students first wrote down two positive outcomes they associatedwith completing all of the practice tests in the workbook (e.g. “I would feel good about myself”), and twoobstacles of the present reality (e.g. “I’m distracted”) that could interfere with this task. They then rewrote thepreviously stated first outcome, imagined it as vividly as possible, and then wrote their thoughts and imagesdown. This procedure was repeated for the first obstacle, the second named positive outcome, and the secondobstacle. Students then proposed a specific solution for each obstacle. Specifically, they completed two if–thenplans in the following way: “If (obstacle), then I will (solution).” Finally, each student received a 12th editionof Barron’s How to prepare for the PSAT workbook (Green & Wolf, 2004). These workbooks were collected inOctober, immediately after students had completed their PSAT. Students who applied MCII completed 60%more questions in their workbooks than control participants who had to write a short essay on an influentialperson or event in their life.

246

MCII also turned out to be helpful for the self-regulation of school-related concerns in middle-schoolschoolchildren at risk for ADHD (Gawrilow, Morgenroth, Schultz, Oettingen, & Gollwitzer, 2013).Students received a standard learning style or a learning style plus MCII intervention. The MCII pertained tostudents’ most important school-related concern (e.g., be more attentive in French class). When parents ratedtheir children’s management of school-related activities (e.g., homework is done reliably, vocabulary islearned, desk is tidy) two weeks later, both children at risk and not at risk for ADHD benefited from MCII,more than from the learning style intervention. Importantly, the more ADHD symptoms the children showedbefore the intervention, the more they benefited from the MCII intervention.

Economically disadvantaged middle-school children participated in a further MCII study (Duckworth,Kirby, A. Gollwitzer, & Oettingen, 2013). Prior to the intervention, teachers were asked to rate children intheir classroom behavior during the previous month. Baseline academic performance was assessed using threeindicators from the official record: grade point average (GPA), attendance, and conduct. At the beginning ofthe third quarter, children were randomly assigned to complete either the MCII or a positive thinking controlexercise. The children in both conditions targeted their most important personal wishes related to schoolwork.Trained interventionists met with the children in groups of four to five during three 1-hour sessions. After thethird quarter, the three indicators of academic performance (GPA, attendance, and conduct) were obtainedagain. Compared to children in the control condition, children that were taught how to apply MCIIsignificantly improved their GPA, attendance, and conduct.

Summary. MCII is a volitional strategy that combines two effective self-regulation strategies. By mentallycontrasting the desired future with the present reality, students, teachers, and administrators identify what inthemselves holds them back from attaining what they would like to achieve in the future. If they deem theirwished-for future as reachable, they become energized and actively pursue the desired future; if they deem it asfutile they let go and turn to alternative pursuits. Forming implementation intentions on top of mentalcontrasting enables them to master even highly challenging obstacles. MCII is easy to apply and particularlyeffective for people with special needs, such as children at risk for ADHD and children of low socioeconomicbackground. Therefore, MCII qualifies as an effective volitional strategy that children, teachers, andadministrators can use to better their everyday life and long-term development (for instructions of how tolearn and apply MCII in students and during one’s everyday life, see woopmylife.org. WOOP stands for wishoutcome obstacle plan).

247

http://woopmylife.org

Individual Differences in Volition

The previous sections focused on understanding the processes underlying volition and respective behaviorchange interventions. But there are also individual difference perspectives on volition (see Corno, 2001, onhabits). One line of research relates education outcomes to individual differences, such as theconscientiousness factor of the Big Five personality model, which encompasses dependability, punctuality, andorderliness (see Chapter 13, this volume; and McCrae & Costa, 1987). Another line (see Duckworth, 2009)distinguishes between two person-related attributes relevant to volition: grit and self-control.

Grit is defined as the tendency to sustain interest in and effort toward long-term goals; it is operationalizedusing the grit scale, a Likert-type self-report scale that includes items such as: “I am a hard worker” and “Ifinish whatever I begin.” Self-control is defined as voluntary regulation of behavioral, emotional, andattentional impulses in the presence of momentarily gratifying temptations or diversions; it is operationalizedusing a scale with items such as: “My mind wandered when I should have been listening” and “I talked back tomy teacher or parent when I was upset” (Duckworth & Carlson, 2013). Grit and self-control predictobjectively measured performance over and above measures of talent. For instance, in longitudinal studies, gritpredicts surviving the arduous first summer of training at West Point, reaching the final rounds of theNational Spelling Bee, retention in the U.S. Special Forces, retention and performance among noviceteachers, and graduation from Chicago public high schools. These predictions are observed after statisticallyadjusting for measures such as IQ, SAT, or standardized achievement test scores, as well as physical fitnessscores. In cross-sectional studies, grit correlates with lifetime educational attainment and, inversely, lifetimecareer changes and divorce.

Self-control predicts report card grades (and changes in report card grades over time) more strongly thanmeasures of intelligence (Duckworth, Tsukayama, & May, 2010). Finally, recent research has looked at twodistinct measures of academic performance—report card grades and standardized achievement test scores—and their different relations with self-control and intelligence. In three separate samples, self-controlprospectively predicted changes in report card grades more accurately than intelligence scores, but intelligencewas found to be a better predictor of changes in standardized achievement test scores (Duckworth, Quinn, &Tsukayama, 2012).

It is important to recognize that individual difference approaches to volition can easily be integrated intothe process models described above. For instance, Webb et al. (2007) used an implementation intentionintervention to help undergraduate college students to show up for class on time. They observed that onlystudents low in conscientiousness benefited from the implementation intention intervention (as the studentshigh in conscientiousness showed up on time to begin with). In other words, it was those students who hadproblems with showing up on time (i.e., the students who needed volition to overcome barriers to achievingthe desired outcome of being punctual) who benefited from employing a self-regulation strategy.

While Webb et al. (2007) combine personality and process approaches to volition in terms of moderation ofself-regulation processes by personality variables, there is also the possibility of combining the two approachesin terms of explicating distinct self-regulation processes for different types of people. This approach has beenexplored by Kuhl (1985) who differentiates individuals with an action orientation from those with a stateorientation (i.e., individuals who show high vs. low cognition-behavior consistencies). His extensive empirical

248

research (Kuhl, 2000; Kuhl, Kazén, & Koole, 2006) has by now delineated in detail what psychologicalprocesses (i.e., patterns of interactions among four cognitive systems: intention memory, extension memory,discrepancy-sensitive object recognition, and intuitive behavior control) ultimately lead to different levels ofself-regulatory abilities.

Corno et al. (2002) suggest that studying individual differences in self-regulation across school settingsmight benefit from differentially looking at cognitive versus affective versus conative (motivational andvolitional) individual differences (see also Corno, 2001). For example, with respect to affect, models may bedeveloped for studying the influence of anxiety and mood on self-regulation; these influences may bemoderated by temperament-related differences in reactivity and motivations, such as efficacy as well asproblem- versus emotion-focused styles of coping with stress (Boekaerts, 1987; see also Folkman & Lazarus,1985). Further, with respect to cognition, it might be worthwhile differentiating a deep approach toprocessing of information in learning situations versus a surface approach (Entwistle, 1989) and investigatinghow these different approaches relate to grit and self-regulation. Finally, with respect to conation, one mightwant to investigate how students’ work styles, such as detached and disengaged versus committed, hopeful,and engaged (Ainley, 1993), may determine to what extent students benefit from using various volitionalstrategies.

249

Conclusion and Future Research

One of the major challenges in education is to keep students striving for the attainment of future outcomesthat are beneficial for them and for their context. A first step to master this challenge is to strengthenstudents’ motivation by heightening both incentive value of academic achievement (desirability) and relevantexpectations (feasibility). But in addition to the motivational processes that establish desirability and feasibilityof academic goals, students need volition to ensure that they do not pull back from challenging tasks andlong-term goals in the face of resistance and conflict (e.g., do not give up in math, or drop out of school).Volitional strategies like mental contrasting and forming implementation intentions, and especially thecombination of the two, can help students reach attractive and attainable future outcomes by preparingthemselves to master upcoming obstacles and setbacks. A big advantage of MCII is its simplicity. It can betaught as a metacognitive strategy in a very short time, and it can be applied during everyday life with relativeease. Importantly, students need no special skills or personal attributes to learn and apply MCII as it can beacquired by students of different walks of life and cultures, and used in diverse contexts to solve a wide array ofdifferent tasks.

Because MCII can be used for any wish and concern, it should benefit the mastery of the various challengesarising from the individual vulnerabilities described above. For example, referring to the example of a deepversus surface approach to information processing, it would be important to investigate whether MCII andother self-regulation strategies (e.g., goal shielding, distancing) can help students to flexibly adopt one modeof information processing versus another (Entwistle, 1989). Research might test whether teaching MCII tostudents would foster the surface approach when preparing for an upcoming test, especially when there is onlya short time left to study. In contrast, for learning basic skills that a student needs in order to build a career ina particular field (e.g., basics in physics for a student aspiring to attend graduate school), MCII shouldpromote adopting a deep approach. By allowing the student to fully understand which future she wishes forand which of her own obstacles are in the way, MCII will provide clarity whether surface or deep informationprocessing is called for.

Similarly, future research may investigate MCII or other self-regulation procedures in the context of thedual processing self-regulation model (Boekaerts & Niemivirta, 2000). Specifically, MCII might readilysupport the growth pathway (top-down process of goal achievement) as the wished-for future in this casepertains to an improvement of the status quo. In the well-being pathway, students focus on preventingnegative futures. Here the desired future pertains to keeping the status quo (“How nice would it be if I keptmy GPA high?” or “How nice would it be if I continued to have a close relationship with my teacher?”).Alternatively, students could be asked to mentally contrast a potential negative future (e.g., “I might upset myteacher”) with the positive reality that they might lose (e.g., the close relationship to the teacher right now), sothat the students commit to preserving the valuable present reality. Such mental contrasting instills avoidance(rather than approach goals, e.g., Oettingen, Mayer, & Thorpe, 2010b), which may be particularly helpful ineducational settings whenever the well-being path is concerned and when emotion regulation is called for.

Finally, Corno et al. (2002) have noted that the research in education has not focused enough onhypotheses of how affect and conation relate to cognition. Our own research investigating self-regulationstrategies may be seen as a step in that direction. Research on MCII addresses the regulation of cognition

250

(e.g., stereotypic or schematic thinking), emotion (e.g., anxiety, anger), and behavior (effort, performance).Indeed, applying MCII to a particular wish (e.g., being more friendly to a teacher) will have downstreamconsequences benefiting all three areas—cognition (e.g., interpreting the teacher’s behavior in a more friendlylight), emotion (e.g., feeling better after interacting with the teacher), and conation (e.g., using a morerespectful tone of voice when talking to the teacher). Finally, engaging in MCII entails procedures thatinvolve all three pathways. It instigates cognition (e.g., mental associations outside of awareness), emotions(e.g., feelings of energy; anticipated disappointment), and behavior (e.g., fighting back in light of setbacks)that mediate changes in observable performance and actual success. Future research should also focus on howthese pathways interact when it comes to long-term consequences of MCII (e.g., to what extent do changes inemotion predict changes in performance, or the other way around).

In the present chapter, we defined volition and discussed respective processes that help to face resistanceand resolve conflict in goal pursuit and behavior change. We also identified effective volitional strategies thatstudents can learn and then apply on their own. These strategies support students, teachers, andadministrators in identifying what they really want in the future and what kinds of obstacles stand in theirway; they also help individuals to make plans and to ultimately overcome these obstacles. Importantly, fortheory and research, these volitional strategies build on existing incentive values and expectations of success,thereby translating motivation into behavior change and goal attainment.

251

References

Achtziger, A., Bayer, U. C., & Gollwitzer, P. M. (2012). Committing to implementation intentions: Attention and memory effects for selectedsituational cues. Motivation and Emotion, 36, 287–300.

Achtziger, A., Gollwitzer, P. M., & Sheeran, P. (2008). Implementation intentions and shielding goal striving from unwanted thoughts andfeelings. Personality and Social Psychology Bulletin, 34, 381–393.

Adriaanse, M. A., de Ridder, D. T. D., & de Wit, J. B. F. (2009). Finding the critical cue: Implementation intentions to change one’s diet workbest when tailored to personally relevant reasons for unhealthy eating. Personality and Social Psychology Bulletin, 35, 60–71.

Adriaanse, M. A., Gollwitzer, P. M., de Ridder, D. T. D., de Wit, J. B. F., & Kroese, F. M. (2011). Breaking habits with implementationintentions: A test of underlying processes. Personality and Social Psychology Bulletin, 37, 502–512.

Adriaanse, M. A., Oettingen, G., Gollwitzer, P. M., Hennes, E. P., de Ridder, D. T. D., & de Wit, J. B. F. (2010). When planning is notenough: Fighting unhealthy snacking habits by mental contrasting with implementation intentions (MCII). European Journal of SocialPsychology, 40, 1277–1293.

Adriaanse, M. A., van Oosten, J. M., de Ridder, D. T., de Wit, J. B., & Evers, C. (2011). Planning what not to eat: Ironic effects ofimplementation intentions negating unhealthy habits. Personality and Social Psychology Bulletin, 37, 69–81.

Ainley, M. (1993). Styles of engagement with learning: A multidimensional assessment of their relationship with strategy use and schoolachievement. Journal of Educational Psychology, 85, 395–405.

Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64, 359–372.Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215.Bayer, U. C., Achtziger, A., Gollwitzer, P. M., & Moskowitz, G. B. (2009). Responding to subliminal cues: Do if-then plans facilitate action

preparation and initiation without conscious intent? Social Cognition, 27, 183–201.Bayer, U. C., & Gollwitzer, P. M. (2007). Boosting scholastic test scores by willpower: The role of implementation intentions. Self and Identity,

6, 1–19.Boekaerts, M. (1987). Individual differences in the appraisal of learning tasks: An integrative view on emotion and cognition. Communication

and Cognition, 20, 207–224.Boekaerts, M., & Cascallar, E. (2006). How far have we moved toward the integration of theory and practice in self-regulation? Educational

Psychology Review, 18, 199–210.Boekaerts, M. & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and intervention. Applied Psychology: An

International Review, 54, 199–231.Boekaerts, M., & Niemivirta, M. (2000). Self-regulated learning: Finding a balance between learning goals and ego-protective goals. In M.

Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 417–451). San Diego, CA: Academic Press.Brandstätter, V., Lengfelder, A., & Gollwitzer, P. M. (2001). Implementation intentions and efficient action initiation. Journal of Personality

and Social Psychology, 81, 946–960.Burgess, P. W., Dumontheil, I., & Gilbert, S. J. (2007). The gateway hypothesis of rostral PFC (Area 10) function. Trends in Cognitive Sciences,

11, 290–298.Carver, C. S., & Scheier, M. (1998). On the self-regulation of behavior. New York, NY: Cambridge University Press.Cohen, A-L., Bayer, U. C., Jaudas, A., & Gollwitzer, P. M. (2008). Self-regulatory strategy and executive control: Implementation intentions

modulate task switching and Simon task performance. Psychological Research, 72, 12–26.Corno, L. (1993). The best-laid plans: Modern conceptions of volition and educational research. Educational Researcher, 22, 14–22.Corno, L. (1994). Student volition and education: Outcomes, influences, and practices. In D. Schunk & B. Zimmerman (Eds.), Self-regulation

of learning and performance: Issues and applications (pp. 229–254). Hillsdale, NJ: Erlbaum.Corno, L. (2001). Volitional aspects of self-regulated learning. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic

achievement: Theoretical perspectives (2nd ed., pp. 191–226). Mahwah, NJ: Lawrence Erlbaum.Corno, L. (2004). Introduction to the special issue work habits and work styles: Volition in education. Teachers College Record, 106, 1669–1694.Corno, L. (2011). Studying self-regulation habits. In B. Zimmerman & D. Schunk (Eds.), Handbook of self-regulation of learning and

performance (pp. 361–375). New York: Routledge.Corno, L., Cronbach, L. J., Kupermintz, H., Lohman, D., Mandinach, E.B., Porteus, A. W., . (2002). Remaking the concept of aptitude:

Extending the legacy of R. E. Snow. Mahwah, NJ: Lawrence Erlbaum Associates.Corno, L., & Kanfer, R. (1993). The role of volition in learning and performance. Review of Research in Education, 19, 301–341.Duckworth, A. L. (2009). (Over and) beyond high-stakes testing. American Psychologist, 64, 279–280.Duckworth, A. L., & Carlson, S. M. (2013). Self-regulation and school success. In B. W. Sokol, F. M. E. Grouzet, & U. Müller (Eds.), Self-

regulation and autonomy: Social and developmental dimensions of human conduct (pp. 208–230). New York: Cambridge University Press.

252

Duckworth, A. L., Grant, H., Loew, B., Oettingen, G., & Gollwitzer, P. M. (2011). Self-regulation strategies improve self-discipline inadolescents: Benefits of mental contrasting and implementation intentions. Educational Psychology, 31, 17–26.

Duckworth, A. L., Kirby, T. A., Gollwitzer, A., & Oettingen, G. (2013). From fantasy to action: Mental Contrasting with ImplementationIntentions (MCII) improves academic performance in children. Social Psychological and Personality Science, 4, 745–753.

Duckworth, A. L., Quinn, P., & Tsukayama, E. (2012). What No Child Left Behind leaves behind: The roles of IQ and self-control inpredicting standardized achievement test scores and report card grades. Journal of Educational Psychology, 104, 439–451.

Duckworth, A. L., Tsukayama, E., & May, H. (2010). Establishing causality using longitudinal hierarchical linear modeling: An illustrationpredicting achievement from self-control. Social Psychology and Personality Science, 1, 311–317.

Duffy, E. (1934). Emotion: An example of the need for reorientation in psychology. Psychological Review, 41, 184–198.Entwistle, N. J. (1989). Styles of learning and teaching: An integrated outline of educational psychology for students, teachers and lecturers. London,

UK: David Fulton Publishers.Fishbach, A., Dhar, R., & Zhang, Y. (2006). Subgoals as substitutes or complements: The role of goal accessibility. Journal of Personality and

Social Psychology, 91, 232–242.Fishbach, A., Shah, J. Y., & Kruglanski, A. W. (2004). Emotional transfer in goal systems. Journal of Experimental Social Psychology, 40, 723–

738.Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906–

911.Folkman, S., & Lazarus, R. S. (1985). If it changes it must be a process: Study of emotion and coping during three stages of a college

examination. Journal of Personality and Social Psychology, 48, 150–170.Gallo, I. S., Keil, A., McCulloch, K. C., Rockstroh, B., & Gollwitzer, P. M. (2009). Strategic automation of emotion regulation. Journal of

Personality and Social Psychology, 96, 11–31.Gawrilow, C., & Gollwitzer, P. M. (2008). Implementation intentions facilitate response inhibition in children with ADHD. Cognitive Therapy

and Research, 32, 261–280.Gawrilow, C., Gollwitzer, P. M., & Oettingen, G. (2011a). If-then plans benefit executive functions in children with ADHD. Journal of Social

and Clinical Psychology, 30, 616–646.Gawrilow, C., Gollwitzer, P. M., & Oettingen, G. (2011b). If-then plans benefit delay of gratification performance in children with and

without ADHD. Cognitive Therapy and Research, 35, 442–455.Gawrilow, C., Morgenroth, K., Schultz, R., Oettingen, G., & Gollwitzer, P. M. (2013). Mental contrasting with implementation intentions

enhances self-regulation of goal pursuit in schoolchildren at risk for ADHD. Motivation and Emotion, 37, 134–145.Gilbert, S. J., Gollwitzer, P. M., Cohen, A. L., Oettingen, G., & Burgess, P. W. (2009). Separable brain systems supporting cued versus self-

initiated realization of delayed intentions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 905–915.Gollwitzer, A., Oettingen, G., Kirby, T. A., Duckworth, A. L., & Mayer, D. (2011). Mental contrasting facilitates academic performance in

school children. Motivation and Emotion, 35, 403–412.Gollwitzer, P. M. (1990). Action phases and mind-sets. In E. T. Higgins & R. M. Sorrentino (Eds.), Handbook of motivation and cognition:

Foundations of social behavior (Vol. 2, pp. 53–92). New York, NY: Guilford Press.Gollwitzer, P. M. (1993). Goal achievement: The role of intentions. European Review of Social Psychology, 4, 141–185.Gollwitzer, P. M. (1999). Implementation intentions: Strong effects of simple plans. American Psychologist, 54, 493–503.Gollwitzer, P. M. (2012). Mindset theory of action phases. In P. Van Lange, A. W. Kruglanski, & E. T. Higgins (Eds.), Handbook of theories of

social psychology (Vol. 1, pp. 526–545). London: Sage Publications.Gollwitzer, P. M. (2014). Weakness of the will: Is a quick fix possible? Motivation and Emotion, 38, 305–322.Gollwitzer, P. M., Bayer, U. C., & McCulloch, K. C. (2005). The control of the unwanted. In R. Hassin, J. Uleman, & J. A. Bargh (Eds.), The

new unconscious (pp. 485–515). Oxford: Oxford University Press.Gollwitzer, P. M., & Brandstätter, V. (1997). Implementation intentions and effective goal pursuit. Journal of Personality and Social Psychology,

73, 186–199.Gollwitzer, P. M., & Oettingen, G. (2011). Planning promotes goal striving. In K. D. Vohs & R. F. Baumeister (Eds.), Handbook of self-

regulation: Research, theory, and applications (2nd ed., pp. 162–185). New York, NY: Guilford.Gollwitzer, P. M., & Schaal, B. (1998). Metacognition in action: The importance of implementation intentions. Personality and Social

Psychology Review, 2, 124–136.Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances

in Experimental Social Psychology, 38, 69–119.Green, S. W., & Wolf, I. K. (2004). Barron’s how to prepare for the PSAT/NMSQT: PSAT/national merit scholarship qualifying test (12th ed.).

Hauppauge, NY: Barron’s Educational Series.

253

Hall, P. A., Zehr, C., Ng, M., & Zanna, M. P. (2012). Implementation intentions for physical activity in supportive and unsupportiveenvironmental conditions: An examination of intention-behavior consistency. Journal of Experimental Social Psychology, 48, 432–436.

Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills intervention on student learning: A meta-analysis. Review of EducationalResearch, 66, 99–136.

Henderson, M. D., Gollwitzer, P. M., & Oettingen, G. (2007). Implementation intentions and disengagement from a failing course of action.Journal of Behavioral Decision Making, 20, 81–102.

Hull, C. L. (1943). Principles of behavior: An introduction to behavior theory. The Century Psychology Series. New York, NY: Appleton-Century-Crofts.

James, W. (1890). The principles of psychology. New York, NY: Holt.Johannessen, K. B., Oettingen, G., & Mayer, D. (2012). Mental contrasting of a dieting wish improves self-reported health behavior. Psychology

& Health, 27, 43–58.Kappes, A., & Oettingen, G. (2014). The emergence of goal pursuit: Mental contrasting connects future and reality. Journal of Experimental

Social Psychology, 54, 25–39.Kappes, A., Oettingen, G., & Pak, H. (2012). Mental contrasting and the self-regulation of responding to negative feedback. Personality and

Social Psychology Bulletin, 38, 845–857.Kappes, A., Wendt, M., Reinelt, T., & Oettingen, G. (2013). Mental contrasting changes the meaning of reality. Journal of Experimental Social

Psychology, 49, 797–810.Kirk, D., Oettingen, G., & Gollwitzer, P. M. (2013). Promoting integrative bargaining: Mental contrasting with implementation intentions.

International Journal of Conflict Management, 24, 148–165.Knäuper, B., Roseman, M., Johnson, P. J., & Krantz, L. H. (2009). Using mental imagery to enhance the effectiveness of implementation

intentions. Current Psychology, 28, 181–186.Kuhl, J. (1985). Volitional mediators of cognition-behavior consistency: Self-regulatory processes and action versus state orientation. In J. Kuhl

& J. Beckmann (1985), Action control. From cognition to behavior (pp. 101–128). New York, NY: SpringerKuhl, J. (2000). The volitional basis of Personality Systems Interaction Theory: Applications in learning and treatment contexts. International

Journal of Educational Research, 33, 665–703.Kuhl, J., & Beckmann, J. (1994). Volition and personality: Action versus state orientation. Goettingen: Hogrefe & Huber.Kuhl, J., Kazén, M., & Koole, S. L. (2006). Putting self-regulation theory into practice: A user’s manual. Applied Psychology: An International

Review, 55, 408–418.Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall.Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions in Psychological Science, 15, 265–268.Loft, M. H., & Cameron, L. D. (2013). Using mental imagery to deliver self-regulation techniques to improve sleep behaviors. Annals of

Behavioral Medicine, 46, 260–272.Louro, M. J., Pieters, R., & Zeelenberg, M. (2007). Dynamics of multiple-goal pursuit. Journal of Personality and Social Psychology, 93, 174–193.McClelland, D. C. (1985). How motives, skills, and values determine what people do. American Psychologist, 40, 812–825.McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality

and Social Psychology, 52, 81–90.Mendoza, S. A., Gollwitzer, P. M., & Amodio, D. M. (2010). Reducing the expression of implicit stereotypes: Reflexive control through

implementation intentions. Personality and Social Psychology Bulletin, 36, 512–523.Mischel, W. (1973). Toward a cognitive social learning reconceptualization of personality. Psychological Review, 80, 252–283.Muraven, M., & Baumeister, R. F. (2000). Self-regulation and depletion of limited resources: Does self-control resemble a muscle? Psychological

Bulletin, 126, 247–259.Oettingen, G. (2000). Expectancy effects on behavior depend on self-regulatory thought. Social Cognition, 18, 101–129.Oettingen, G. (2012). Future thought and behavior change. European Review of Social Psychology, 23, 1–63.Oettingen, G., Hönig, G., & Gollwitzer, P. M. (2000). Effective self-regulation of goal attainment. International Journal of Educational

Research, 33, 705–732.Oettingen, G., Marquardt, M. K., & Gollwitzer, P. M. (2012). Mental contrasting turns positive feedback on creative potential into successful

performance. Journal of Experimental Social Psychology, 48, 990–996.Oettingen, G., & Mayer, D. (2002). The motivating function of thinking about the future: Expectations versus fantasies. Journal of Personality

and Social Psychology, 83, 1198–1212.Oettingen, G., Mayer, D., & Brinkmann, B. (2010a). Mental contrasting of future and reality: Managing the demands of everyday life in health

care professionals. Journal of Personnel Psychology, 9, 138–144.Oettingen, G., Mayer, D., Sevincer, A. T., Stephens, E. J., Pak, H.-J., & Hagenah, M. (2009). Mental contrasting and goal commitment: The

254

mediating role of energization. Personality and Social Psychology Bulletin, 35, 608–622.Oettingen, G., Mayer, D., & Thorpe, J. (2010b). Self-regulation of commitment to reduce cigarette consumption: Mental contrasting of future

with reality. Psychology and Health, 25, 961–977.Oettingen, G., Mayer, D., Thorpe, J. S., Janetzke, H., & Lorenz, S. (2005). Turning fantasies about positive and negative futures into self-

improvement goals. Motivation and Emotion, 29, 237–267.Oettingen, G., Pak, H.-J., & Schnetter, K. (2001). Self-regulation of goal-setting: Turning free fantasies about the future into binding goals.

Journal of Personality and Social Psychology, 80, 736–753.Oettingen, G., Stephens, E. J., Mayer, D., & Brinkmann, B. (2010c). Mental contrasting and the self-regulation of helping relations. Social

Cognition, 28, 490–508.Oettingen, G., Wittchen, M., & Gollwitzer, P. M. (2013). Regulating goal pursuit through mental contrasting with implementation intentions.

In E. A. Locke & G. P. Latham (Eds.), New developments in goal setting and task performance (pp. 523–548). New York, NY: Brunner-Routledge.

Parks-Stamm, E. J., Gollwitzer, P. M., & Oettingen, G. (2007). Action control by implementation intentions: Effective cue detection andefficient response initiation. Social Cognition, 25, 248–266.

Parks-Stamm, E. J., Gollwitzer, P. M., & Oettingen, G. (2010). Implementation intentions and test anxiety: Shielding academic performancefrom distraction. Learning and Individual Differences, 20, 30–33.

Perels, F., Dignath, C., & Schmitz, B. (2009). Is it possible to improve mathematical achievement by means of self-regulation strategies?Evaluation of an intervention in regular math class. European Journal of Psychology of Education, 24, 17–31.

Pressley, M., Woloshyn, V., Lysynchuk, L. M., Martin, V., Wood, E., & Willoughby, T. (1990). A primer of research on cognitive strategyinstruction: The important issues and how to address them. Educational Psychology Review, 2, 1–58.

Randi, J. (2005). Teachers as self-regulated learners. Teachers College Record, 10, 1825–1853.Schunk, D. H. & Zimmerman, B. J. (2007). Influencing children’s self-efficacy and self-regulation of reading and writing through modeling.

Reading & Writing Quarterly: Overcoming Learning Difficulties, 23, 7–25.Sevincer, A. T., Busatta, P. D., & Oettingen, G. (2014). Mental contrasting and transfer of energization. Personality and Social Psychology

Bulletin, 40, 139–152.Sevincer, A. T., & Oettingen, G. (2013). Spontaneous mental contrasting and selective goal pursuit. Personality and Social Psychology Bulletin,

39, 1240–1254.Shah, J. Y., Friedman, R., & Kruglanski, A. W. (2002). Forgetting all else: On the antecedents and consequences of goal shielding. Journal of

Personality and Social Psychology, 83, 1261–1280.Sheeran, P., Webb, T. L., & Gollwitzer, P. M. (2005). The interplay between goal intentions and implementation intentions. Personality and

Social Psychology Bulletin, 31, 87–98.Sonuga-Barke, E. J. (2002). Psychological heterogeneity in AD/HD—A dual pathway model of behavior and cognition. Behavioural Brain

Research, 130, 29–36.Stewart, B. D., & Payne, B. K. (2008). Bringing automatic stereotyping under control: Implementation intentions as efficient means of thought

control. Personality and Social Psychology Bulletin, 34, 1332–1345.Tolman, E. (1932). Purposive behavior in animals and men. Berkeley, CA: University of California Press.Varley, R., Webb, T. L., & Sheeran, P. (2011). Making self-help more helpful: A randomized controlled trial of the impact of augmenting self-

help materials with implementation intentions in promoting the effective self-management of anxiety symptoms. Journal of Consulting andClinical Psychology, 79, 123–128.

Webb, T. L., Christian J., & Armitage, C. J. (2007). Helping students turn up for class: Does personality moderate the effectiveness of animplementation intention intervention? Learning and Individual Differences, 17, 316–327.

Webb, T. L., Onanaiye, M., Sheeran, P., Reidy, J., & Lavda, S. (2010). Using implementation intentions to modify the effects of social anxietyon attention and responses to evaluative situations. Personality and Social Psychology Bulletin, 36, 612–627.

Webb, T. L., & Sheeran, P. (2003). Can implementation intentions help to overcome ego-depletion? Journal of Experimental Social Psychology,39, 279–286.

Webb, T. L., & Sheeran, P. (2007). How do implementation intentions promote goal attainment? A test of component processes. Journal ofExperimental Social Psychology, 43, 295–302.

Webb, T. L., & Sheeran, P. (2008). Mechanisms of implementation intention effects: The role of goal intentions, self-efficacy, and accessibilityof plan components. British Journal of Social Psychology, 47, 373–395.

Webb, T. L., Sheeran, P., & Luszczynska, A. (2009). Planning to break unwanted habits: Habit strength moderates implementation intentioneffects on behaviour change. British Journal of Social Psychology, 48, 507–523.

Wieber, F., & Gollwitzer, P. M. (2010). Overcoming procrastination through Implementation Intentions. In C. Andreou and M. D. White

255

(Eds.), The thief of time: Philosophical essays on procrastination (pp. 185–205). New York: Oxford University Press.Wieber, F., von Suchodoletz, A., Heikamp, T., Trommsdorff, G., & Gollwitzer, P. M. (2011). If-then planning helps school-aged children to

ignore attractive distractions. Social Psychology, 42, 39–47.Xu, J., Yuan, R., Xu, B., & Xu, M. (2014). Modeling students’ time management in math homework. Learning and Individual Differences, 34,

33–42.

256

Part IIILearner Readiness and Development

257

9Human Cognitive Abilities

Their Organization, Development, and Use

PATRICK C. KYLLONEN

Educational Testing Service Some students learn quickly; others take much longer. One explanation is that students come to school withwidely varying learning experiences and are differentially prepared. Another is that students differ in theircognitive abilities. Even if students came to school with the same set of learning experiences, they would stilldiffer in how quickly they learned due to ability differences. These explanations are not incompatible andmost people—experts and non-experts—believe both to be true. Their relative importance has been thesubject of heated debate for decades, if not centuries, and is unlikely to be resolved soon. This chaptersummarizes what we know about human abilities, framed in the context of their measurement andorganization. It also discusses cognitive ability development and the use of ability measures.

258

The Organization of Human Abilities

The General (g) Factor

The idea that people can be characterized by ability has been with us for centuries, but the scientificidentification of human cognitive ability is typically attributed to Spearman (1904). Spearman studiedperformance on a wide variety of school (e.g., spelling, arithmetic) and psychology laboratory tasks (e.g.,susceptibility to illusions) and found that performance on one task positively predicted performance on theothers, a finding known as positive manifold. He further proposed a simple mathematical model, vanishingtetrad differences, to account for the correlations: For any subset of four tests, the product of the correlations ofany two of them equals the product of the correlations of any other two, indicating unidimensionality.Spearman proposed general ability, or g, as an explanation for positive manifold, and the sufficiency of g,combined with test specificities, which he called s, as a way to characterize human abilities.

Spearman’s finding of a general ability factor has held up remarkably well. To illustrate, the Program forInternational Student Assessment (PISA), which measures 15-year-old students’ cognitive abilities inapproximately 70 countries, found high correlations among three achievement domains: r (mathematics,reading) = 0.84, r (mathematics, science) = 0.89, and r (reading, science) = 0.87 (OECD, 2012, Table 12.4),suggesting a strong, common factor. Similar results are found for all four PISA cycles, including problemsolving for PISA 2003 (r = 0.89, 0.82, 0.78, between problem solving and mathematics, reading, and science,respectively).

Group Factors

Despite the power of the g factor in predicting performance on cognitive tests, since Spearman many studieshave sought to identify additional factors. A key figure was Thurstone (1934), who proposed that in a batteryof tests there might be group factors with several tests having something in common beyond the general factor,for example, “the ability to write fast, facility with geometrical figures, or a large vocabulary” (p. 4). Thurstoneintroduced exploratory factor analysis to test this hypothesis, in which factors are extracted successively from acorrelation matrix, with communality estimates on the diagonal of that matrix, and rotation of axes to simplestructure. This method led to the primary mental abilities proposal (Thurstone, 1938)—word fluency, verbalcomprehension, spatial visualization, number facility, associative memory, reasoning, and perceptual speed.Later research programs, such as Guilford’s (1956), attempted to identify an even larger number of cognitiveability factors.

Although historical accounts often cast “general” vs. “multiple intelligences” perspectives as opposing, theyare reconcilable. Thurstone (1947) suggested that correlated group factors indicated a g factor, and Holzingerand Swineford (1937) introduced a bifactor method that identified both general and group factors (seeGustafsson & Balke, 1993). Schmid and Leiman (1957) developed a hierarchical method of extracting factors,and Vernon (1965) summarized research on a variety of tests best summarized by a hierarchical model.(Hierarchical and bifactor models themselves are compatible: Yung, Thissen, & McLeod, 1999.) The conceptof a general factor and more specific group factors is not completely dependent on factor analysis: It is alsocaptured with alternative methods of characterizing correlation matrices such as non-metric multidimensional

259

scaling (Snow, Kyllonen, & Marshalek, 1984).

The Fluid-Crystallized Distinction

A perspective not strictly compatible with the Spearman–Thurstone hierarchical framework is the Horn–Cattell (1966) model, which distinguishes two general abilities. Fluid intelligence (Gf) is the ability calledupon during abstract reasoning and problem solving, particularly in novel situations; crystallized intelligence(Gc) reflects conceptual and verbal knowledge acquired through education and experience. Fluid ability isinvested in learning to yield crystallized ability (Horn & Cattell, 1967). A source of evidence for the Gf–Gcdistinction is differential growth patterns: Gf peaks as early as in one’s twenties and declines slowly thereafter;in contrast, Gc grows, or at least does not decline until much later in life.

So, are there one or two gs? Gustafsson (1984) proposed that there is only one g, but it is identical to Gf.He based this on descriptions of g and Gf by various researchers, and showed that the two were factoranalytically identical. Some studies have not corroborated this result, and so Valentin Kvist and Gustafsson(2008) suggested that this might be due to the effects of differential opportunities to learn. They found thatwithin culturally hom*ogeneous groups (Swedish non-immigrants, European immigrants, and non-Europeanimmigrants) g and Gf were identical (r = 1.0), and only when the data were pooled did the g–Gf correlationattenuate (to r = 0.83). That is, in a group with roughly the same opportunities to learn (e.g., Swedish non-immigrants) the expected r (g, Gf) = 1.0 relationship was observed, but when groups with differingopportunities to learn (e.g., Swedish native speakers vs. immigrants who spoke Swedish as a second language)were pooled, the correlation was reduced because a factor in addition to Gf, opportunity to learn, contributedto performance on other g measures.

The Consensus Model: Carroll and CHC

Carroll (1993) summarized essentially all available research on human cognitive abilities by reanalyzingapproximately 450 datasets (with several thousand test scores) using an exploratory hierarchical method. Theresults suggested a three-stratum model with g at the third stratum, eight factors at the second stratum—fluidintelligence (Gf), crystallized intelligence (Gc), general learning and memory (Gy), broad visual perception(Gv), broad auditory perception (Ga), broad retrieval ability (Glr), broad cognitive speediness (Gs), andreaction time/decision speed (Gt)—and numerous factors at the third stratum (e.g., within Gf there is generalsequential reasoning, induction, quantitative reasoning, and Piagetian reasoning; within Gc there is languagedevelopment, verbal language comprehension, lexical knowledge, and 13 more abilities).

A combination of the Carroll (1993) framework, acknowledging the importance of the Gf–Gc distinction(Cattell, 1941) and the expansion of Gf–Gc into a broader abilities framework (e.g., Horn, 1989), is nowcommonly referred to as CHC (Cattell–Horn–Carroll) theory (McGrew, 2009). The CHC framework servesas the basis for several commercial intelligence test batteries (Kaufman & Kaufman, 2004; Woodco*ck,McGrew, & Mather, 2007) and is regarded as the foundation for psychoeducational assessment in schoolpsychology (Flanagan & Harrison, 2012).

260

261

Table 9.1

Critiques: Achievements and Limitations of the Consensus Abilities Framework

The consensus cognitive abilities framework can be seen as representing a kind of “end of history” of humanabilities. In 1994, Sternberg said of Carroll’s (1993) tour de force that, “There is nothing like it, nor is therelikely to be”; it represents “a culmination . . . of the psychometric approach to intelligence” (p. 65). This waswritten over 20 years ago, and there has not been a game-changing critique of the general framework duringthat time. Sternberg in that same article suggested that the Carroll framework was in danger of not appealingto as wide an array of “stakeholders” as newer alternatives, such as Gardner’s (1983) multiple intelligencestheory and his own practical intelligence framework, which takes context into account (Sternberg & Wagner,1986). However, these alternatives have not fared well in the interim (e.g., Gottfredson, 2003; McDaniel &Whetzel, 2005; Visser, Ashton, & Vernon, 2006a, 2006b).

Still, the consensus abilities framework can be improved and elaborated upon as evidence accrues within thefactor-analytic tradition (e.g., Carroll, 2003). McGrew (2009) provided justifications for new abilities such astactile (Gh), kinesthetic (Gk), and olfactory (Go) abilities; domain-specific knowledge (Gkn); psychom*otorability (Gp); and psychom*otor speed (Gps). Johnson and Bouchard (2005) proposed a related model, calledthe g-VPR (for verbal, perceptual, and image rotation), which does not seem qualitatively different from theCHC model, but has a different emphasis. This model may be more useful in studies emphasizing theneuroscientific underpinnings of cognitive ability. In particular, Hunt (2011) argues that, consistent with theg-VPR model, there is evidence for separate brain systems handling language and perceptual processing, andthat there is a biological distinction that mirrors the distinction between mental rotation (where male–femaledifferences are large) and the analysis of static figures (where the distinctions are small or favor women).

It is useful to consider a broader perspective on the status of the consensus abilities framework. Certainlythe cognitive abilities measurement framework is sound and useful, but it is not universally accepted as a basisfor understanding human abilities. Table 9.1 presents a list of some of the key criticisms, which are elaboratedon in later sections of this chapter.

Common Criticisms of the Consensus Framework on Cognitive Ability

1. Tests are not authentic or relevant to real life. Tests are too brief a sample of behavior and there is a need for authentic assessment (Hakel,1998).Counter: “Authentic tasks” duplicate traditional tests (e.g., Sonnleitner, Keller, Martin, & Brunner, 2013). Tests predict real-worldoutcomes.

2. The consensus framework is limited to the tests analyzed. This is a theory about tests that happen to have been developed, particularlypaper-and-pencil multiple-choice tests, not about human abilities per se.Counter: Alternative formats have been evaluated (e.g., motion picture tests during the 1950s). Still, technology opens the door to newkinds of measures, such as videos, simulations, eye movements, and physiological responses.

3. The methodology of the analysis of test score covariances is limited. Additional methodologies should be used. Intelligence “is tooimportant to be left to the psychometricians” (Gardner, 1999).Counter: Psychometricians are exploring alternative methodologies for measuring cognitive ability using games and simulations (e.g.,Mislevy et al., 2014), data mining (He, 2013), and physiological neuroscience measurements (Kane & Engle, 2002), among others.

4. Abilities are fixed, if not inherited, and there is not much that can be done about them in school.Counter: Abilities are not fixed—schooling reliably improves them (Brinch & Galloway, 2011), and a steady growth in test scores overthe past half-century suggests that abilities, measured by tests, improve (Flynn, 2007).

5. We do not know what human abilities are, or what tests actually measure: “intelligence as a measurable capacity must at the start bedefined as the capacity to do well in an intelligence test. Intelligence is what the tests test” (Boring, 1923).

262

Counter: There has been a concerted effort to understand the meaning of test scores from a basic psychological (Hunt, 2012) andneuroscientific perspective (e.g., see Chapter 5, this volume).

6. There are factors that threaten the validity of test scores, such as fatigue, emotional strain, practice, room temperature, affect, test-wiseness, motivation (Cronbach, 1970).Counter: This is an active area of research with studies on the effects of score disclosure and financial incentives (Duckworth, Quinn,Lynam, Loeber, & Stouthamer-Loeber, 2011; Liu, Bridgeman, & Adler, 2013), stereotype threat (Ganley, Mingle, Ryan, Ryan,Vasilyeva, & Perry, 2013; Steele & Aronson, 1995), and beliefs about the nature of intelligence (Dweck and Master, 2009).

7. Tests may be unfair to subgroups.Counter: The importance of establishing fairness is recognized in the Test Standards (American Educational Research Association,American Psychological Association, National Council on Measurement in Education, 2014).

8. Aptitude (ability) and achievement are clearly separate concepts.Counter: Since Binet (1905) ability tests have been cognitive tasks drawn from school curricula that separate those who do well in schoolfrom those who do poorly (e.g., vocabulary, reading comprehension, sentence completion, arithmetic). There is not a distinction in theheritability of Gf and Gc (e.g., Mackintosh, 2011, Chapter 2). Secular growth (Flynn, 2007) affects Gf more than Gc, Although there aredifferences between Gf and Gc measures, the two factors are so highly correlated that the distinction can be ignored for many purposes(e.g., Hunt, 2012).

263

Individual-Level Validity

Cognitive test scores are relatively rank-order stable over time. For example, Deary, Whalley, Lemmon,Crawford, and Starr (2000) readministered a general abilities test, the Moray House Test, consisting of verbal,spatial, numerical, and reasoning items, to a group of 77-year-olds who had taken the same test 66 yearsearlier as 11-year-olds, and found a correlation of r = 0.63 (adjusted to 0.73 when correcting for rangerestriction in the retested sample). Thirteen years later, with the 90-year-old cohort group (and a 77-year timedifference) the correlation was r = 0.54 (adjusted to 0.67 when correcting for range restriction) (Deary, Pattie,& Starr, 2013).

Predictions of Education and Workplace Success

But do cognitive test scores predict other life events? From its very beginnings, the rise in popularity andscientific acceptance of modern cognitive abilities measurement has been tied to its capability for predictingimportant and complex real-world outcomes relatively efficiently. The world’s first intelligence test, the Binet-Simon scale, was promoted for its ability to differentiate normal from mentally challenged schoolchildren inFrance. The Army Alpha test was lauded for its success “(a) to aid in segregating the mentally incompetent,(b) to classify men according to their mental capacity, (c) to assist in selecting competent men for responsiblepositions.” (Yoachum & Yerkes, 1920, p. xi). Since then hundreds of studies have been conducted on therelationship between cognitive ability test scores and various life outcomes. Thorndike (1986) showed acorrelation between an overall score on the Differential Aptitude Test Battery and grades (across manyschools) of r = 0.5–0.6. In higher education admissions studies, cognitive ability measured by admissions testsof verbal comprehension, mathematics reasoning, and logical reasoning predict outcomes such as first-yeargrade point average (r = 0.35–0.60), degree completion (r = 0.10–0.40), qualifying and licensure exams (r =0.30–0.65) and research productivity and publication citations (r = 0.10–0.25) (Kuncel & Hezlett, 2007).

Cognitive ability measures also predict success in the workplace. At the high end of ability, Lubinski,Benbow and colleagues have shown that students scoring relatively high on the SAT at age 13 are far morelikely than others to attain a degree, get a doctorate, publish novels, have high incomes, secure patents, and behappy (Ferriman-Robertson, Smeets, Lubinski, & Benbow, 2010; Lubinski & Benbow, 2006; Park, Lubinski,& Benbow, 2013). In the normal-ability range, Schmidt and Hunter (1998) suggested high correlationsbetween cognitive test scores and workplace outcomes (r = 0.56 with test scores in job training courses and r =0.51 with supervisor ratings, after range restriction adjustments). Ree and Earles (1992) similarly found a highcorrelation (r = 0.42, no range restriction adjustment) between a first principal component score from theArmed Services Vocational Aptitude Battery (ASVAB, comprising ten diverse subtests—general science,arithmetic reasoning, word knowledge, paragraph comprehension, math knowledge, electronics information,auto information, shop information, mechanical comprehension, and assembling objects) and U.S. Air Forcetraining grades (from multiple-choice knowledge tests from 82 job training courses, ranging from electronicstroubleshooting to security police, with n = 72,000). Findings from the Army’s Project A study (McHenry,Hough, Toquam, Hanson, & Ashworth, 1990) were similar. Ree, Earles, and Teachout (1994) alsoinvestigated prediction of other criteria—hands-on-performance tests, an interview work sample test, and awalk-through performance test—and found uncorrected correlations from r = 0.10–0.34 on jobs ranging from

264

personnel specialist to radio operator, mechanic, and air traffic controller. Gottfredson (2002) summarizesdata suggesting that the predictiveness of test scores increases as job complexity increases (e.g., from packer toattorney). She, along with Lubinski and Humphreys (1997), Jensen (1998), O’Toole and Stankov (1992), alsoidentified other real-world variables correlated with cognitive test scores, such as health, crime, divorce rate,illegitimate births, accidents, and longevity.

Differential Validity

From this summary of predictive validity studies it should be noted that Gf and Gc are often treatedinterchangeably as measures of general cognitive ability, the Gf–Gc compound (Corno et al., 2002). Testssuch as the SAT and ASVAB largely measure Gc (Roberts et al., 2000), but are sometimes described asmeasures of intelligence; this is true of other measures reviewed here as well. An ASVAB review panelrecommended including specific Gf measures in future versions (Drasgow, Embretson, Kyllonen, & Schmitt,2006). But the finding also raises the broader question, what is the evidence that other cognitive abilitymeasures besides Gf and Gc predict life outcomes? An issue of the 1986 volume of the Journal of VocationalBehavior was devoted to “the g factor in employment.” Humphreys (1986) summarized his experiences in theAir Force on evaluating evidence for differential validity. He wrote that he and his coworkers were influencedby Thurstone’s primary mental abilities scheme, and that they attempted to create factor-pure tests, but hewas “quickly disillusioned”

as we accumulated predictive validities and factor analyses of dozens of tests and thousands of examinees. We were able to definedependably many group factors, but found little differential validity for the multiple factors. (Differential validity is defined as stabledifferential regression weights for the prediction of multiple criteria.) (p. 421)

This characterizes much of the state of affairs with the consensus model: It is possible to find replicablestructures of test intercorrelations through factor analysis. However, consensus model factors other than g, Gf,and Gc do not appear to have differential validity. The Brunswick symmetry hypothesis (Wittmann & Suess,1999) explains this as a shortage of studies that have examined criterion outcomes matching factors other thang, Gf, and Gc. And there is still evidence for differential validity even for the ASVAB (Alley, 1994; Zeidner& Johnson, 1994).

However, there are at least two consensus model factors other than g, Gf, and Gc, for which there isparticularly strong evidence for differential validity. One is spatial ability (broad visual perception, Gv, inCarroll’s (1993) scheme). Lubinski and colleagues (Lubinski, 2010; Wai, Lubinski, and Benbow, 2009;Webb, Lubinski, & Benbow, 2007) summarized results from several studies suggesting that spatial ability hadincremental validity over other measures (e.g., SAT scores) in identifying young students who went on tomajor in a science, technology, engineering, and mathematics (STEM) discipline, take a job in a STEM field,majored and ended up in occupations in visual arts, and had publications and patents in technical areas (Kell,Lubinski, Benbow, & Steiger, 2013). Snow (1999) lamented the fact that spatial ability has long beenunderappreciated as adding value to traditional assessments.1

Another construct showing differential validity is creativity (idea production or broad retrieval ability, Gr, inCarroll’s scheme). Several studies (Bennett & Rock, 1995; Frederiksen & Ward, 1975) have found that ideaproduction measures, such as ones obtained from tests of formulating hypotheses, and measuring constructs,predicted graduate school outcomes beyond the prediction by GRE verbal and mathematics scores. Similarly,

265

another idea production task, consequences, has been shown to predict leadership abilities and Army officercareer outcomes controlling for cognitive ability test scores (Mumford, Marks, Connelly, Zaccaro, & Johnson,1998).

Why do Cognitive Tests Predict Outcomes?

The evidence is fairly clear that abilities measured by cognitive tests predict important real-world outcomes,and that different abilities—g, Gf, Gc, Gv, Gr—differentially predict outcomes. This is, and has been for thepast 100 years, of enormous practical significance, which explains why schools and employers continue to useability tests for making admissions, placement, selection, and assignment decisions. However, the causes forwhy ability tests predict outcomes are not established. The fundamental problem is that predictive studies arebased on correlations. Abilities cannot be randomly assigned to individuals at various dosage levels; we canexplore their effects in a randomized trial. Instead we have to rely on correlation patterns and occasionalnatural experiments to investigate the causal relationships between abilities and outcomes.

Why does this make a difference? The issue of nature vs. nur-ture has been at the core of the abilitiesdebate since its beginnings (e.g., Harris, 1998). Group differences (e.g., Herrnstein & Murray, 1994; Murray,1998), the predictive power of personality and interests vs. abilities (Ackerman & Beier, 2003), personalityand socioeconomic status vs. abilities (Lubinski, 2009; Roberts, Kuncel, Shiner, Caspi, & Goldberg, 2007;Sackett, Kuncel, Arneson, Cooper, & Waters, 2009), and the relative importance of abilities vs. educationalattainment (e.g., Bowles, Gintis, & Osborne, 2001) have competed as explanations for student achievement,workplace, and life outcomes, and have sparked debates about the best societal investments.

For example, although it has long been thought that cognitive abilities are the most important determinantsof educational attainment and labor market outcomes (e.g., Herrnstein & Murray, 1994; Jensen, 1998), thereis good evidence that non-cognitive skills (such as persistence) may be as important and perhaps even moreimportant in some cases. One source of evidence for this is that cognitive skills, as measured by test scores,only account for a small fraction of the effect of educational attainment on labor market outcomes (Bowles,Gintis, & Osborne, 2001). This leaves the possibility that something else, its effect on the development ofnon-cognitive skills, is what is responsible for schooling’s effect on wages and employment and staying out oftrouble (Heckman, Stixrud, & Urzua, 2006; Levin, 2012). A longitudinal study (Lindqvist & Vestman, 2011)evaluated over 14,000 Swedish enlistees aged 18 or 19 who were given two assessments during conscription: acomprehensive abilities battery (synonyms, induction, metal folding, and technical comprehension), and a 25-minute interview by a trained clinical psychologist intended to evaluate an applicant’s responsibility,independence, outgoingness, persistence, emotional stability, initiative, and ability to adjust to life in thearmed forces. Twenty years later, cognitive skill was found to be a stronger predictor of educational attainmentthan non-cognitive skill, suggesting the importance of education in transforming cognitive ability to labormarket outcomes. However, non-cognitive skills compared to cognitive skills were found to be strongerpredictors of employment and stronger predictors of earnings, particularly for those at the low end of theearnings distribution (see also Chapters 7, 8, and 11, this volume). Cognitive skills were found to be a moreimportant predictor of wages for those earning above the fiftieth percentile. This is a finding consistent withGottfredson’s (1997) finding that cognitive tests were more predictive of success at higher occupational levels.

266

267

Country-Level Validity

Cognitive ability tests can be evaluated at the national as well as individual level. Several researchers inpsychology and economics have compared nations on their average cognitive ability test scores, based on eitherintelligence quotient (IQ) or achievement tests. Jones (2011) argues that cognitive skills have only moderaterelations to earnings, but are strongly correlated with national outcomes, and suggests several reasons why thatmight be so. National cognitive ability predicts gross domestic product (GDP) (Jones & Schneider, 2006) andGDP growth over the past three to five decades; cognitive ability inequality within a country also predictsearnings inequality within that country (Hanushek & Woessmann, 2010). Educational attainment predictsGDP growth, but adding cognitive ability substantially increases the prediction. Disentangling the effects ofschooling compared to the effects of cognitive abilities is difficult because schooling increases cognitive ability,but cognitive ability also increases educational attainment (Heckman et al., 2006). There are some differencesin whether cognitive ability is measured by achievement or IQ scores; achievement scores provide betterpredictions (Hunt & Wittmann, 2008).

National cognitive ability predicts outcomes besides earnings and economic growth. Rindermann (2007)showed high correlations (between r = 0.30 and 0.70) between national cognitive ability and rule of law,quality of bureaucracy, economic freedom, and rate of solved homicides, and negative correlations (r = –0.22to –0.73) with fertility rate, economic inequality, HIV infection rate, government spending (per GDP),homicide rate, and war. Whetzel and McDaniel (2006) showed a high correlation between national cognitiveability and national well-being (also replicated at the state level; Pesta, McDaniel, & Bertsch, 2010).

Hunt (2012) argues that national differences in cognitive ability exist, whether measured by achievement orIQ tests. Whether these national (as well as individual) differences are innate or due to the environment hasbeen a long-standing controversy (e.g., Ceci & Williams, 2009). Hunt argues that national differences arelargely due to environmental factors, such as the amount of schooling (early education and educationalattainment), studying practices, the home environment, and attitudes towards learning and education. Clearlythere is substantial evidence that environmental factors are important and therefore it is important for policymakers to acknowledge national cognitive ability differences and develop policies to address them.

Research on country-level validity has been conducted using both IQ and achievement tests. However, testshave been treated interchangeably, due to their high correlation, and other than the Hunt and Wittmann(2008) study, there has been little, if any, research on differential validity of human cognitive abilities at thenational level.

268

What is an Ability?

Spearman suggested that g could be characterized as mental energy, but this was metaphorical, as there was noindependent evidence for the claim. Another metaphorical attempt was Thomson’s bonds theory(Bartholomew, Deary, & Lawn, 2009), which suggested that a general factor did not imply a single cognitivecapability, but could just as easily arise from a situation in which any two tasks required a common set ofindependent bonds (e.g., stimulus–response associations), different tasks required different overlapping bonds,and that no subset of bonds was common to all tasks. But again, this was primarily metaphorical, as noindependent evidence outside of the correlation matrix itself was brought forward as evidence.

The Cognitive Revolution

In the 1970s, with the growth of cognitive psychology, the situation changed. Numerous researchers began toidentify the underpinnings of cognitive ability test behavior. The “cognitive correlates” method involvedidentifying basic information-processing tasks that correlated with cognitive abilities. The idea was to usebasic information-processing tasks to help shed light on the processing requirements of abilities tests (e.g.,Hunt, Lunneborg, & Lewis, 1975). The “cognitive components” method used subtraction to identify thebasic components of reasoning, spatial ability, or other cognitive abilities (e.g., Sternberg, 1977). Processingstages were isolated by comparing performance on a regular test item with simplified versions of that item.There were many variants on these methods and considerable enthusiasm about their potential for revealingsomething deep about the nature of abilities. For example, Ackerman (1987) integrated traditional abilitiesmodels with skill development models from the human factors literature.

Working Memory and the Cognitive Abilities Measurement Framework

A successful example of the use of a cognitive correlates approach was a study by Kyllonen and Christal(1990), which administered the ASVAB and dozens of cognitive abilities tests to a couple thousand militaryenlistees along with dozens of measures of working-memory capacity prepared from Baddeley and Hitch’s(1974) definition of tasks requiring simultaneous storage and processing of incoming information. The twoconstructs were highly correlated (the latent variable correlation was estimated at about 0.90). Since then anumber of studies have attempted to determine which abilities and working-memory components areresponsible for the relationship. A recent paper by Conway and Kovacs (2013) reviewed neuroimaging andbehavioral research, suggesting that it is the executive/attentional component of working memory and the Gfcomponent of ability that is responsible for the high correlation between g and working memory.

A larger framework for thinking about the information-processing underpinnings of cognitive abilities wasthe cognitive abilities measurement framework (Kyllonen, 1994, 1995), based on a consensus information-processing model. The framework posited that the major information-processing components for cognitiveability were working memory, long-term declarative and procedural knowledge stores, and a parametergoverning the speed of information flow through the system. Each of these components could serve as asource of individual differences—the capacity of working memory, the speed of processing, the breadth of thedeclarative and procedural stores, and the efficiency of declarative and procedural learning. A test battery wasdeveloped from this framework (the Advanced Personnel Testing battery, APT), and validated against

269

training grades (Sawin, Earles, Goff, & Chaiken, 2001). Although there were some suggestions forincremental validity of the APT over the ASVAB, the more important aspect of the effort was its heuristicvalue in showing the commonality between information-processing psychology and traditional psychometrics.The modern view (e.g., Underwood, 1975) is that the two fields are not really distinguishable, and that thepsychometric methodology serving as the basis for differential psychology, the discipline for identifyingabilities, can be used also for identifying skills in information-processing psychology. For example, Miyake etal. (2000) used confirmatory factor analysis to explore the structure of executive functions (shifting attention,updating, and inhibition) in working memory.

Neuroscience

In recent years, much of the energy in trying to define abilities seems to have shifted towards neuroscience(Gray & Thompson, 2004; see also Chapter 5, this volume). Several reviews summarize currentunderstanding of the relationship between working memory, executive functioning, and neurologicalcorrelates (Conway, Moore, & Kane, 2009; Gray, Chabris, & Braver, 2003; Osaka, Logie, & D’Esposito,2007). Although the field is plagued by small sample sizes due to high costs for data collection, as well asconstraints on the nature of tasks due to the susceptibility to extraneous effects in brain imaging, datacollection and analysis technology is improving rapidly and one can imagine a comprehensive brain-mappingstudy soon that identifies neurological correlates of all the abilities in the CHC consensus framework.

270

Development of Abilities

Cognitive abilities increase with age at least through late adolescence and early adulthood, and even lateradulthood in some cases. Thus, the concept of the IQ was invented to enable comparisons between peopleconditioned on age. A person with a high IQ is a person who outperforms others his or her age on cognitiveability tests. Test scores (e.g., SAT scores) required to qualify in talent searches (Webb et al., 2007) are notremarkable but for the fact that they are achieved at an early age (e.g., at age 12 rather than at age 17). All thisdemonstrates the obvious, that abilities develop with age. And, again, one of the arguments for the fluid vs.crystallized abilities distinction is that fluid abilities (including perceptual speed, inductive reasoning, spatialability) peak relatively early (mid twenties) and decline in early adulthood, whereas crystallized abilities(including vocabulary, verbal ability, verbal memory) rise through one’s thirties and perhaps even later, remainstable, and only decline much later in life (McArdle, Hamagami, Meredith, & Bradway, 2000). These resultsvary somewhat depending on whether studies are conducted using a dataset that is cross-sectional (peak tendsto be earlier, perhaps due to general trends in growth of IQ scores over time) or longitudinal (peak tends to belater).

Desjardins and Warnke (2012) summarized the literature on cognitive skills growth, identifying a variety ofeffects (e.g., cohort, period, genetic, social, skills practice, physical and mental activity, education, andbackground effects), and distinguishing different types of cognitive skills. A key finding is that “education,training, and a number of physical, social and mental activities have all been implicated as possible factorswhich help to mitigate the age-related decline in cognitive skills [which] suggests that policy can make adifference” (p. 55). The authors point out the important role cross-national cognitive surveys, such as theProgram for International Assessment of Adult Competencies, can play in disentangling the variousinfluences on cognitive skill growth and decline.

The number of years a person spends studying in school is correlated with cognitive ability levels. Ceci andWilliams (1997) identified seven types of evidence suggesting the relationship is not just correlational but thatschooling raised cognitive ability. These were: (a) a negative correlation between IQ and age for children whoattended school intermittently during their school-age years; (b) lower IQ and achievement scores for childrenwho began school late; (c) high scores for those staying in school longer to avoid the draft in the United Statesduring the Viet Nam war; (d) lower scores for those dropping out of school before graduation; (e) declines inscores due to summer vacation; (f) higher IQs for those with birthdays early in the year due to longermandatory school attendance; and (g) children of the same age who receive different levels of schooling due toage entry requirements have scores that are most reflective of their years in school than of their chronologicalage. Most of these sources of evidence are essentially natural experiments that enable a causal interpretation ofthe effects of schooling on IQ.

Cliffordson and Gustafsson (2008) applied a method called the continuous age/continuous treatment (CCmethod, similar to regression discontinuity) to a dataset of 48,269 male military enlistees’ test scoresmeasuring Gf, Gc, and Gv abilities. They addressed whether schooling per se affected test scores beyondnatural developmental growth and concluded that it did, and at the rate of 2.7 IQ points per year, consistentin size with other estimates. They also noted differential effects on growth due to educational tracks and evenwith curricular emphases within tracks; for example, the highest growth was associated with the technology

271

track. They also found that for Gf ability specifically, schooling was associated with 4.2–4.8 IQ points peryear.

A comparable estimate of 3.7 IQ points per year of school was suggested by a natural experiment inNorway, which changed compulsory schooling from seventh grade to ninth grade in the 1960s (Brinch &Galloway, 2011). Different communities in Norway adopted the change at different times (during a shortwindow), making it possible to evaluate the effects of the additional mandatory schooling. All citizens weregiven a comprehensive abilities test at age 19 due to compulsory military service, which enabled the estimationof the effects of additional schooling on cognitive abilities.

In addition to the growth in cognitive abilities due to schooling, Flynn (1987) documented massive gains inAmerican IQ scores from 1932 to 1978, followed by the observation of similar gains in 14 nations and laterstill more. The American Psychological Association commissioned a task force resulting in a book (Neisser,1998) exploring the phenomenon (e.g., growth primarily in Gf, not in Gc; growth more at the lower thanhigher ends of the distribution), its possible causes (e.g., home factors, family size, nutrition, increasedschooling, test familiarity) and its implications (e.g., narrowing of the achievement gap between subgroups).The Brinch and Galloway (2011) study suggested that increased schooling might have accounted for aboutone-third of the Flynn effect in Norway. It also appears to be the case that the Flynn effect may be slowing toa halt in developed nations (Teasdale & Owen, 2008), even though it may be continuing in developingnations (Flynn & Rossi-Casé, 2012), suggesting that national cognitive ability differences could be shrinking.

Finally, there is evidence that, like schooling, work experience contributes to cognitive skills development.The International Adult Literacy Survey (OECD & Statistics Canada, 2000) from 22 countries (16languages) showed that cognitive skill levels (quantitative, verbal, and document literacy) were correlated withlabor force participation, particularly with more high-status jobs (i.e., ones requiring greater education levels),how skills were used on the job, participation in education and workforce training, and the demand forcognitive skills at home.

272

Trainability of Abilities

Studies of identical and fraternal twins reared apart (e.g., Plomin, Pedersen, Lichtenstein, & McClearn,1994), as well as long-term test–retest studies (Deary et al., 2013), have led to a commonplace view thatcognitive abilities are heritable, fixed, unaffected by schooling and the environment, and rank-order stableacross time. While there is some evidence for those positions, the review above suggests that schooling,environmental effects, and a secular trend all have improved cognitive abilities, sometimes dramatically. Inaddition studies have specifically targeted cognitive abilities improvement. These range from nutritionalinterventions, early-childhood intervention programs, programs taught in schools designed to teach generalcognitive skills, to recent programs designed to improve working-memory capacity.

Early Childhood Studies

Protzko, Aronson, and Blair (2013) maintain the Database of Raising Intelligence, a continuously updatedcatalog of randomized trials designed to increase cognitive abilities. Meta-analyses from this database haveshown that several interventions do appear to boost intelligence of young children: dietary supplements topregnant mothers or infant formula supplements (0.24 effect size, about 3–4 IQ points), early-childhoodintervention programs for low-socioeconomic status children (0.45 effect size; higher than the 0.23 given byCamilli, Vargas, Ryan, & Barnett, 2010), interactive reading with children (0.40 effect size), and sendingchildren to preschool (0.51 effect size). Two programs have been studied in great detail: the Highscope PerryPreschool program (Schweinhart, Barnes, & Weikart, 1993) and the Abecederian program (Campbell et al.,2008), with the finding that the programs did boost IQ, although the effects faded somewhat over time (forthe Abecedarian program, benefits were retained to age 40). Barnett (2011) concluded that these two smallprograms were among the most successful and their features ought to be compared to others to determinewhy. Regardless, it is important to note that there are additional benefits for these programs besides boostingIQ, such as higher educational attainment, decreased likelihood of being unemployed, on welfare, or havingcommitted a crime, and greater earnings. These consequences have suggested that the rate of return on theseprograms is approximately 7–10% (Heckman, 2006; Heckman, Moon, Pinto, Savelyev, & Yavitz, 2010).

Interventions with Schoolchildren and Adults

During the 1980s there were a number of programs developed and tried out in schools designed to improvegeneral cognitive abilities, rather than specific curricular skills (Chipman, Segal, & Glaser, 1985; Nickerson,Perkins, & Smith, 1985; Segal, Chipman, & Glaser, 1985). Perhaps the expectations were too high, or theimplementations were unsound, but these programs and this approach of teaching general cognitive abilitieswere not deemed wildly successful, and so attention moved to other topics. One program that was successful isVenezuela’s Project Intelligence. This was a 3-year national intervention to improve the national cognitiveability level of the population. The first 2 years were essentially dress rehearsals, but in the final year, arandomized trial (matching classes) of six schools, 24 classes, and 30–40 seventh-graders per class were givennon-curricular thinking skills lessons and practice for 4 days per week. Lessons covered classification,deductive and inductive reasoning, critical language use, hypothesis generation, problem solving, and decisionmaking (Herrnstein, Nickerson, de Sanchez, & Swets, 1986). A battery of multiple-choice tests (e.g,. Otis-

273

Lennon School Ability Test, Cattell Culture Fair Test, General Ability Tests; plus a 3.5-hour Target AbilityTest designed to reflect the processes but not content taught) was administered by ministry officials aspretests, interim tests, and posttests. In addition, some far transfer tests, including a design task, oral andopen-ended tasks, and an oral reasoning task, were administered. For all tasks there was a significanttreatment effect, effect sizes ranging from 0.10 to 0.75, with suggestions that the benefits were in line withlesson emphasis. Unfortunately, no follow-up, short- or long-term, was attempted and so we know little aboutthe long-term benefits of the program.

Working-memory Training

It seems reasonable that an intense year-long cognitive training treatment such as was given in ProjectIntelligence would result in improvements in performance on cognitive tasks, even tasks that were not thatsimilar to the ones that were the focus of training. But a rather surprising result came from a study showingthat even a relatively short cognitive training program, this one training working-memory skills, resulted inimprovements in one’s ability to perform Gf reasoning tasks (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008).The intervention, given to 70 young male and female adults for 8–19 days (with a control group) was a dual“n-back” task requiring participants to monitor auditory and visual targets presented sequentially, 3 secondsper target, and to indicate whether the target was in the same position or was the same consonant sound asthe target from n (= 3 or 4) trials previous. Before the first training session, and after the last one, participantswere administered Gf tasks, Raven’s Progressive Matrices (RPM), and a more difficult variant of the RPM,and two working-memory span tests. The findings were that both treatment and control groups showedimprovement in Gf from pretest to posttest (for the control group, a retest effect), but the treatment groupshowed significantly more (effect size of 0.65 vs. 0.25); the effect size was correlated with training dosage, andwas not solely due to an improvement in working-memory span. The authors attributed the effect to practicein controlling attention. The study has received considerable attention (e.g., Morrison & Chein, 2011) but thedust has not yet settled on how reliable the phenomenon is and there is a concern that this might be a short-term, task-specific learning effect that does not generalize (Melby-Lervåg & Hulme, 2013; Shipstead, Redick,& Engle, 2012). Nevertheless the study—and others like it—is one more indicator that cognitive abilitiesmight not be as immutable as once thought, and that they may be amenable to direct instruction.

Expertise

Since the 1960s, studies have shown that specific cognitive abilities, ranging from playing chess, memorizingcircuit diagrams, taxi driving, interpreting radiology images, to memory span itself, are highly amenable totraining. And that expertise is not so much a matter of heritable, immutable abilities as it is a matter of sheerpractice. For example, Ericsson, Chase, and Faloon (1980) were able to improve the memory span score ofone student from a normal 7 to an unbelievable 79 (with digits read at one per second), as a result of 230hours of practice (interestingly, after that training, the student’s letter span remained at 6). This has led to the“10 years, 10,000 hours” mantra of expertise, popularized by Gladwell (2009) based on expertise research(Ericsson, Charness, Feltovich, & Hoffman, 2006). The expertise movement can be understood as a stronglyenvironmental position on the nature of skills development. The field provides many demonstrationssuggesting that any cognitive skill or ability is improvable through lots of practice, particularly deliberate

274

practice—structured, mindful, and purposeful.Ackerman (1987, 2007) has shown that expertise growth trajectories are moderated by cognitive abilities,

particularly general cognitive ability in the early stages of skill development and knowledge acquisition.Macnamara, Hambrick, and Oswald (2014) conducted a meta-analysis showing that hours of deliberatepractice accounted for only 12% of variance in performance overall (correlation of 0.35), but 26% of thevariance in game performance, 21% for music, 18% for sports, and less than 5% for education and professions,suggesting that expertise is not accounted for by deliberate practice alone and might be explained by otherfactors, such as the age at which a person starts practice, and cognitive abilities.

275

Other Influences on Test Scores

There are many influences on test scores besides abilities. These construct-irrelevant factors can affectinferences drawn from test score distributions, such as age and subgroup differences. For example, if asubgroup were less motivated on average than another subgroup, or a subgroup’s first language were alanguage other than the test language, then it would be inappropriate to draw conclusions about groupdifferences based on test scores. In general there are many possible other influences (e.g., disabilities, fatigue,stereotype threat, beliefs about the nature of intelligence, time of day, stressors, room temperature, drugs andalcohol, life events). Two of the most studied influences are motivation and the speed–accuracy tradeoff.

Motivation

Several recent studies suggest that motivation can play a large role in test performance, particularly when thetest is perceived by the test taker to be low stakes. Researchers often examine item response patterns forindications of random responding during data analysis but this is not systematically done, and there are likelymany datasets contaminated with this kind of random responding. If aberrant response patterns are lowfrequency and distributed evenly, then there is not a large problem (other than attenuated validitycoefficients); but if contaminated data are produced more by some groups than others, then this would createproblems in inferences about group differences. Wise and Kong (2005) have suggested examining data forresponse time effort, indicated by short response times, which may indicate low motivation.

The magnitude of the motivation effect is potentially large. Duckworth, Quinn, Lynam, Loeber, andStouthamer-Loeber (2011) conducted a meta-analysis of random-assignment studies that offered materialincentives (e.g., typically small monetary payments) for test performance. They found a large effect size (0.64SD) on test performance and found that motivation incentives were more effective for low scorers, implicatingmotivation as a possible cause for low scores in low-stakes testing. Segal (2012) compared performance on thecoding speed test (a) given as part of a low-stakes survey study (the National Longitudinal Survey of Youth)versus (b) given as part of a high-stakes entrance exam for the armed forces (ASVAB) and found differences(e.g., prediction of future earnings, controlling for cognitive ability) suggesting a motivation component. Liu,Bridgeman, and Adler (2013) conducted a random-assignment incentive study examining motivation effectson a multiple-choice and essay test of critical thinking given to college students. Both the treatment andcontrol groups were given standard written instructions (e.g., “your answers on the tests will only be used forresearch purposes”) but the treatment group was given the additional instruction that “your test scores may bereleased to faculty in your college or to potential employers to evaluate your academic ability.” They found aneffect size of 0.41 for this simple manipulation (see their Table 4, dCP).

Speed–Accuracy Tradeoff

There has been a long-standing recognition in abilities measurement that response time is a dimension of testperformance separate from accuracy. Tests have been categorized as power (e.g., most of the sample completesthe test) or speeded (e.g., most of the sample fails to complete), and the difference in performance has beencalled the speed-level distinction (Lohman, 1989). It is reflected in Carroll’s (1993) and the CHC model astwo separate second stratum factors, one called cognitive speediness (Gs) and the other called

276

decision/reaction time/speed (Gt). Test takers can choose to go slowly and be more accurate or go morequickly and make more errors, a speed–accuracy tradeoff, and only the scoring procedure itself can determinethe optimal tradeoff. Van der Linden (2007) developed a general framework for modeling accuracy andresponse time together in a hierarchical item–response theory model. It can be used to combine speedinesswith accuracy to make a better measure of ability. That is, response speed serves as collateral information tomeasure ability, which turns out to be particularly useful in measuring ability at the extremes of the abilitydistribution (Ranger, 2013).

277

Outside the Realm of Cognitive Constructs

Sternberg (1985) and Gardner (1999) have argued that conventional discussions about human abilitiesoverlook important ones, such as practical intelligence, creativity (Sternberg), and bodily-kinesthetic ability(Gardner). In fact, a human performance taxonomy based on the same principles as the cognitive abilitytaxonomies does include bodily-kinesthetic ability (Fleishman, 1972). Gottfredson (2003) points out thatgeneral cognitive ability relates to many aspects of practical intelligence. And creativity itself is part of theconsensus model (the factor Gr, broad retrieval ability). Thus, the criticisms of Sternberg and Gardner areaddressed in current abilities frameworks. However, there are cognitive abilities that are not included in theconsensus framework because they are mostly uncorrelated with general cognitive ability, and thus truly mightmeasure something different.

Cognitive Biases and Heuristics

The field of cognitive biases and heuristics is based on studies that show that people tend to make systematicerrors in thinking due to mental shortcuts (invoking “System 1 thinking”) rather than thinking things through(“System 2 thinking”) (Kahneman, 2011). Examples include confirmation bias (attending to evidence thatconfirms hypotheses and ignoring disconfirming evidence), fundamental attribution error (attributing others’mistakes to their character, but one’s own to circ*mstance), base rate fallacy (calculating the likelihood of anevent based on available information and ignoring the prior probability), projection fallacy (believing thatothers share one’s own beliefs and attitudes), and the anchoring effect (using a local context to frame a currentchoice rather than taking a big-picture perspective). Stanovich and West (2008) investigated several biases andconcluded that many were uncorrelated with cognitive ability (their Table 8 lists 14 biases that fail to correlatewith cognitive ability and 14 that do correlate). Their explanation is that people will take the mental shortcutsthat lead to errors unless characteristics of the task or reminders tell them not to. Some tasks do not invite themental shortcuts, and others do. For example, “cognitive reflection tasks” (Frederick, 2005), which inviteSystem 1 responding, have been found to be better at predicting susceptibility to cognitive biases than acomposite measure of cognitive ability (Toplak, West, & Stanovich, 2011).

Confidence and Metacognition

A particular kind of cognitive bias (cognitive optimism) is our inability to recognize our lack of intellectualand social skill in domains where we lack skill and failure to recognize others’ skill, unless we are trained forthat skill (Dunning, Johnson, Ehrlinger, & Kruger, 2003). Stankov and colleagues see this as a lack ofcalibration between confidence and performance. They have shown that confidence operates like anindependent ability, correlated moderately with metacognition (Kleitman & Stankov, 2007), separate fromgeneral cognitive ability, but nevertheless incrementally predictive of intellectual tasks, such as tests oflistening comprehension, speaking ability, and numeracy (Stankov & Lee, 2008). This may relate to self-regulation, as discussed in Chapter 8, this volume.

Emotional Intelligence

Since Goleman’s (1995) popular book, the concept of emotional intelligence (EI) has attracted considerable

278

attention in scientific as well as pop culture circles. At its core EI represents cognitive skills that relate toprocessing and reasoning about emotional information (Mayer, Roberts, & Barsade, 2008), specifically,perceiving, using, understanding, and managing emotions (Mayer, Salovey, & Caruso, 2012). A growingconsensus has emerged for two distinct ways to measure EI, through ratings (“trait EI”) and performancemeasures (“ability EI” or “information-processing EI”) (Petrides & Furnham, 2000). Trait EI ratings aresimilar to standard personality tests; EI performance measures include tests such as determining an emotiondenoted by combining two adjectives, or perceiving the emotion expressed in a picture of a face, or abstractart. MacCann, Joseph, Newman, and Roberts (2014) have recently suggested that ability EI deserves standingas a second-stratum CHC factor, Gei, similar in status to Gc or Gv. They administered performance EImeasures along with measures of fluid ability (Gf), crystallized intelligence (Gc), quantitative reasoning (Gq),visual processing (Gv), and broad retrieval ability (Gr) to 700 U.S. college students, and found support for aCHC-like hierarchical model, with EI at the second stratum, loading approximately 0.8 on a third-stratum gfactor. This is one of the larger, more sophisticated studies of EI measures to date, placing them into a largercontext of cognitive abilities measurement.

279

Conclusions and New Directions

Cognitive abilities measurement, traced back to Galton’s observations of normally distributed individualdifferences, Spearman’s observations regarding the sufficiency of a general cognitive abilities factor, andBinet’s construction of an intelligence test battery to identify the academic abilities of schoolchildren, is anapplied and theoretical success story in educational psychology. In applications, ability measurement isperhaps more ubiquitous than ever, with more implications for policy, as indicated by the attention givenlarge-scale domestic and international assessments of mathematical, verbal, and problem-solving literacy. Suchassessments are used to compare states and nations and to draw lessons through comparative analysis toimprove educational systems around the world. Despite the focus on curricular improvements in the UnitedStates resulting from No Child Left Behind legislation, there has nevertheless been a steady thread ofattention given to the development of general, transferable cognitive abilities, for example through early-childhood education efforts. The general–specific pendulum swings back and forth, and a prediction is thatefforts such as working-memory training, and general skills training will make a comeback.

In measurement, technology enables a much richer yet affordable means to present information (testquestions) and collect responses from examinees, which can be seen already in identification of new abilitiessuch as EI, scientific inquiry skills, or technology and engineering literacy skills that benefit from richerstimulus presentations and response possibilities (see also Chapter 3, this volume). In the future we are likelyto see more applications using game-like tests, that present more realistic information in the form of videoand computer-generated imagery graphics, and enabling the recording of eye movements and gesturesthrough systems like Microsoft’s Kinect. We also should expect concomitant increased synergies betweenneuroscience and abilities measurement (see Chapter 5, this volume).

On the construct side, we are likely to see increasing attempts to measure what have sometimes been callednon-cognitive skills—teamwork, work ethic, communication skills, collaborative problem solving, tolerancefor diversity—using performance measures. The EI work (MacCann et al., 2014) represents a step along theselines, but more is coming. PISA 2015 introduced a collaborative problem-solving measure, involvingcollaboration with an agent; future cycles of international assessments are likely to introduce human–humancollaborative problem solving. Efforts like these will continue and undoubtedly expand our ideas on theorganization, development, and use of human cognitive abilities.

280

Acknowledgments

Support for this chapter comes from Educational Testing Service’s Research Center for Academic andWorkforce Readiness and Success (R&D). The author thanks Lyn Corno, David Lubinski, and Jan-EricGustafsson for their helpful comments and suggestions.

281

Note

He wrote: “There is good evidence that [visual-spatial reasoning] relates to specialized achievements in fields such as architecture, dentistry,engineering, and medicine . . . Given this plus the longstanding anecdotal evidence on the role of visualization in scientific discovery, . . . itis incredible that there has been so little programmatic research on admissions testing in this domain” (p. 136). (The author thanks DavidLubinski for this find.)

282

References

Ackerman, P. L. (1987). Individual differences in skill learning: An integration of psychometric and information processing perspectives.Psychological Bulletin, 102, 3–27.

Ackerman, P. L. (2007). New developments in understanding skilled performance. Current Directions in Psychological Science, 16, 235–239.Ackerman, P. L., & Beier, M. E. (2003). Intelligence, personality, and interests in the career choice process. Journal of Career Assessment, 11,

205–218.Alley, W. E. (1994). Recent advances in classification theory and practice. In M. G. Rumsey, C. B. Valker, and J. H. Harris (Eds.), Personnel

selection and classification (pp. 431–442). Hillsdale, NJ: Lawrence Erlbaum Associates.American Educational Research Association, American Psychological Association, National Council on Measurement in Education (2014).

The standards for educational and psychological testing. Washington, DC: AERA Books and Publications.Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research

and theory (Vol. 8, pp. 47–89). New York: Academic Press.Barnett, W. S. (2011). Effectiveness of early educational intervention. Science, 333, 975–978.Bartholomew, D. J., Deary, I. J., & Lawn, M. (2009). A new lease of life for Thomson’s bonds model of intelligence. Psychological Review, 116,

567–579.Bennett, R. E., & Rock, D. A. (1995). Generalizability, validity, and examinee perceptions of a computer-delivered formulating hypotheses

test. Journal of Educational Measurement, 32, 19–36.Binet, A. (1905/1916). New methods for the diagnosis of the intellectual level of subnormals. In E. S. Kite (Trans.), The development of

intelligence in children. Vineland, NJ: Publications of the Training School at Vineland. (Originally published 1905 in L’Année Psychologique,12, 191–244.)

Boring, E. G. (1923). Intelligence as the tests test it. New Republic, 36, 35–37.Bowles, S., Gintis, H., & Osborne, M. (2001). The determinants of individual earnings: Skills, preferences, and schooling. Journal of Economic

Literature, 39, 1137–1176.Brinch, C. N. & Galloway, T. A. (2011). Schooling in adolescence raises IQ scores. Proceedings of the National Academy of Sciences (PNAS), 109,

425–430.Camilli, G., Vargas, S., Ryan, S., & Barnett, W. S. (2010). Meta-analysis of the effects of early education interventions on cognitive and social

development. Teachers College Record, 112, 579–620.Campbell, F. A., Wasik, B. H., Pungello, E., Burchinal, M., Barbarin, O., Kainz, K., Sparling, J. J., & Ramey, C. T. (2008). Young adult

outcomes of the Abecedarian and CARE early childhood educational interventions. Early Childhood Research Quarterly, 23, 452–466.Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.Carroll, J. B. (2003). The higher-stratum structure of cognitive abilities: Current evidence supports g and about ten broad factors. In H. Nyborg

(Ed.), The scientific study of general intelligence: Tribute to Arthur R. Jensen (pp. 5–21). Boston, MA: Pergamon Press.Cattell, R. B. (1941). Some theoretical issues in adult intelligence testing. Psychological Bulletin, 38, 592.Ceci, S. J., & Williams, W. M. (1997). Schooling, intelligence, and income. American Psychologist, 52, 1051–1058.Ceci, S. J., & Williams, W. M. (2009). Should scientists study race and IQ? The scientific truth must be pursued. Nature, 437, 788–789.Chipman, S. F., Segal, J. W., & Glaser, R. (Eds.) (1985). Thinking and learning skills. Vol. 2: Research and open questions. Hillsdale, NJ:

Lawrence Erlbaum Associates.Cliffordson, C., & Gustafsson, J.-E. (2008). Effects of age and schooling on intellectual performance: Estimates obtained from analysis of

continuous variation in age and length of schooling. Intelligence, 36, 143–152.Conway, A. R., & Kovacs, K. (2013). Individual differences in intelligence and working memory: A review of latent variable models. Psychology

of Learning and Motivation, 58, 233–270.Conway, A. R. A., Moore, A. B., & Kane, M. J. (2009). Recent trends in the cognitive neuroscience of working memory. Cortex, 45, 262–268.Corno, L., Cronbach, L. J., Kupermintz, H., Lohman, D. F., Mandinach, E. B., Porteus, A. W., & Talbert, J. E. (2002). Remaking the concept

of aptitude: Extending the legacy of Richard E. Snow. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.Cronbach, L. J. (1970). Essentials of psychological testing. New York: Harper & Row.Deary, I. J., Pattie, A., & Starr, J. M. (2013). The stability of intelligence from age 11 to age 90 years: The Lothian birth cohort of 1921.

Psychological Science, 24(12), 2361–2368.Deary, I. J., Whalley, L. J., Lemmon, H., Crawford, J. R., & Starr, J. M. (2000). The stability of individual differences in mental ability from

childhood to old age: Follow-up of the 1932 Scottish Mental Survey. Intelligence, 28(1), 49–55.Desjardins, R., & Warnke, A. (2012). Ageing and skills: A review and analysis of skill gain and skill loss over the lifespan and over time. OECD

Education Working Papers, No. 72. Paris: OECD Publishing.

283

Drasgow, F., Embretson, S., Kyllonen, P., & Schmitt, N. (2006). Technical review of the Armed Forces Vocational Aptitude Battery. Alexandria,VA: Human Resources Research Organization.

Duckworth, A. L., Quinn, P. D., Lynam, D. R., Loeber, R., & Stouthamer-Loeber, M. (2011). Role of test motivation in intelligence testing.Proceedings of the National Academy of Sciences of the United States of America, 108, 7716–7720.

Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions inPsychological Science, 12(3), 83–87.

Dweck, C. S., & Master, A. (2009). Self theories and motivation. In K. Wentzel, & A. Wigfield (Eds.), Handbook of motivation at school (pp.123–140). New York: Routledge.

Ericsson, K. A., Charness, N., Feltovich, P., & Hoffman, R. R. (2006). Cambridge handbook on expertise and expert performance. New York:Cambridge University Press.

Ericsson, K. A., Chase, W. G., & Faloon, S. (1980). Acquisition of a memory skill. Science, 208(4448), 1181–1182.Ferriman-Robertson, K., Smeets, S., Lubinski, D., & Benbow, C. P. (2010). Beyond the threshold hypothesis: Even among the gifted and top

math/science graduate students, cognitive abilities, vocational interests, and lifestyle preferences matter for career choice, performance, andpersistence. Current Directions in Psychological Science, 19, 346–351.

Flanagan, D. P., & Harrison, P. L. (2012). Contemporary intellectual assessment: Theories, tests, and issues (3rd ed.). New York: The GuilfordPress.

Fleishman, E. A. (1972). On the relation between abilities, learning, and human performance. American Psychologist, 27, 1017–1032.Flynn, J. R. (1987). Massive IQ gains in 14 nations: What IQ tests really measure. Psychological bulletin, 101(2), 171–191.Flynn, J. R. (2007). What is intelligence? Beyond the Flynn effect. Cambridge, MA: Cambridge University Press.Flynn, J. R., & Rossi-Casé, L. (2012). IQ gains in Argentina between 1964 and 1998. Intelligence, 40, 145–150.Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.Frederiksen, N., & Ward, W. C. (1975). Development of measures for the study of creativity. GRE Board Professional Report GREB No. 72-2P;

also ETS RB No. 75-18. Princeton, NJ: Educational Testing Service.Ganley, C. M., Mingle, L. A., Ryan, A. M., Ryan, K., Vasilyeva, M., & Perry, M. (2013). An examination of stereotype threat effects on girls’

mathematics performance. Developmental Psychology, 49(10), 1886–1897.Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic Books.Gardner, H. (1999). Who owns intelligence? The Atlantic Monthly, 283, 67–76.Gladwell, M. (2009). Outliers: The story of success. New York: Little, Brown.Goleman, D. (1995). Emotional intelligence: why it can matter more than IQ. London: Bloomsbury.Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24(1), 79–132.Gottfredson, L. S. (2002). Where and why g matters: Not a mystery. Human Performance, 15(1–2), 25–46.Gottfredson, L. (2003). Dissecting practical intelligence theory: Its claims and its evidence. Intelligence, 31, 343–397.Gray, J. R., Chabris, C. F., & Braver, T. S. (2003). Neural mechanisms of general fluid intelligence. Nature Neuroscience, 6(3), 316–322.Gray, J. R., & Thompson, P. M. (2004). Neurobiology of intelligence: Science and ethics. Nature Reviews, 5, 471–482.Guilford, J. P. (1956). The structure of intellect. Psychological Bulletin, 53, 267–293.Gustafsson, J.-E. (1984). A unifying model for the structure of intellectual abilities. Intelligence, 8, 179−203.Gustafsson, J. E., & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28,

407–434.Hakel, M. D. (Ed.) (1998). Beyond multiple choice. Mahwah, NJ: Lawrence Erlbaum Associates.Hanushek, E. A., & Woessmann, L. (2010). The economics of international differences in educational achievement. Working Paper 15949.

Cambridge, MA: National Bureau of Economic Research.Harris, J. R. (1998). The nurture assumption: Why children turn out the way they do. New York: Free Press.He, J. (2013). Text mining and IRT for psychiatric and psychological assessment. PhD thesis, University of Twente, Enschede, the Netherlands.Heckman, J. J. (2006). Skill formation and the economics of investing in disadvantaged children. Science, 312, 1900–1902.Heckman, J. J., Moon, S. H., Pinto, R., Savelyev, P. A., & Yavitz, A. (2010). The rate of return to the HighScope Perry preschool program.

Journal of Public Economics, 94, 114–128.Heckman, J. J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social

behavior. Journal of Labor Economics, 24, 411–482.Herrnstein, R. J., & Murray, C. (1994). The bell curve: Intelligence and class structure in American life. New York: Free Press.Herrnstein, R. J., Nickerson, R. S., de Sanchez, M., & Swets, J. A. (1986). Teaching thinking skills. American Psychologist, 41, 1279–1289.Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2, 41–54.Horn, J. L. (1989). Models for intelligence. In R. Linn (Ed.), Intelligence: Measurement, theory, and public policy (pp. 29–73). Urbana, IL:

284

University of Illinois Press.Horn, J. L., & Cattell, R. B. (1966). Refinement of the theory of fluid and crystallized intelligences. Journal of Educational Psychology, 57, 253–

270.Horn, J. L., & Cattell, R. B. (1967). Age differences in fluid and crystallized intelligence. Acta Psychologica, 26, 107–129.Humphreys, L. G. (1986). An analysis and evaluation of test and item bias in the prediction context. Journal of Applied Psychology, 71(2), 327–

333.Hunt, E. (2011). Human intelligence. New York: Cambridge University Press.Hunt, E. (2012). What makes nations intelligent? Perspectives on Psychological Science, 7, 284–306.Hunt, E. B., Lunneborg, C., & Lewis, J. (1975). What does it mean to be high verbal? Cognitive Psychology, 7, 194–227.Hunt, E., & Wittmann, W. (2008). National intelligence and national prosperity. Intelligence, 36, 1–9.Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Perrig, W. J. (2008). Improving fluid intelligence with training on working memory. Proceedings of

the National Academy of Sciences, 105(19), 6829–6833.Jensen, A. R. (1998). The g factor: The science of mental ability (Human evolution, behavior, and intelligence). Westport, CT: Praeger.Johnson, W., & Bouchard, T. J., Jr. (2005). The structure of human intelligence: It is verbal, perceptual, and image rotation (VPR), not fluid

and crystallized. Intelligence, 33, 393–416.Jones, G. (2011). National IQ and national productivity: The hive mind across Asia. Asian Developmental Review, 28, 51–71.Jones, G., & Schneider, W. J. (2006). Intelligence, human capital, and economic growth: A Bayesian averageing of classical estimates (BACE)

approach. Journal of Economic Growth, 11, 71–93.Kahneman, D. (2011). Thinking fast and slow. New York: Farrar, Straus, & Giroux.Kane, M. J., & Engle, R. W. (2002). The role of prefrontal cortex in working memory capacity, executive attention, and general fluid

intelligence: An individual differences perspective. Psychonomic Bulletin & Review, 9, 637–671.Kaufman, A., & Kaufman, N. (2004). Manual of the Kaufman Assessment Battery for Children, 2nd ed. San Antonio, TX: Pearson Assessment.Kell, H. J., Lubinski, D., Benbow, C. P., & Steiger, J. H. (2013). Creativity and technical innovation: Spatial ability’s unique role. Psychological

Science, 24, 1831–1836.Kleitman, S., & Stankov, L. (2007). Self-confidence and metacognitive processes. Learning and Individual Differences, 17, 161–173.Kuncel, N. R., & Hezlett, S. A. (2007). Standardized tests predict graduate students’ success. Science, 315, 1080–1081.Kyllonen, P. C. (1994). Cognitive abilities testing: An agenda for the 1990s. In M. G. Rumsey, C. B. Walker, & J. H. Harris (Eds.), Personnel

selection and classification (pp. 103–129). Hillsdale, NJ: Lawrence Erlbaum Associates.Kyllonen, P. C. (1995). CAM: A theoretical framework for cognitive abilities measurement. In D. K. Detterman (Ed.), Current topics in human

intelligence (Vol. 4, pp. 307–359). New York: Springer-Verlag.Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is (little more than) working-memory capacity?! Intelligence, 14(4), 389–433.Levin, H. M. (2012). The utility and need for incorporating noncognitive skills into large-scale educational assessments. In M. von Davier, E.

Gonzalez, I. Kirsch, & K. Yamamoto (Eds.), The role of international large-scale assessments: Perspectives from technology, economy, & educationalresearch (pp. 67–86). New York: Springer.

Lindqvist, E., & Vestman, R. (2011). The labor market returns to cognitive and noncognitive ability: Evidence from the Swedish enlistent.American Economic Journal: Applied Economics, 3, 101–128.

Liu, O. L., Bridgeman, B., & Adler, R. (2013). Learning outcomes assessment in higher education: Motivation matters. Educational Researcher,41, 352–362.

Lohman, D. (1989). Estimating individual differences in information processing models using speed-accuracy models. In R. Kanfer, P.Ackerman, & R. Cudeck (Eds.), Abilities, motivation, and methodology: The Minnesota symposium on learning and individual differences (pp.119–164). Hillsdale, NJ: Lawrence Erlbaum Associates.

Lubinski, D. (2009). Cognitive epidemiology: With emphasis on untangling cognitive ability and socioeconomic status. Intelligence, 37, 625–633.

Lubinski, D. (2010). Neglected aspects and truncated appraisals in vocational counseling: Interpreting the interest-efficacy association from abroader perspective. Journal of Counseling Psychology, 57, 226–238.

Lubinski, D., & Benbow, C. P. (2006). Study of mathematically precocious youth after 35 years: Uncovering antecedents for the developmentof math-science expertise. Perspectives on Psychological Science, 1, 316–345.

Lubinski, D., & Humphreys, L. G. (1997). Incorporating general intelligence into epidemiology and the social sciences. Intelligence, 24, 159–201

MacCann, C., Joseph, D. L., Newman, D. A., & Roberts, R. D. (2014). Emotional intelligence is a second-stratum factor of intelligence:Evidence from hierarchical and bifactor models. Emotion, 14(2), 358–374.

Mackintosh, N. (2011). IQ and human intelligence (2nd ed.). Oxford: Oxford University Press.

285

Macnamara, B. N., Hambrick, D. Z., & Oswald F. L. (2014). Deliberate practice and performance in music, games, sports, education, andprofessions: A meta-analysis. Psychological Science, 25(8), 1608–1618.

Mayer, J. D., Roberts, R. D., & Barsade, S. G. (2008). Human abilities: Emotional intelligence. Annual Review of Psychology, 59, 507–536.Mayer, J. D., Salovey, P., & Caruso, D. R. (2012). The validity of the MSCEIT: Additional analyses and evidence. Emotion Review, 4, 403–

408.McArdle, J. J., Hamagami, F., Meredith, W., & Bradway, K. P. (2000). Modeling the dynamic hypotheses of Gf–Gc theory using longitudinal

life-span data. Learning and Individual Differences, 12(1), 53–79.McDaniel, M. A., & Whetzel, D. L. (2005). Situational judgment test research: Informing the debate on practical intelligence theory.

Intelligence, 33, 515–525.McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric

intelligence research. Intelligence, 37, 1–10.McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between

predictor and criterion domains. Personnel Psychology, 43, 335–354.Melby-Lervåg, M., & Hulme, C. (2013). Is working memory training effective? A meta-analytic review. Developmental Psychology, 49(2), 270–

291.Mislevy, R. J., Oranje, A., Bauer, M. I., von Davier, A., Hao, J., Corrigan, S., Hoffman, E., DiCerbo, K., & John, M. (2014). Psychometric

considerations in game-based assessment. CreateSpace Independent Publishing Platform. Retrieved from http://www.instituteofplay.org/wp-co‐ntent/uploads/2014/02/GlassLab_GBA1_WhitePaperFull.pdf (accessed 15 March 2014).

Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executivefunctions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49–100.

Morrison, A. B., & Chein, J. M. (2011). Does working memory training work? The promise and challenges of enhancing cognition by trainingworking memory. Psychonomic Bulletin & Review, 18(1), 46–60.

Mumford, M. D., Marks, M. A., Connelly, M. S., Zaccaro, S. J., & Johnson, J. F. (1998). Domain-based scoring in divergent-thinking tests:Validation evidence in an occupational sample. Creativity Research Journal, 11(2), 151–163.

Murray, C. (1998). Income, inequality, and IQ. Washington, DC: American Enterprise Institute.Neisser, U. E. (1998). The rising curve: Long-term gains in IQ and related measures. Washington, DC: American Psychological Association.Nickerson, R. S., Perkins, D. N., & Smith, E. E. (1985). The teaching of thinking. Hillsdale, NJ: Erlbaum.OECD (2012). PISA 2009 Technical Report. PISA, OECD Publishing. Retrieved from http://dx.doi.org/10.1787/9789264167872-en

(accessed 13 March 2015).OECD and Statistics Canada. (2000). Literacy in the information age: Final report of the international adult literacy survey. Paris: OECD

Publications.Osaka, N., Logie, R. H., & D’Esposito, M. (Eds.). (2007). The cognitive neuroscience of working memory. New York: Oxford University Press.O’Toole, B. I., & Stankov, L. (1992). Ultimate validity of psychological tests. Personality and individual differences, 13(6), 699–716.Park, G., Lubinski, D., & Benbow, C. P. (2013). When less is more: Effects of grade skipping on adult STEM accomplishments among

mathematically precocious youth. Journal of Educational Psychology, 105, 176–198.Pesta, B. J., McDaniel, M. A., & Bertsch, S. (2010). Toward an index of well-being for the fifty U.S. States. Intelligence, 38, 160–168.Petrides, K. V., & Furnham, A. (2000). On the dimensional structure of emotional intelligence. Personality and Individual Differences, 29(2),

313–320.Plomin, R., Pedersen, N. L., Lichtenstein, P., & McClearn, G. E. (1994). Variability and stability in cognitive abilities are largely genetic later

in life. Behavior Genetics, 24(3), 207–215.Protzko, J., Aronson, J., & Blair, C. (2013). How to make a young child smarter: Evidence from the Database of Raising Intelligence.

Perspectives on Psychological Science, 8(1), 25–40.Ranger, J. (2013). A note on the hierarchical model for responses and response times in tests of van der Linden (2007). Psychometrika, 78(3),

538–544.Ree, M. J., & Earles, J. A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science, 1, 86–89.Ree, M. J., Earles, J. A., & Teachout, M. S. (1994). Predicting job performance: Not much more than g. Journal of Applied Psychology, 79(4),

518.Rindermann, H. (2007). The g-factor of international cognitive ability comparisons: The hom*ogeneity of results in PISA, TIMSS, PIRLS and

IQ-tests across nations. European Journal of Personality, 21, 667–706.Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A., & Goldberg, L. R. (2007). The power of personality: The comparative validity of

personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological Science, 2(4),313–345.

286

http://www.instituteofplay.org/wp-content/uploads/2014/02/GlassLab_GBA1_WhitePaperFull.pdf

http://www.dx.doi.org/10.1787/9789264167872-en

Roberts, R. D., Goff, G. N., Anjoul, F., Kyllonen, P. C., Pallier, G., & Stankov, L. (2000). The Armed Services Vocational Batter (ASVAB) –Little more than acculturated learning (Gc)!?, Learning and Individual Differences 12, 81–103.

Sackett, P. R., Kuncel, N. R., Arneson, J. J., Cooper, S. R., & Waters, S. D. (2009). Does socioeconomic status explain the relationshipbetween admissions tests and post-secondary academic performance? Psychological Bulletin, 135, 1–22.

Sawin, L., Earles, J., Goff, N., & Chaiken, S. R. (2001). Advanced personnel testing project. Final report. Technical report no. AFRL-HE-AZ-TP-2001-0004 ADA399062. Mesa, AZ: Air Force Research Laboratory Warfighter Training Research Division (AFRL/HEA).

Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22(1), 53–61.Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical

implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274.Schweinhart, L. J., Barnes, H. V., & Weikart, D. P. (1993). Significant benefits: The High/Scope Perry preschool study through age 27. Ypsilanti,

MI: High/Scope Press.Segal, C. (2012). Working when no one is watching: Motivation, test scores, and economic success. Management Science, 58(8), 1438–1457.Segal, J. W., Chipman, S. F., & Glaser, R. (1985). Thinking and learning skills: Relating instruction to research. London: Routledge.Shipstead, Z., Redick, T. S., & Engle, R. W. (2012). Is working memory training effective? Psychological Bulletin, 138(4), 628.Snow, R. E. (1999). Commentary: Expanding the breadth and depth of admissions testing. In S. Messick (Ed.), Assessment in higher education

(pp. 133–140). Hillsdale, NJ: Erlbaum.Snow, R.E., Kyllonen, P.C., & Marshalek, B. (1984). The topography of ability and learning correlations. In R. J. Sternberg (Ed.), Advances in

the psychology of human intelligence (Vol. 2., pp. 47–103). Hillsdale, NJ: Lawrence Erlbaum Associates.Sonnleitner, P., Keller, U., Martin, R., & Brunner, M. (2013). Students’ complex problem-solving abilities: Their structure and relations to

reasoning ability and educational success. Intelligence, 41, 289–305.Spearman, C. (1904). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15(2), 201–292.Stankov, L. & Lee, J. (2008). Confidence and cognitive test performance. Journal of Educational Psychology, 100, 961–976.Stanovich, K. E., & West, R. F. (2008). On the relative independence of thinking biases and cognitive ability. Journal of Personality and Social

Psychology, 94, 672–695.Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and

Social Psychology, 69(5), 797–811.Sternberg, R. J. (1977). Intelligence, information processing, and analogical reasoning: The componential analysis of human abilities. Hillsdale, NJ:

Lawrence Erlbaum Associates.Sternberg, R. J. (1985). Beyond IQ: A triarchic theory of intelligence. Cambridge: Cambridge University Press.Sternberg, R. J. (1994). 468 factor-analyzed data sets: What they tell us and don’t tell us about human intelligence. [Review of John B. Carroll,

Human cognitive abilities.] Psychological Science, 5(2), 63–65.Sternberg, R. J., & Wagner, R. K. (Eds.). (1986). Practical intelligence: Nature and origins of competence in the everyday world. New York:

Cambridge University Press.Teasdale, T. W., & Owen, D. R. (2008). Secular declines in cognitive test scores: A reversal of the Flynn effect. Intelligence, 36, 121–126.Thorndike, R. L. (1986). The role of general ability in prediction. Journal of Vocational Behavior, 29(3), 332–339.Thurstone, L. L. (1934). The vectors of mind. Psychological Review, 41(1), 1.Thurstone, L. L. (1938). Primary mental abilities. Psychometric Monographs, 1, ix–121.Thurstone, L. L. (1947). Multiple factor analysis. Chicago, IL: University of Chicago Press.Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The Cognitive Reflection Test as a predictor of performance on heuristics and biases

tasks. Memory & Cognition, 39, 1275–1289.Underwood, B. J. (1975). Individual differences as a crucible in theory construction. American Psychologist, 30, 128–134.Valentin Kvist, A. V., & Gustafsson, J.-E. (2008). The relation between fluid intelligence and the general factor as a function of cultural

background: A test of Cattell’s investment theory. Intelligence, 36(5), 422–436.van der Linden, W. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287–308.Vernon, P. E. (1965). Ability factors and environmental influences. American Psychologist, 20(9), 723–733.Visser, B. A., Ashton, M. C., & Vernon, P. A. (2006a). Beyond g: Putting multiple intelligences theory to the test. Intelligence, 34, 487–502.Visser, B. A., Ashton, M. C., & Vernon, P. A. (2006b). g and the measurement of multiple intelligences: A response to Gardner. Intelligence,

34, 507–510.Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological

knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817–835.Webb, R. M., Lubinski, D., & Benbow, C. P. (2007). Spatial ability: A neglected dimension in talent searches for intellectually precocious

youth. Journal of Educational Psychology, 99, 397–420.

287

Whetzel, D. L., & McDaniel, M. A. (2006). Prediction of national wealth. Intelligence, 34(5), 449–458.Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in

Education, 18(2), 163–183.Wittmann, W. W., & Suess, H. M. (1999). Investigating the paths between working memory, intelligence, knowledge, and complex problem-

solving performances via Brunswick symmetry. In P. Ackerman, P. C. Kyllonen, & R. D. Roberts (Eds.), Learning and individual differences:Process, trait, and content determinants (pp. 77−108). Washington, DC: American Psychological Association.

Woodco*ck, R. W., McGrew, K. S., & Mather, N. (2007). Woodco*ck-Johnson¯ III Normative Update (NU) complete. Rolling Meadows, IL:Riverside Publishing.

Yoachum, C. S., & Yerkes, R. M. (Eds.) (1920). Army mental tests. New York: Henry Holt.Yung Y-F., Thissen, D., & McLeod, L. D. (1999). On the relation between the higher-order factor model and the hierarchical factor model.

Psychometrika, 64, 113–128.Zeidner, J., & Johnson, C. D. (1994). Is personnel classification a concept whose time has passed? In M. G. Rumsey, C. B. Valker, & J. H.

Harris (Eds.), Personnel selection and classification (pp. 377–410). Hillsdale, NJ: Lawrence Erlbaum Associates.

288

10Cognition and Cognitive Disabilities

H. LEE SWANSON

University of California, Riverside The study of cognition as applied to human behavior has been characterized as an attempt to understand thenature of human intelligence and how people think (Anderson, 1976). Thus, to understand disorders ofcognition in children, as well as adults, requires a focus on various domains of intelligent behavior. Thesedomains cover a host of areas such as language, reading, arithmetic, motor skills, attention, and socialinteractions, to mention a few. To limit our review, we will focus on children with specific cognitivedifficulties. Specific cognitive disabilities can be contrasted with general cognitive disabilities. Children withgeneral cognitive disabilities experience inefficiencies and deficits across a wide range of skills. Children withDown’s syndrome, for example, frequently have problems mastering multiple academic skills, whereaschildren with specific learning disabilities have isolated deficits in cognition related to problems in areas suchas reading or math. In practice, the distinction between specific and general learning difficulties is based on astandardized intelligence test (Cheung et al., 2012; Haworth et al., 2009; Hulme & Snowling, 2009;McGrath et al., 2011). Individuals with intelligence quotient (IQ) scores below a certain threshold, such as75, may be viewed as having general to moderate disabilities, whereas those within the average range ofintelligence (e.g., 85–120) may suffer more specialized deficits in specified areas of learning (e.g., math orreading). There are several excellent texts (e.g., Alloway & Gathercole, 2007; Hulme & Snowling, 2009;Yeates, Ris, Taylor, & Pennington, 2010) that provide an indepth review of developmental disorders ofcognition related to language and learning. The reader is referred to those sources for a more comprehensivereview of the literature than this chapter can provide.

Cognitive Models

Without a doubt, there are a number of models that experts in the field have used to explain inefficiencies incognition for children who have average intelligence (e.g., McGrath et al., 2011). For example, one set ofmodels tends to specify how the brain is related to behavior (brain–behavior relationships; e.g., Di Martino etal., 2009). Another seeks to account for the nature of specific cognitive deficits in specific domains offunctioning, such as language, spatial cognition, social behavior, and executive capacity (e.g., Nittrouer &Pennington, 2010). Others draw in part from the cognitive sciences and in part from experimentalpsychology. There are some models that take cross-fertilization into consideration, such as research related toneuroscience, advances in technology (e.g., transcranial magnetic stimulation, functional magnetic resonanceimaging, and diffuse tensor imaging), and genetics (e.g., Pennington et al., 2009, 2012). These data arebrought into the context of the child’s experience within her environment and the cognitive changes that

289

occur within the child over the course of development.Given the constraints of this chapter, however, we cannot cover the various degrees of complexity that may

underlie cognitive disabilities in children. Instead, we narrow our focus on cognitive processes that mayunderlie some academic difficulties experienced by such children. Specifically, we elected to discuss problemsrelated to working memory (WM) because it encapsulates several areas of cognition and academicachievement (e.g., Swanson & Alloway, 2012). To capture this literature, the outcomes of several meta-analyses are reviewed.

However, prior to summarizing the literature on this topic, it is important to indicate how memoryperformance was categorized and the advantages of meta-analyses.

Memory

To understand the role of memory as an underlying factor of specific disabilities, we divide our review alongthe lines outlined by Baddeley (Baddeley, 2012; Baddeley & Logie, 1999) on WM. The majority of researchlinking memory to performance on academic measures follows the tripartite view of WM by Baddeley (e.g.,Baddeley, 1986; Baddeley & Logie, 1999). This relatively simple model has accommodated a number ofexperimental studies of normal adults, children, and neuropsychological patients (see Gathercole & Baddeley,1993, for a review of earlier studies). Although Baddeley’s multicomponent model was primarily developedfrom research on adult samples, the model also has an excellent fit to the WM performance of children(Alloway, Gathercole, Willis, & Adams, 2004; Gathercole, Pickering, Ambridge, & Wearing, 2004;Swanson, 2008).

This tripartite view characterizes WM as comprising a central executive controlling system that interactswith a set of two subsidiary storage systems: the speech-based phonological loop and the visual-spatialsketchpad. The central executive is involved in the control and regulation of the WM system. According toBaddeley (Baddeley, 2012; Baddeley & Logie, 1999), WM coordinates the two subordinate systems, focusingand switching attention, and activating representations within long-term memory (LTM). The centralexecutive is thought to play an important role in “controlled attention,” which coincides with Norman andShallice’s (1986) supervisory attentional system (SAS) model. The phonological loop, commonly referred to asverbal short-term memory (STM) storage, is responsible for the temporary storage of verbal information;items are held within a phonological store of limited duration, and the items are maintained within a storethrough the process of subvocal articulation. The visual-spatial sketchpad is responsible for the storage ofvisual-spatial information over brief periods and plays a key role in the generation and manipulation of mentalimages. This model has been revised to include an episodic buffer (Baddeley, 2000); however, support for thetripartite model has been found across various age groups of children (Gathercole et al., 2004; Swanson,Jerman, & Zheng, 2008). We will briefly review published syntheses that have divided the findings onchildren’s disabilities along the dimensions of Baddeley’s multicomponent model.

Synthesis of the Memory Literature

Because of the extensive number of studies on children’s cognitive disabilities and memory, this chapter reliedon the published outcomes of meta-analyses, when possible, to summarize the findings on the relationshipbetween the components of WM and children’s cognitive disabilities. Meta-analysis, coined earlier by Gene

290

Glass (Glass, McGraw, & Smith, 1981), refers to a statistical technique used to synthesize data from separatecomparable studies in order to obtain a quantitative summary of research that addresses a common question.There are many different metrics to describe an effect size; one of the most popular is the d-index. The d-index, by Cohen (1988), is a scale-free measure of the separation between two group means that is used whenone variable in the relation is dichotomous (e.g., children with cognitive disabilities vs. children withoutcognitive disabilities) and the other is continuous. Calculating the d-index for any study involves dividing thedifference between the two group means by either their average standard deviation or the standard deviationof the control group. To make ds more interpretable, statisticians have adopted Cohen’s (1988) system forclassifying ds in terms of their size (i.e., 0.00–0.19 is described as trivial; 0.20–0.49, small; 0.50–0.79,moderate; 0.80 or higher, large).

There are a number of advantages of meta-analysis over traditional narrative techniques for synthesizingresearch. First, the structured methodology of meta-analysis requires careful review and analysis of allcontributing research. As such, meta-analysis overcomes biases associated with the reliance on single studies,or subsets of studies that inevitably occur in narrative reviews of a literature. Second, meta-analysis allows foreven small and non-significant effects to contribute to the overall conclusions and avoids wasting data becausea sample size was too small and significance was not achieved. Finally, meta-analysis can address questionsabout variables that moderate effects. Specifically, meta-analysis provides a formal means for testing whetherdifferent features of studies can explain variation in their outcomes.

Given this introduction, we summarize meta-analyses findings on WM performance of children withcognitive disabilities. The criteria for selecting articles reporting outcomes related to a meta-analysis werethose: (a) that included children of public school age (age 5–18 years) with specific cognitive disabilities; (b)that reported comparisons between children with cognitive disabilities with average intelligence and childrenwithout cognitive disabilities on memory measures; and (c) for which the results were reported in a refereedjournal. As previously mentioned, the findings from these syntheses were divided into the three componentsof the Baddeley model. Although this division is important to understand the process that may underlievarious cognitive disability groups, it is important to note that problems related to these components of WMare not completely independent. For example, problems in the central executive system would no doubt berelated to potential problems in the various storage systems (e.g., visual-spatial sketchpad). Unfortunately, notall syntheses on WM and cognitive disabilities reported effect sizes. However, when possible we report thecomponent or components found in the synthesized literature that have yielded the largest effect sizes and/ordifferences between children with and without cognitive disabilities.

291

Executive System

The central executive monitors the control processes in WM (e.g., Baddeley, 2012; Kane, Conway,Hambrick, & Engle, 2007). There have been a number of cognitive activities assigned to the central executive,including coordination of subsidiary memory systems, control of encoding and retrieval strategies, switchingof attention in manipulation of material held related to the verbal and visual-spatial systems, and the retrievalof knowledge from LTM (e.g., Baddeley, 2012; Miyake, Friedman, Emerson, Witzki, & Howerter, 2000).Although there is an issue of whether the central executive is a unitary system (it is conceptualized as eitherbeing unitary (Baddeley, 1986) or composed of multiple domain-specific executives (Goldman-Rakic, 1995)),there is some agreement that the central executive has some capacity limitations that influence the efficiencyof these operations (e.g., for allocating attention to, performing operations on). Executive functioning hasseparable operations (e.g., inhibition, updating, attention switching), but these operations may share someunderlying commonalities (e.g., see Miyake et al., 2000, for a review).

Several studies suggest that individual differences in the executive component of WM are directly related toachievement (e.g., reading comprehension) in individuals with average or above-average intelligence (e.g.,Booth, Boyle, & Kelly, 2010; Daneman & Carpenter, 1980; Swanson & Alloway, 2012). Thus, childrenscoring within the normal range on an IQ measure may have difficulties with executive processing that are notisolated to those with depressed intelligence (i.e., intellectually disabled individuals). As indicated below,deficits in the executive system of WM have been attributed to children with specific difficulties in math orreading comprehension. Not all children with executive processing deficits have problems in WM. Forexample, children with autism may have deficits in executive processing that do not entail the executive systemof WM. We review some of the synthesis findings on these three groups of children.

Specific Learning Disorder with Impairment in Math

The new American Psychiatric Association (2013) Diagnostic and Statistical Manual of Mental Disorders(DSM-5) views a specific learning disorder in math as reflecting a neurodevelopmental disorder of biologicalorigin. This disability is manifested in math performance that is markedly below age level that is notattributed to intellectual, developmental, neurological, or motor disorders. This broad category of difficultiesencapsulates children referred to as having math disability (MD) or suffering from dyscalculia. Several studies(Badian, 1983; Geary, 2013) estimate that approximately 6–7% of the school-age population has MD.Although this figure may be inflated due to variations in definition (e.g., Reigosa-Crespo et al., 2012, suggestthe figure varies around 3%), a significant number of children in U.S. schools demonstrate poor achievementin mathematics. Although not a quantitative analysis, one of the most comprehensive syntheses of thecognitive literature of MD was provided by Geary (1993; also see Geary, 2013). His earlier review indicatedthat children with MD are a heterogeneous group and show one of three types of cognitive disorders:semantic memory (e.g., characterized as having weak fact retrieval and high error rates in recall), procedural(developmentally immature procedures in numerical calculations), and visual-spatial math disorders (e.g., havedifficulties representing numerical information spatially).

The cognitive, as well as neural, mechanisms underlying these types of math disorders are still underinvestigation. There is some consensus, however, that arithmetic facts in children with MD are not retrieved

292

accurately, and various deficits may reflect various forms of deficits related to neural structures, specifically theleft basal ganglia, thalamus, and the left parieto-occipitotemporal areas (e.g., Dehaene & Cohen, 1997).Regardless of the type of disorder and the theoretical account for MD, the majority of studies suggest thatchildren with MD have some type of memory deficit. Theories of the representation of arithmetic facts inLTM indicate that performance on simple arithmetic depends on retrieval from LTM. The strength to whichassociations are stored, and hence, the probability of retrieving them correctly, depends in part on experience,with associations being formed each time an arithmetic problem is encountered, regardless of whether theassociation is correct. Thus, the ability to utilize WM resources to temporarily store numbers whenattempting to reach an answer is of significant importance in learning arithmetic.

A comprehensive meta-analysis of the published literature focused on identifying the cognitive processesthat underlie MD (Swanson & Jerman, 2006). The synthesis focused on the cognitive functioning of childrenwith MD when compared to: (a) average-achieving children, (b) children with reading disabilities (RD); and(c) children with comorbid disabilities (RD + MD). A summary of the results on memory functioning relatedto children with MD and their comparison group is as follows. Average achievers outperformed children withMD on measures of verbal WM (M = –0.70), visual-spatial WM (M = –0.63), STM for words (M = –0.45),STM for numbers (M = –0.26), and LTM (M = –0.72). The results further indicated that children with MDoutperformed children with comorbid disabilities (MD + RD) on measures of LTM (M = 0.44), STM forwords (M = 0.71), verbal WM (M = 0.30), but not STM for numbers (M = –0.08). Interestingly, the effectsizes on the same measures for children with MD and those with RD were small (effect sizes range from –0.30 to 0.16).

Hierarchical linear modeling showed that the magnitude of effect sizes in overall cognitive functioningbetween MD and average achievers was primarily related to WM deficits related to the executive system whenthe effects of all other variables (e.g., age, IQ, reading level, other cognitive domain categories) were partialedout. These findings are consistent with a more recent meta-analysis (David, 2012) that yielded larger effectsizes in favor of controls when compared to children with MD on the executive component of WM.However, no clear-cut differences emerged between children with MD and RD on several memory measures.Swanson, Jerman, and Zheng (2009) extended their meta-analysis to address this issue. They reasoned thatthe poor differentiation between children with MD and those with RD occurred because the studies includedsamples with poor arithmetic skills accompanied by relatively low reading skills.

The Swanson et al. (2009) results indicated moderate (0.50 to high) effect sizes in favor of age-matchedaverage-achieving children on measures of verbal WM (M = –0.53), visual-spatial WM (M = –0.63), andLTM (M = –0.87). Children with MD were also differentiated from children with combined reading andmath disabilities. Specifically, the effect sizes in favor of the MD group when compared to the comorbidgroup were found on measures of verbal WM (M = 0.88), LTM (M = 0.58) and visual-spatial WM (M =0.63). Interestingly, an advantage was found for the comorbid group on measures of STM for digits (M = –0.55). In contrast to comparisons with the comorbid group, children with MD could not be clearlydifferentiated from children with RD on measures related to the phonological loop (STM) or executivesystem of WM (effect size ranges from –0.01 to 0.10). Children with RD did yield small advantage onmeasures of visual WM when compared to children with MD (M = 0.30). Overall, the results from these twometa-analyses provide weak support for the assumption that distinct WM processes separate children with

293

MD from children with RD.In summary, syntheses of the literature have attributed to MD, when compared to average achievers,

deficits in the executive components of WM (see Geary, 2013). However, comparisons between children withMD and children with RD on measures that tap the components of WM have yielded trivial or small effectsizes. Thus, it is possible that an important correlate of memory problems in children with MD is reading.

Reading Comprehension Impairment

Children with reading comprehension impairments can recognize words accurately, but have problemsunderstanding the meaning of what they have read in terms of accuracy and speed. Although there are nopopulation-based studies of this disorder, individual studies (e.g. Nation, Adams, Bowyer-Crane, &Snowling, 1999) suggest that approximately 10% of samples of children with reading problems have readingcomprehension difficulties (Snowling & Hulme, 2012). No doubt, understanding text involves some of thesame processes as comprehending spoken language. Memory problems in children with dyslexia areunderstood in terms of decoding deficits, which are strongly associated with the phonological loop (to bediscussed). Therefore, it is expected that the phonological memory would be normal in children with readingcomprehension impairments.

Because phonological STM is just one component of a WM system, other components of a WM systemhave been investigated. A meta-analysis of this research (Carretti, Borella, Cornoldi, & De Beni, 2009) foundproblems in comprehension related to a general WM system as well as a specific system in children with poorcomprehension. For example, Carretti et al.’s meta-analysis of the literature (18 published studies) on specificreading comprehension difficulties found that tasks that require attention control and the processing of verbalinformation were the best measures to distinguish between poor and good comprehenders. Poorcomprehenders were more disadvantaged on complex span tasks (tasks that draw on the executive system ofWM) that involved verbal material than good comprehenders. In contrast, poor comprehenders werecomparable in performance to good comprehenders on measures of visual-spatial complex (effect size = 0.29)span tasks and simple span (STM or phonological loop) tasks (effect size = 0.36). They concluded that poorcomprehension depends partially on the verbal WM modality (effect sizes in favor of good comprehendersvaried from 0.75 to 1.07). This synthesis also suggested that a failure of the attention control component ofWM underlies poor comprehension. The synthesis found that poor comprehenders were more likely toexpress difficulties in updating information as well as inhibiting irrelevant information when compared togood comprehenders.

In summary, the literature suggests that WM is an important marker of reading comprehension difficulties(also see Locascio, Mahone, Eason, & Cutting, 2010; Pimperton & Nation, 2010; Ricketts, 2011). Inaddition, the literature suggests that children of average intelligence who have a specific deficit incomprehension suffer problems related to the executive system of WM.

Autism Spectrum Disorder

According to the DSM-5 (American Psychiatric Association, 2013) autism spectrum disorder is defined asdescribing such children that reflect persistent communication and social interaction deficits in multiplesituations. These deficits reflect restricted, repetitive behavior and interests and are manifested in the early

294

developmental period. These deficits may be associated with or without intellectual impairment or with orwithout accompanying language impairment. In general, autism reflects a severe persistent social impairmentthat occurs in combination with problems in both verbal and non-verbal communication. There is somedebate in the literature about the prevailing incidence of autism and this most likely relates to a widelyaccepted condition referred to autism spectrum disorder. Autism spectrum disorder includes classic autism, aswell as Asperger syndrome, high-functioning autism, and atypical autism. Current estimates place theprevalence of autism spectrum disorder at approximately one in every hundred children (e.g., Davidovitch,Hemo, Manning-Courtney, & Fombonne, 2013; Russell, Rodgers, Ukoumunne, & Ford, 2014).

An executive processing hypothesis has gained some interest in describing autism because behaviors in suchchildren reflect poor executive controls (e.g., rigidity, perseveration, and repetitive behaviors). However, thereis little evidence of how WM is impaired in children with autism (see Belleville, Ménard, Mottron, &Ménard, 2006, for review). Although WM is not a single hom*ogeneous system, the majority of studiesindicate that the WM abilities of children with autism are in the normal range. For example, a study byOzonoff and Strayer (2001) examined the WM in high-functioning autistic children, children diagnosed withTourette’s syndrome, and a typically developing control group. They found no group differences across variousdependent measures of WM. Although performance was correlated with both age and IQ, they concludedthat WM is not one of the executive functions seriously impaired in autism.

In summary, the results of the various syntheses suggest that autistic children do not show moderate orsevere deficits in WM when compared to typically developing children.

295

Phonological Loop

In Baddeley’s model (2012; Baddeley & Logie, 1999), the phonological loop is specialized for the retention ofverbal information over short periods of time. It is composed of both a phonological store, which holdsinformation in phonological form, and a rehearsal process, which serves to maintain representations in thephonological store (see Baddeley, Gathercole, & Papagno, 1998, for an extensive review). The phonologicalloop has been considered a key area of impairment for children with RD and specific language impairment(e.g., Nithart et al., 2009).

Specific Learning Disorder with Impairment in Reading

The new DSM-5 (American Psychiatric Association, 2013) views a specific learning disorder in reading asreflecting a neurodevelopmental disorder of biological origin. This disability is manifested in readingperformance markedly below age level that is not attributed to intellectual, developmental, neurological, ormotor disorders. This broad category includes more specific deficits referred to as dyslexia or RD. Theincidence of dyslexia in public schools has been reported to vary between 5% and 17% in the United States(McCandliss & Noble, 2003), although more conservative estimated prevalence rates range from 5% to 7% ofthe general population. The National Institute of Neurological Disorders and Stroke (2010) gives thefollowing definition for dyslexia:

Dyslexia is a brain-based type of learning disability that specifically impairs a person’s ability to read. These individuals typically read atlevels significantly lower than expected despite having normal intelligence. Although the disorder varies from person to person, commoncharacteristics among people with dyslexia are difficulty with spelling, phonological processing (the manipulation of sounds), and/or rapidvisual-verbal responding.

Several studies indicate that children with RD (dyslexia) have specific localized low-order processing deficits.A cognitive process consistently implicated in RD is phonological awareness. Phonological awareness is “theability to attend explicitly to the phonological structure of spoken words” (Scarborough, 1998, p. 95). Dyslexiais viewed as a specific developmental disorder for which a modular impairment in phoneme-grapheme systemknowledge, or a phonological deficit, has been postulated. Thus, there is a strong application related to ourunderstanding of the role of the phonological loop as it relates to dyslexia or also referred to as RD.

The manifestations of deficits in the phonological loop include poor acquisition of sight words, poorperformance on phonological awareness tasks, slow naming speed, and impaired verbal STM. Several studiessuggest that difficulties in the phonological loop may lie at the root of word-learning problems in children(e.g., see Melby-Lervåg, Lyster, & Hulme, 2012). For example, children with RD are less able to generatepronunciations for unfamiliar or nonsense words than skilled readers (e.g., Siegel, 1993), suggesting a deficitin utilization or operation of the phonological recoding function of the articulatory control process.

Swanson, Zheng, and Jerman (2009) synthesized research that compared children with and without RD onmeasures of the phonological loop (STM) and the executive system of WM (tasks that included simultaneousprocessing and storage). In general, 578 effect sizes were computed across a broad range of age, reading, andIQ scores, yielding a mean effect size across studies of –0.89 (SD = 1.03) in favor of children without RD. Inall, 257 effect sizes were in the moderate range for STM measures (M = –0.61, 95% confidence range of –0.65to –0.58) and 320 effect sizes were in the moderate range for WM measures (M = –0.67, 95% confidence

296

range of –0.68 to –0.64). The results indicated that children with RD were distinctively disadvantaged

compared to average readers on (a) STM measures requiring the recall of phonemes and digit sequences and(b) WM measures requiring the simultaneous processing and storage of digits within sentence sequences andfinal words from unrelated sentences. No significant moderating effects emerged for age, IQ, or reading levelon memory effect sizes. In addition these difficulties, related to STM and WM, have emerged whensynthesizing the literature on adults with RD (e.g., Swanson & Hsieh, 2009).

In summary, syntheses of the published literature on RD indicated that domain-specific STM (measures ofthe phonological loop) and WM (measures reflective of the executive system) deficits persisted across age,suggesting that children with RD fail to efficiently draw or monitor resources from both a phonological andexecutive system. The outcomes of this earlier synthesis are consistent with more recent studies (Johnson,Humphrey, Mellard, Woods, & Swanson, 2010; Menghini, Finzi, Carlesimo, & Vicari, 2011), again showingthat RD involves difficulties in both the phonological loop and the executive system. What is unclear from theliterature, however, is whether deficits in phonological loop create a bottleneck in the processing ofinformation in the executive system or whether problems in the phonological loop and executive system reflectindependent difficulties.

Specific language impairment. The term communication disorder is used in the DSM-5 (AmericanPsychiatric Association, 2013) to describe children with persistent deficits in comprehension or production oflanguage (e.g., spoken, written) substantially below age level, beginning in the early developmental period.Specific language impairment is typically identified through the achievement of low scores on a standardizedlanguage measures and intelligence scores falling within the normal range. Hulme and Snowling’s (2009)comprehensive review indicated that 3–6% of the child population suffers from various aspects related tospecific language impairment. This figure varies related to the type of language impairment becauseheterogeneity exists within the sample related to vocabulary, grammar, and phonology (Archibald &Gathercole, 2006a, 2006b).

A comprehensive review of the literature by Montgomery, Magimairaj, and Finney (2010) showed a strongrelationship between STM (phonological loop) and processing speed in children with specific languageimpairment which in turn led to widespread negative effects in language learning and functioning, includingthe partial processing of words, grammatical forms, and syntactic structures. Memory limitations were viewedas affecting the acquisition representations of language processes as well as children’s efficacy and how theystore, access, and retrieve coordinate-stored information related to input and output of language.

Although Montgomery and colleagues’ (2010) review focused on the literature related to phonological loop(STM system), the researchers also reviewed studies showing that such children suffered problems in a generalexecutive capacity or attentional capacity system. Three activities of the executive system of WM werereviewed: shifting, updating, and attentional control. Children with specific language impairment were foundto be comparable in shifting, but were weak in updating related to WM, and poor attention control (also seeHenry, Messer, & Nash, 2012). Their review also suggested that WM capacity and linguistic knowledge werenot necessarily separable mental constructs. Rather, WM capacity reflected the activation of specific linguisticrepresentations in LTM. In this view, WM capacity was viewed as a reflection of weak linguisticrepresentations.

297

The above syntheses do not imply however, that children with specific language impairment do not sufferfrom problems in visual-spatial WM. A recent meta-analysis (Vugs, Cuperus, Hendriks, & Verhoeven, 2013)suggests that visual-spatial WM (visual-spatial sketchpad) is affected in these children. Their results indicatedsuch children showed effect sizes for visual-spatial storage of d = 0.49 and of storage + processing of d = 0.63when compared to their counterparts. However, their synthesis suggested that the deficit in verbal aspects ofWM in children with specific language impairment could be “two to three times larger than the deficit intheir visuospatial WM” (Vugs et al., 2013, p. 2593).

Taken together, the results suggest that clear deficits in verbal components of WM emerge for childrenwith SLI. However, the findings related to problem in visual-spatial WM, suggesting that problems mayextend to more domain-general deficits in WM.

298

Visual-Spatial Sketchpad

The visual-spatial sketchpad is specialized for the processing and storage of visual material, spatial material, orboth, and for linguistic information that can be recoded into imaginal forms (see Baddeley, 2000, 2012 for areview). Measures of visual-spatial WM (visual-spatial sketchpad) have primarily focused on memory forvisual patterns (e.g., Logie, 1986). Gathercole and Pickering (2000) found that visual-spatial WM abilities, aswell as measures of central executive processing, were associated with attainment levels on a nationalcurriculum (U.K.) for children aged 6–7 years. Children who showed marked difficulties in curriculumattainment also showed marked difficulties in visual-spatial WM.

There have been few quantitative syntheses, to the author’s knowledge, that have covered the visual-spatialdifficulties of children with cognitive disabilities. A meta-analysis (Swanson & Jerman, 2006) of children withmath disabilities, for example, did not find strong support for the notion that math disabilities were related tovisual-spatial disorders (however, see Mammarella, Lucangeli, & Cornoldi, 2010). Some links to specificcognitive deficits in visual-spatial WM, however, have been found in children with attention deficithyperactivity disorder (ADHD), coordination disorders, and Williams syndrome. These findings are brieflyreviewed.

Attention Deficit Hyperactivity Disorder

ADHD is a chronic condition that impairs an individual’s ability to control attention in an optimal manner.DSM-5 (American Psychiatric Association, 2013) views ADHD as reflecting a “persistent pattern ofinattention and/or hyperactivity-impulsivity that interferes with functioning or development beginning inchildhood, and present across more than one setting” (p. 61). The incidence of ADHD is betweenapproximately 3 and 5% of children of primary school age (e.g., DSM-5). ADHD is somewhat dissimilarfrom some of the other cognitive disabilities we have reviewed thus far, since disabilities related to RD,specific language deficits, and math disabilities might be seen as modular disorders. In contrast, ADHD ismuch less clear in terms of specific cognitive explanations. However, behavioral inhibition or executivefunctioning as reflected in high-level supervisory systems has characterized this disability (e.g., Tillman,Eninger, Forssman, & Bohlin, 2011). As such, ADHD is a difficult disorder to characterize on the cognitivelevel, but there have been several studies that tie ADHD to problems in visual WM performance.

For example, Martinussen, Hayden, Hogg-Johnson, and Tannock’s (2005) meta-analysis reviewed studiesthat compared ADHD and control children on WM tasks. Their synthesis showed that children withADHD reflected greater deficits in spatial-memory storage tasks (average effect size of 0.85) and spatialcentral executive tasks (average effect size of 1.06) than verbal storage or verbal central executive tasks (meaneffect sizes range from 0.40 to 0.47). Their synthesis of the literature also suggested that these WM problemscould not be accounted for by general differences in language skill or general intelligence. In general, theirsynthesis suggested that spatial WM is a specific area of difficulty for children with ADHD.

Other meta-analyses have examined executive processing tasks as a means of differentiating ADHD fromother disabilities (Willcutt, Doyl, Nigg, Faraone, & Pennington, 2005). These executive processing tasks haveassessed response inhibition, updating (WM), and task shifting. The largest difference in favor of the controlgroup emerged on measures of response inhibition and WM. However, the magnitude differences on some of

299

these tasks (effect sizes ranged from 0.40 to 0.60) were somewhat smaller than would be expected if theseexecutive processing tasks were a major cause of the disorder. Overall, the Willcutt et al. (2005) synthesisraised concerns as to whether deficits in executive functioning per se are a sufficient or necessary cause ofADHD.

In summary, the literature suggests that children with ADHD suffer problems in WM, primarily on tasksthat draw upon the visual-spatial sketchpad. However, problems in WM are perhaps secondary whencompared to cognitive activities related to other areas of executive processing, such as the monitoring ofattention (e.g., Willcutt et al., 2005).

Coordination Disorders

The DSM-5 (American Psychiatric Association, 2013) defines coordination disorders as motor skilldevelopment substantially below a child’s age group that interferes with normal activities, and begins in the“early developmental period.” These problems cannot be attributed to intellectual disabilities, visual problems,or a neurological condition such as cerebral palsy. The criteria for diagnosing children with developmentalcoordination disorders (DCD) focus on problems in motor coordination that are clearly out of line with thechild’s chronological age and intellectual functioning. These problems interfere with academic achievement inareas such as handwriting and/or activities related to everyday life such as sports games and perhaps learningto dress. The prevalence of this disability in the general population is estimated between 5% and 18%,depending on the level or point of the cutoff scores for determining risk (Alloway, 2011). The cognitiveexplanation for DCD has focused on perceptual deficits, visual perceptual deficits, problems in kinestheticperception, and balance/postural control (Alloway, 2011).

Alloway’s (2006) and Wilson, Ruddock, Smits-Engelsman, Polatajko, and Blank’s (2013) syntheses of theliterature suggested that visual-spatial deficits were related to children’s impaired development of a“sensorimotor map” and “one’s position in space.” Their syntheses suggested that children with DCD performpoorly on all WM measures in terms of standardized scores; however, their performance levels on visual-spatial WM (visual-spatial sketchpad) measures were substantially lower than verbal WM, verbal STM, andvisual-spatial STM tasks. An interesting finding from Alloway’s synthesis (2006) was that the performance ofchildren with DCD on visual-spatial STM tasks was no worse than their performance on verbal STM andverbal WM tasks. Although the visual STM tests involved some motor skills, her synthesis suggested thatchildren with DCD struggled with dual task demands (process and storage component on a visual-spatialWM task) related to the processing of visual-spatial information and motor coordination.

In summary, the literature suggests that children with DCD experience a selective deficit in visual-spatialWM that is linked with movement planning and motor control.

Williams syndrome. Williams syndrome is a rare neural developmental disorder characterized by a distinctivefacial appearance and a developmental delay in visual-spatial abilities coupled with strong language skills(Gonçalves et al., 2011). Williams syndrome is caused by a deletion of 26 genes from the long arm ofchromosome 7. Williams syndrome is viewed as a disorder in which individuals have an extensive reliance onverbal WM and language acquisition, which is stronger than would be found for typically developing children.The study of Williams syndrome is of interest because it suggests some of the independence among verbal and

300

visual-spatial systems. The cognitive profile for Williams syndrome is characterized by a relative strength inthe phonological loop (verbal STM, as typically measured by a digit recall task), but a severe weakness in thevisual-spatial sketchpad (visual-spatial memory).

Rowe and Mervis’ (2007) review of the literature found that WM tasks that included verbal material forindividuals with Williams syndrome when compared to typically developing children followed the normaleffects of word length, phonological similarity, and concreteness. However, children with Williams syndromeexperienced clear difficulties in tasks that required visual-spatial construction. A meta-analysis by Lifsh*tz etal. (Lifsh*tz, Shtein, Weiss, & Svisrisky, 2011a; Lifsh*tz, Shtein, Weiss, & Vakil, 2011b) that included fivearticles that compared children with Williams syndrome to typically developing children found that the meaneffect size differences on verbal memory tasks was approximately 0.45. Although the effect size was moderate,the authors argue that children with Williams syndrome have a preserved explicit verbal memory system whencompared to their visual-spatial WM system.

In summary, children with Williams syndrome are characterized as having a severe weakness in the visual-spatial WM system when compared to their verbal WM skills. However, weaknesses in verbal WM are alsoapparent when compared to their normal-achieving counterparts.

301

The Paradox of Normal Intelligence

There is no doubt that a paradox exists in the findings related to specific cognitive disabilities and overallnormal intellectual functioning. Given that children within the average range on intelligence measuresperform with difficulties on tasks that tap specific components of WM (updating, suppression of competingtraces), what is the role of WM on intelligence in children with specific cognitive disabilities? How is it thatchildren with specific deficits in cognitive processes have average intelligence? We did not come across asingle review that examined these issues. These are particularly complex issues because performance on WMtasks is strongly correlated with intelligence (e.g., Ackerman, Beier, & Boyle, 2002; Duncan, Schramm,Thompson, & Dumontheil, 2012; Chuderski, 2013; Kyllonen & Christal, 1990).

Three points can be considered when tackling this issue. First, the relationship between WM andintelligence may be indirect (e.g., Alloway, 2009). Crinella and Yu (2000) reviewed literature suggesting aweak relationship between IQ and executive processing with normal-achieving children. Similarly, theliterature clearly shows that poor readers with high IQ levels, when compared to poor readers with low IQlevels, can yield statistically equivalent performance on cognitive measures (e.g., phonological processing;Hoskyn & Swanson, 2000; Siegel, 1993). Further, these commonalities in performance are not isolated tomemory or phonological processing measures (see Hoskyn & Swanson, 2000 for a meta-analysis comparingRD and garden-variety poor achievers across an array of cognitive measures). For example, weak to moderaterelations exists between WM and fluid intelligence (performance on the Raven Colored Progressive Matricestest) in children with RD. Swanson and Alexander (1997) found that the magnitude of the correlationsbetween executive processing and fluid intelligence (Raven Colored Progressive Matrices test) varied between0.04 and 0.34 in RD and between –0.05 and 0.46 in average readers (see Swanson & Alexander, 1997, Table4).

Second, children with specific cognitive disabilities may use different routes or processes to problem solve,even though solution accuracy is comparable to chronologically matched peers. For example, Swanson (1988,1993) found that children with RD successfully set up a series of subgoals for task solution. Further, thechildren with RD’s problem-solving performance was statistically comparable to their chronologicallymatched peers on a number of fluid measures of intelligence (Picture Arrangement subtest on the WechslerChildren’s Intelligence Test, Swanson, 1988; Tower of Hanoi, Combinatorial, and Pendulum Task; Swanson,1993). However, the studies also found that individuals with RD in some cases relied on different cognitiveroutes than skilled readers in problem solving. For example, on measures of fluid intelligence, problem solvingwas augmented by “emphasizing problem representation (defining the problem, identifying relevantinformation or facts given about the problem) rather than procedural knowledge or processes used to identifyalgorithms” (Swanson, 1993, p. 864). Thus, there is evidence to suggest that performance by individuals withRD on fluid measures of intelligence may involve some form of compensatory processing. However, littleresearch has focused on the compensatory processes that underlie the links between intelligence and executiveprocessing.

Finally, individuals may achieve scores within the range of normal intelligence because the information theyexperience in their environment does not always place high demands on their WM. A standardized test ofWM (S-Cognitive Processing Test: Swanson, 1995) shows, for example, that the majority of children with

302

serious academic difficulties (learning disabilities) scored in the 21st percentile on WM measures (scaledscores across 11 subtests hovered around 8, or a standard score of 88; see Swanson, 1995, p. 167), suggestingthey have a very weak, but adequate WM ability to process information and then store information over thelong term. Of course, they may use other experiences by pulling up from LTM things that they already knowto help in the processing of information. With the accumulation of LTM links and connections, there is somecontrol over the processing demands of new information. Thus, this control over processing demands mayreduce any potential links between intelligence and WM. No doubt, the processes that link specific processingdeficits with normal intelligence have not been elucidated.

303

Discussion and Implications

This chapter reviewed studies that synthesized research on WM for children with specific cognitive deficits.WM was selected as a focus because of its significant relationship to behaviors such as learning to read,compute, acquire language, coordinate motor behaviors, and/or monitor attention. Performance on specificcomponents of WM was further considered in the analysis since confirmatory factor analysis models haveprovided support of a multicomponent model (Baddeley, 2012; Baddeley & Logie, 1999) when applied tochildren (e.g., Alloway et al., 2004; Gathercole et al., 2004; Swanson, 2008).

The synthesized studies reviewed here have shown that children with average intelligence experiencespecific difficulties in reading, math, language, and writing (i.e., motor coordination) that are related toproblems in WM. However, in our review we did not find clear support for the notion that a specific deficit ina particular component of WM underlies a specific cognitive disability. That is, when children with cognitivedisabilities were compared to children matched on measures of intelligence without cognitive disabilities, theeffect sizes in favor of children without cognitive difficulties may be larger in one component of WM whencompared to another, but children with cognitive disabilities may also yield a weakness across othercomponents of WM. For example, children with RD (dyslexia) when compared to their peers were found toexperience deficits in the phonological loop, but were also found deficient in the executive component of WM(e.g., Swanson et al., 2009a). Likewise, children with serious difficulties related to the visual-spatial sketchpad(Williams syndrome) were found to have difficulties related to the phonological loop (Lifsh*tz et al., 2011a,2011b). What these findings suggest is that, although a neurodevelopmental inefficiency of biological originmay underlie such children’s disability, the manifestations of these inefficiencies may reflect problems acrossmultiple components of WM.

So what are the educational implications of these findings? We suggest two applications. The first relates toassessment. Assuming that variables such as general intelligence, quality of instruction, and relatedenvironmental variables in the classroom are controlled or accounted for, the literature suggests that some ofthe specific learning difficulties experienced by children are related to the phonological loop (a component ofWM that specializes in the retention of speech-based information), the visual-spatial sketchpad (a componentthat focuses on visual-spatial processing) and/or the executive system (a component that focuses on controlledattention) of WM. A number of the syntheses suggested that some children with cognitive disabilities aremore likely to experience more severe deficits in one component of WM than another. For example, childrenwith RD (dyslexia) and specific language deficits are distinctly disadvantaged compared to their peers forremembering verbal information, specifically phonological items (phonemes). Children with DCD or ADHDdo poorly on visual-spatial WM tasks. In contrast, children with math disabilities or reading comprehensiondeficits do poorly on tasks that reflect the monitoring of process and storage demands, a characteristic of theexecutive system.

However, it is also important to note in this assessment process that problems in one area of WM do notmean they will not experience difficulties in another. For example, it is important to note that thephonological loop is of service in complex cognition, such as reading comprehension and problem solving.Thus, this simple subsystem is not the only aspect of WM that is deeply rooted in more complex activitiesexperienced by children who have serious academic problems. Situations that place high demands on

304

processing, which in turn place demands on controlled attentional processing (such as monitoring limitedresources, suppressing conflicting information, updating information), place children with problems in verbaland visual storage (phonological loop, visual-spatial sketchpad) at a clear disadvantage when compared withtheir chronological-aged average-achieving counterparts.

The second relates to intervention. Current research has provided some directions for remediation of WMdifficulties. For example, there is evidence emerging that children’s WM can be improved upon with training(e.g., Klingberg, 2012; Melby-Lervåg & Hulme, 2013) and dynamic testing (e.g., Swanson, 2011). However,these levels of improvement have as yet to be directly linked to improvements in children’s academicperformance (Melby-Lervåg & Hulme, 2013). Unfortunately, at this point the evidence that links WM toachievement is correlation and/or relies on quasi-experimental designs and therefore further experimentationis necessary.

Where should we go from here? Our synthesis of the “synthesized literature” suggests there are gapingholes in our knowledge about how WM and learning in children with cognitive disabilities are related, andtherefore additional research is needed. Some areas of “residual ignorance” (to coin a phrase by Baddeley) areas follows:

Why is WM related to cognitive disabilities and academic achievement? Additional research needs to be directedtoward explaining why WM tasks are good predictors of academic performance. Although, for example, itmakes sense that controlled attention ability (e.g., the ability to switch attention between processing andstorage requirements) may be particularly good accounting for reading comprehension and/or problem-solvingdeficits, but not necessarily for simple sight word recognition (dyslexia), this has not yet been testedexperimentally. No doubt, the complexity of the task determines whether general or domain-specific factorscome into play. However, different capacity-limited factors may come into play in predicting achievementacross elementary, junior high and high school. Further, one can only speculate on how children with readingand math disabilities are able to attain normal levels of functioning in everyday cognition.

How processes are represented. We do not know how the basic mechanisms of WM are represented inchildren who experience serious deficits in reading and/or math. Basic mechanisms have been explored insome detail in the area of phonological loop (Baddeley et al., 1998); however, processes related to theintegrated nature of storage and processing characteristic of WM tasks have not been adequately studied. Inaddition, research is unclear about primary mechanisms within the executive system that separate low- andhigh-WM groups as it applies to information maintenance. This is because monitoring activities areintertwined with maintaining information. That is, monitoring activities such as: (a) switching attentionbetween multiple tasks; (b) active inhibition or suppression of irrelevant information; (c) updatinginformation; and (d) planning and sequencing intended actions are very much related to informationmaintenance. Further, it is difficult to know how all of these particular activities are related to one another, orif, in fact, they are independent.

Instructional contributions. Classroom research has not identified all factors of WM amenable to particularmanipulations, such as extended practice, specific instructions, and strategy use. For instructional purposes,specific research needs to be directed toward the ways in which LTM contribute to WM. Further research is

305

necessary to determine how LTM representations can be activated or taught to support WM operations. If wecan show that an enhancement of domain-specific WM factor is primarily related to a learned skill orknowledge, then clearly an environmental factor (e.g., instruction) plays an important role.

In summary, this review suggests that a WM system underlies some of the cognitive disabilities experiencedby children. Although WM is obviously not the only skill that contributes to academic difficulties in children,WM does play a significance role in accounting for individual differences in children’s academic performance.Depending on the academic task, age, and type of learning problem (reading and/or math), general andspecific WM systems may be involved in learning and academic difficulties.

306

References

Ackerman, P. L., Beier, M. E., & Boyle, M. O. (2002). Individual differences in working memory within a nomological network of cognitiveand perceptual speed abilities. Journal of Experimental Psychology: General, 131, 567–589.

Alloway, T. P. (2006). Working memory skills in children with developmental coordination disorder. In T. P. Alloway & S. E. Gathercole(Eds.), Working memory and neurodevelopmental disorders (pp. 161–186). New York, NY: Psychology Press.

Alloway, T. P. (2009). Working memory, but not IQ, predicts subsequent learning in children with learning difficulties. European Journal ofPsychological Assessment, 25(2), 92–98.

Alloway, T. P. (2011). A comparison of working memory profiles in children with ADHD and DCD. Child Neuropsychology, 17(5), 483–494.Alloway, T. P., & Gathercole, S. E. (Eds.) (2007). Working memory and neurodevelopmental disorders. New York, NY: Psychology Press.Alloway, T. P., Gathercole, S. E., Willis, C., & Adams, A. (2004). A structural analysis of working memory and related cognitive skills in

young children. Journal of Experimental Child Psychology, 87, 85–106.American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, fifth edition. Washington, DC: American

Psychiatric Association.Anderson, J. (1976). Cognitive psychology. New York: Freeman.Archibald, L. M. D., & Gathercole, S. E. (2006a). Short-term and working memory in specific language impairment. International Journal of

Language & Communication Disorders, 41(6), 675–693.Archibald, L. M. D., & Gathercole, S. E. (2006b). Short-term memory and working memory in specific language impairment. In T. P.

Alloway & S. E. Gathercole (Eds.), Working memory and neurodevelopmental disorders (pp. 139–160). New York, NY: Psychology Press.Baddeley, A. D. (1986). Working memory. London: Oxford University Press.Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4, 417–423.Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63, 1–29.Baddeley, A. D., Gathercole, S. E., & Papagno, C. (1998). The phonological loop as a language learning device. Psychological Review, 105, 158–

173.Baddeley, A. D., & Logie, R. H. (1999). The multiple-component model. In A. Miyake & P. Shah (Eds.), Models of working memory:

Mechanisms of active maintenance and executive control (pp. 28–61). Cambridge, UK: Cambridge University Press.Badian, N. A. (1983). Arithmetic and nonverbal learning. In H. R. Myklebust (Ed.), Progress in learning disabilities (Vol. 5, pp. 235–264). New

York: Grune and Stratton.Belleville, S., Ménard, É., Mottron, L., & Ménard, M. (2006). Working memory in autism. In T. P. Alloway & S. E. Gathercole (Eds.),

Working memory and neurodevelopmental disorders (pp. 213–238). New York, NY: Psychology Press.Booth, J. N., Boyle, J. M. E., & Kelly, S. W. (2010). Do tasks make a difference? Accounting for heterogeneity of performance of children with

reading difficulties on tasks of executive function: Findings from a meta-analysis. British Journal of Developmental Psychology. Special Issue:Developmental Disorders of Language and Literacy, 28(1), 133–176.

Carretti, B., Borella, E., Cornoldi, C., & De Beni, R. (2009). Role of working memory in explaining the performance of individuals withspecific reading comprehension difficulties: A meta-analysis. Learning and Individual Differences, 19(2), 245–251.

Cheung, C. H. M., Wood, A. C., Paloyelis, Y., Arias-Vasquez, A., Buitelaar, J. K., Franke, B., . . . Kuntsi, J. (2012). Aetiology for thecovariation between combined type ADHD and reading difficulties in a family study: The role of IQ. Journal of Child Psychology andPsychiatry, 53(8), 864–873.

Chuderski, A. (2013). When are fluid intelligence and working memory isomorphic and when are they not? Intelligence, 41(4), 244–262.Cohen, J. (1988) Statistical power analysis for the behavioral sciences. New York: Academic Press.Crinella, F., & Yu, J. (2000). Brain mechanisms and intelligence: Psychometric g and executive function. Intelligence, 27, 299–327.Daneman, M., & Carpenter P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal

Behavior, 19, 450–466.David, C. V. (2012). Working memory deficits in math learning difficulties: A meta-analysis. International Journal of Developmental Disabilities,

58(2), 67–84.Davidovitch, M., Hemo, B., Manning-Courtney, P., & Fombonne, E. (2013). Prevalence and incidence of autism spectrum disorder in an

Israeli population. Journal of Autism and Developmental Disorders, 43(4), 785–793.Dehaene, S., & Cohen, L. (1997). Cerebral pathways for calculation: Double disassociation between rote verbal and quantitative knowledge of

arithmetic. Cortex, 33, 2219–2250.Di Martino, A., Ross, K., Uddin, L. Q., Sklar, A. B., Castellanos, F. X., & Milham, M. P. (2009). Functional brain correlates of social and

nonsocial processes in autism spectrum disorders: An activation likelihood estimation meta-analysis. Biological Psychiatry, 65(1), 63–74.Duncan, J., Schramm, M., Thompson, R., & Dumontheil, I. (2012). Task rules, working memory, and fluid intelligence. Psychonomic Bulletin

307

& Review, 19(5), 864–870.Gathercole, S. E., & Baddeley, A. D. (1993). Working memory and language. Hove, UK: Erlbaum.Gathercole, S. E., & Pickering, S. J. (2000). Working memory deficits in children with low achievements in the national curriculum at 7 years

of age. British Journal of Educational Psychology, 70(2), 177–194.Gathercole, S. E., Pickering, S. J., Ambridge, B., & Wearing, H. (2004). The structure of working memory from 4 to 15 years of age.

Developmental Psychology, 40, 177–190.Geary, D. C. (1993). Mathematical disabilities: Cognition, neuropsychological and genetic components. Psychological Bulletin, 114, 345–362.Geary, D. C. (2013). Learning disabilities in mathematics: Recent advances. In H. L. Swanson, K. Harris, & S. Graham (Eds.), Handbook of

learning disabilities (2nd ed., pp. 239–255). New York: Guilford.Glass, G. V., McGraw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.Goldman-Rakic, P. S. (1995). Architecture of the prefrontal cortex and the central executive. Annals of the New York Academy of Sciences, 769,

71–83.Gonçalves, Ó. F., Pinheiro, A. P., Sampaio, A., Sousa, N., Férnandez, M., & Henriques, M. (2011). Autobiographical narratives in Williams

syndrome: Structural, process and content dimensions. Journal of Developmental and Physical Disabilities, 23(4), 289–302.Haworth, C. M. A., Kovas, Y., Harlaar, N., Hayiou-Thomas, M., Petrill, S. A., Dale, P. S., & Plomin, R. (2009). Generalist genes and

learning disabilities: A multivariate genetic analysis of low performance in reading, mathematics, language and general cognitive ability in asample of 8000 12-year-old twins. Journal of Child Psychology and Psychiatry, 50(10), 1318–1325.

Henry, L. A., Messer, D. J., & Nash, G. (2012). Executive functioning in children with specific language impairment. Journal of ChildPsychology and Psychiatry, 53(1), 37–45.

Hoskyn, M., & Swanson, H. L. (2000). Cognitive processing of low achievers and children with reading disabilities: A selective meta-analyticreview of the published literature. School Psychology Review, 29, 102–119.

Hulme, C. C., & Snowling, M. J. (2009). Developmental disorders of language and cognition. West Sussex, UK: Wiley.Johnson, E. S., Humphrey, M., Mellard, D. F., Woods, K., & Swanson, H. L. (2010). Cognitive processing deficits and students with specific

learning disabilities: A selective meta-analysis of the literature. Learning Disability Quarterly, 33(1), 3–18.Kane, M. J., Conway, A. R., Hambrick, D. Z., & Engle, R. W. (2007). Variation in working memory capacity as variation in executive

attention and control. In A. Conway, C. Jarrold, M. J. Kanem, A. Miyake, & J. Towse (Eds.), Variation in working memory (pp. 21–48). NewYork: Oxford University Press.

Klingberg, T. (2012). Is working memory capacity fixed? Journal of Applied Research in Memory and Cognition, 1(3), 194–196.Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is (little more than) working memory capacity?! Intelligence, 14, 389–433.Lifsh*tz, H., Shtein, S., Weiss, I., & Svisrsky, N. (2011a). Explicit memory among individuals with mild and moderate intellectual disability:

Educational implications. European Journal of Special Needs Education, 26(1), 113–124.Lifsh*tz, H., Shtein, S., Weiss, I., & Vakil, E. (2011b). Meta-analysis of explicit memory studies in populations with intellectual disability.

European Journal of Special Needs Education, 26(1), 93–111.Locascio, G., Mahone, E. M., Eason, S. H., & Cutting, L. E. (2010). Executive dysfunction among children with reading comprehension

deficits. Journal of Learning Disabilities, 43(5), 441–454.Logie, R. H. (1986). Visuo-spatial processing in working memory. Quarterly Journal of Experimental Psychology, 38A, 229–247.Mammarella, I. C., Lucangeli, D., & Cornoldi, C. (2010). Spatial working memory and arithmetic deficits in children with nonverbal learning

difficulties. Journal of Learning Disabilities, 43(5), 455–468.Martinussen, R., Hayden, J., Hogg-Johnson, S., & Tannock, R. (2005). A meta-analysis of working memory impairments in children with

attention-deficit/hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 44(4), 377–384.McCandliss, B. D., & Noble, K. G. (2003). The development of reading impairment: A cognitive neuroscience model. Mental Retardation and

Developmental Disabilities Research Reviews, 9(3), 196–204.McGrath, L. M., Pennington, B. F., Shanahan, M. A., Santerre-Lemmon, L. E., Barnard, H. D., Willcutt, E. G., . . . Olson, R. K. (2011). A

multiple deficit model of reading disability and attention-deficit/hyperactivity disorder: Searching for shared cognitive deficits. Journal ofChild Psychology and Psychiatry, 52(5), 547–557.

Menghini, D., Finzi, A., Carlesimo, G. A., & Vicari, S. (2011). Working memory impairment in children with developmental dyslexia: Is itjust a phonological deficit? Developmental Neuropsychology, 36(2), 199–213.

Melby-Lervåg, M., & Hulme, C. (2013). Is working memory training effective? A meta-analytic review. Developmental Psychology, 49(2), 270–291.

Melby-Lervåg, M., Lyster, S. H., & Hulme, C. (2012). Phonological skills and their role in learning to read: A meta-analytic review.Psychological Bulletin, 138(2), 322–352.

Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., & Howerter, A. (2000). The unity and diversity of executive functions and their

308

contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49–100.Montgomery, J. W., Magimairaj, B. M., & Finney, M. C. (2010). Working memory and specific language impairment: An update on the

relation and perspectives on assessment and treatment. American Journal of Speech-Language Pathology, 19(1), 78–94.Nation, K., Adams, J. W., Bowyer-Crane, C. A., & Snowling, M. J. (1999). Working memory deficits in poor comprehenders reflect

underlying language impairments. Journal of Experimental Child Psychology, 73, 139–158.National Institute of Neurological Disorders and Stroke. (12 May 2010). Retrieved from http://www.ninds.nih.gov/disorders/dyslexia/dyslexi‐

a.htm (accessed November 1, 2012).Nithart, C., Demont, E., Majerus, S., Leybaert, J., Poncelet, M., & Metz-Lutz, M. (2009). Reading disabilities in SLI and dyslexia result from

distinct phonological impairments. Developmental Neuropsychology, 34(3), 296–311.Nittrouer, S., & Pennington, B. (2010). New approaches to the study of childhood language disorders. Current Directions in Psychological Science,

19(5), 308–313.Norman, D., & Shallice, T. (1986). Attention to action: Willed and automatic control of behavior. In R. J. Davidson, G. E. Schwartz, & D. E.

Shapiro (Eds.) Consciousness and self-regulation: Advances in research and theory (Vol. 4, pp. 1–18). New York: Plenum.Ozonoff, S., & Strayer, D. L. (2001). Further evidence of intact working memory in autism. Journal of Autism and Developmental Disorders,

31(3), 257–263.Pennington, B. F., McGrath, L. M., Rosenberg, J., Barnard, H., Smith, S. D., Willcutt, E. G., . . . Olson, R. K. (2009). Gene × environment

interactions in reading disability and attention-deficit/hyperactivity disorder. Developmental Psychology, 45(1), 77–89.Pennington, B. F., Santerre-Lemmon, L., Rosenberg, J., MacDonald, B., Boada, R., Friend, A., . . . Olson, R. K. (2012). Individual prediction

of dyslexia by single versus multiple deficit models. Journal of Abnormal Psychology, 121(1), 212–224.Pimperton, H., & Nation, K. (2010). Suppressing irrelevant information from working memory: Evidence for domain-specific deficits in poor

comprehenders. Journal of Memory and Language, 62(4), 380–391.Reigosa-Crespo, V., Valdés-Sosa, M., Butterworth, B., Estévez, N., Rodríguez, M., Santos, E., . . . Lage, A. (2012). Basic numerical capacities

and prevalence of developmental dyscalculia: The Havana survey. Developmental Psychology, 48(1), 123–135.Ricketts, J. (2011). Research review: Reading comprehension in developmental disorders of language and communication. Journal of Child

Psychology and Psychiatry, 52(11), 1111–1123.Rowe, M., & Mervis, C. B. (2007). Working memory in Williams syndrome. In T. P. Alloway & S. E. Gathercole (Eds.), Working memory and

neurodevelopmental disorders (pp. 267–294). New York: Psychology Press.Russell, G., Rodgers, L. R., Ukoumunne, O. C., & Ford, T. (2014). Prevalence of parent-reported ASD and ADHD in the UK: Findings from

the millennium cohort study. Journal of Autism and Developmental Disorders, 44(1), 31–40.Scarborough, H. S. (1998). Early identification of children at risk for reading disabilities: Phonological awareness and some other promising

predictors. In B Shapiro, P. Accardo, & A. Capute (Eds.), Specific reading disability: A view of the spectrum (pp. 75–119). Timonium, MD:York Press.

Siegel, L. S. (1993). Phonological processing deficits as a basis for reading disabilities. Developmental Review, 13, 246–257.Snowling, M. J., & Hulme, C. (2012). Annual research review: The nature and classification of reading disorders—a commentary on proposals

for DSM-5. Journal of Child Psychology and Psychiatry, 53(5), 593–607.Swanson, H. L. (1988). Learning disabled children’s problem solving. Intelligence, 12, 261–278.Swanson, H. L. (1993). An information processing analysis of learning disabled children’s problem solving. American Educational Research

Journal, 30, 861–893.Swanson, H. L. (1995). S-Cognitive Processing Test (S-CPT): A dynamic assessment measure (p. 122). Austin, TX: PRO-ED.Swanson, H. L. (2008). Working memory and intelligence in children: What develops? Journal of Educational Psychology, 100, 581–602.Swanson, H. L. (2011). Dynamic testing, working memory, and reading comprehension growth in children with reading disabilities. Journal of

Learning Disabilities, 44(4), 358–371.Swanson, H. L., & Alexander, J. (1997). Cognitive processes as predictors of word recognition and reading comprehension in learning disabled

and skilled readers: Revisiting the specificity hypothesis. Journal of Educational Psychology, 89, 128–158.Swanson, H. L., & Alloway, T. P. (2012). Working memory, learning, and academic achievement. In K. Harris, S. Graham, & T. Urdan (Eds.),

APA educational psychology handbook (Vol. 1, pp. 327–366). Washington, DC: American Psychological Association.Swanson, H. L., & Hsieh, C. (2009). Reading disabilities in adults: A selective meta-analysis of the literature. Review of Educational Research,

79(4), 1362–1390.Swanson, H. L., & Jerman, O. (2006). Math disabilities: A selective meta-analysis of the literature. Review of Educational Research, 76, 249–

274.Swanson, H. L., Jerman, O., & Zheng, X. (2008). Growth in working memory and mathematical problem solving in children at risk and not at

risk for serious math difficulties. Journal of Educational Psychology, 100(2), 343–379.

309

http://www.ninds.nih.gov/disorders/dyslexia/dyslexia.htm

Swanson, H. L., Jerman, O., & Zheng, X. (2009a). Math disabilities and reading disabilities: Can they be separated? Journal of PsychoeducationalAssessment, 27(3), 175–196.

Swanson, H. L., Zheng, X., & Jerman, O. (2009b). Working memory, short-term memory and reading disabilities: A selective meta-analysis ofthe literature. Journal of Learning Disabilities, 42, 260–287.

Tillman, C., Eninger, L., Forssman, L., & Bohlin, G. (2011). The relation between working memory components and ADHD symptoms froma developmental perspective. Developmental Neuropsychology, 36(2), 181–198.

Vugs, B., Cuperus, J., Hendriks, M., & Verhoeven, L. (2013). Visuospatial working memory in specific language impairment: A meta-analysis.Research in Developmental Disabilities, 34(9), 2586–2597.

Willcutt, E. G., Doyle, A. E., Nigg, J. T., Faraone, S. V., & Pennington, B. F. (2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57(11), 1336–1346.

Wilson, P. H., Ruddock, S., Smits-Engelsman, B., Polatajko, H., & Blank, R. (2013). Understanding performance deficits in developmentalcoordination disorder: A meta-analysis of recent research. Developmental Medicine & Child Neurology, 55(3), 217–228.

Yeates, K. O., Ris, M. D., Taylor, H. G., & Pennington, B. F. (2010). Pediatric neuropsychology: Research, theory, and practice (2nd ed.). NewYork: Guilford.

310

11Personal Capability Beliefs

ELLEN L. USHER

University of Kentucky “Beliefs are rules for action.” In 1885, William James, father of American psychology, echoed these five wordsfrom his friend and fellow philosopher, Charles Sanders Peirce, who considered them to be the ideologicalcornerstone of pragmatism. This chapter adopts the same premise, put forth by numerous others, that thebeliefs individuals hold are excellent indicators of what people choose, what they perceive and experience, andultimately what they do (Dewey, 1933; Rokeach, 1968). The chapter focuses on the development andfunctional value of personal capability beliefs—primarily learners’ beliefs—in educational settings (see Chapter30, this volume, for a thoughtful review of research and theory on teachers’ beliefs).

Despite the emphasis that James (1892/2001) placed on the self as a key determinant of thought andaction, the dominant psychological theories of the first half of the twentieth century relegated self as agent tothe sidelines. To the psychoanalysts, human functioning was explained as the result of concealed innerimpulses. To the behaviorists, internal influences were banished as causal agents altogether; the sole focus wason environmental contingencies as both causes and reinforcers of behavior. Not until the second half of thetwentieth century was self reintroduced as an important determinant of individual functioning, growth, andhealth, this time by humanistic psychologists (e.g., Maslow, 1968; Rogers, 1947) and social cognitive theorists(e.g., Bandura, 1986).

Study of the self-system once again found favor in the science of psychology and, in turn, became of interestto educational researchers and practitioners. This work began to confirm what James and Peirce conjectured100 years prior: learners’ beliefs about their own capabilities are related to cognitive, affective, and behavioraloutcomes (see Pajares & Schunk, 2002, for a historical review). In fact, in the first edition of this Handbook ofEducational Psychology, Graham and Weiner (1996) asserted that the study of the self “reflects what isprobably the main new direction in the field of motivation” and may soon dominate the field altogether (p.77). Abundant research has been conducted on self-processes over the past decade. This explains why twochapters were devoted to competence and self-beliefs in the 2006 edition of this handbook (i.e., Roeser, Peck,& Nasir, 2006; Schunk & Zimmerman, 2006) and one to self-concept in a recent related handbook (i.e.,Marsh, Xu, & Martin, 2012).

The broad aim of this chapter is to bring readers up to date with the research on personal capability beliefsand to chart new pathways for future understanding. The chapter begins by addressing terminology anddefinitions useful in the study of personal capability beliefs in educational settings; it then offers a descriptionand overview of several such beliefs, situating each in its theoretical home. Key research findings are discussed;these demonstrate how beliefs are related to behavioral and affective variables and how they develop and

311

change. This section also reviews notable individual and group differences. The next section describesprominent methodological approaches used to investigate capability beliefs; this includes suggestions for waysin which capability beliefs can be operationalized, assessed, and modeled. The chapter concludes withsuggestions for future research on personal capability beliefs.

312

Personal Capability Beliefs: Terminology and Definitions

Of the studies focused on learner motivation that have filled education and psychology journals in recentdecades, many have included some measure of learners’ beliefs about their capabilities. A discerning readerwill be quick to notice that the terminology researchers use to describe personal capability beliefs is diverse, ifnot downright confusing. Terms such as self-concept of ability, self-efficacy, academic competence beliefs, self-esteem, ability conceptions, expectancy for success, outcome expectations, and implicit theory of intelligence populateeducational and developmental psychology journals. As the research on self-beliefs in academic settings hasgrown, new terms have emerged. With respect to personal capability beliefs as with other motivationconstructs, “similar terminology is being used to mark varied constructs or . . . the same construct is beingreferenced by different language” (Murphy & Alexander, 2000, p. 5). These terms sometimes signifyconfusion on the part of researchers; however, they also likely point to the intricate system of beliefs thathumans hold, each denoting a slightly different self-view. The combined influence of these self-beliefs, alongwith other factors both internal and external to the individual, makes learners more or less motivated to act.

To gain a better understanding of how learners’ beliefs guide action and how those beliefs develop, onemust look beyond the confusing semantic signposts to two critical questions. First, what is the theoreticalhome of the construct under investigation? This, of course, requires some background reading and knowledgeof theory. Second, how are the beliefs in question measured and operationalized? For example, if the beliefsignpost in a study reads “self-efficacy,” the reader should look to see that self-efficacy has been situatedwithin social cognitive theory and assessed in a manner consistent with this theory. If it has not, the criticalreader might ask, “What, then, has been measured?” These questions are addressed in more detail later; first, itis necessary to define terms to be used and review some conceptual distinctions in research on personalcapability beliefs.

Ability Versus Capability

The general term, personal capability beliefs, here refers to the beliefs learners hold about their potential to carryout the various learning tasks before them. Though small, a distinction exists between the meaning of “ability”and “capability.” Ability, which comes from the Latin habilis, refers to a skill or competence in doing orperforming that has already been acquired. Capability refers to the potential to develop a skill or competencyin the future or to perform a given task under varying conditions. I judge myself as able to write; however, Imay not judge myself as capable of writing a novel this summer. In other words, capability depends in part onone’s ability but also on other cognitive and motivational variables, such as effort and persistence, and on one’sassessment of contextual demands. A child might be able to add two-digit numbers but incapable of adding 50such numbers within 30 minutes. A person’s beliefs therefore reflect both actuality and potentiality (Maslow,1968). Students’ beliefs about what they can do presently, what they can do under varying circ*mstances, andwhat they can learn to do in the future might differ, and each can be important to investigate.

Competence Versus Control Beliefs

With the paradigm shift from behaviorism, diverse frameworks accorded self-beliefs a prominent role inpredicting and explaining human motivation and behavior. These frameworks, some of which are described

313

below, emphasized two types of self-beliefs, which Schunk and Zimmerman (2006) classified as competenceand control beliefs. As noted above, competence beliefs refer to individuals’ perceptions of their ability orcapability to meet task and situational demands. They answer the general question, “Can I perform task X?”and are based on individuals’ evaluations of the skills they bring to a given situation or challenge. Competencebeliefs are considered means- or process-related beliefs because they refer to the means by which individualsreach desired ends. Control beliefs, on the other hand, refer to individuals’ perceptions of outcomes or endsthat their actions will bring about. Control beliefs answer the general question, “If I perform X, will I achieveY?” In this sense, control beliefs do not refer to one’s perceived abilities or capabilities but to beliefs that one’sactions will bring about desired outcomes.

Competence and control beliefs are independent predictors of human motivation and behavior (Bandura,1997). A young student might believe herself competent as a reader but might not feel that she has muchcontrol over the grade she will earn on the comprehension test she must take when she finishes her book.Even learners who feel competent may not expect positive results if they do not believe they can control theoutcomes of their efforts. Factors external to the learner, such as a teacher’s inflexible assessment standards orthe competitive nature of the class, may undermine a learner’s sense of control. Conversely, a learner mightfeel that his grade in his mathematics class is within his control, but if he lacks a sense of competence, he maynot achieve desired ends. Both a belief in one’s own competence and a sense that outcomes are within one’scontrol are critical motivating factors. Readers interested in a more detailed account of how competence andcontrol beliefs have been theoretically situated and empirically examined are referred to Schunk andZimmerman (2006).

Relative Versus Absolute Criteria

Beliefs can be formed with relative and absolute criteria in mind. I may consider myself an accomplishedpianist when, in the privacy of my own home, I master a page of music by learning to play it fluently (anabsolute, self-set standard). If I were to invite a concert pianist to play the same piece of music and then judgemy own abilities relative to his (a comparative standard), my self-perception would undoubtedly wane. A labelof proficiency might be conferred by my piano teacher if I could meet her standard of 100% accuracy (anabsolute, externally-set standard). In some situations, clear performance criteria exist by which learners cangauge their progress. In many situations, however, no absolute standards exist. In such circ*mstances peoplemay judge their capabilities by social comparative means or by monitoring their progress toward self-set goals(regardless of the external criteria available). A thorough understanding of self-perceptions requiresconsideration of the relative versus absolute means by which individuals judge what they can do. Thestandards by which a given learner measures herself depend on numerous factors such as the academicdomain, the nature of the task, personal characteristics, age of the learner, and the learning climate.

Much of the research described herein does not distinguish between ability versus capability, competenceand control, or relative and absolute measures of perceived competence. But these distinctions are not simplysemantic; the words researchers choose often reflect differences in the nature of the underlying beliefs towhich they point.

314

315

Personal Capability Beliefs: Diverse Theoretical Perspectives

Types of personal capability beliefs are defined in different theoretical frameworks. In educational settings,much research conducted within the past decade has targeted kindergarten through Grade 12 populations.

Self-Concept Theory

Self-concept has historically referred to a global self-view that comprises the many self-perceptions oneaccumulates across life experiences (Hattie, 1992; James, 1892/2001; Wylie, 1989). One’s self-concept iscommonly conceptualized as both hierarchical and multidimensional in nature (Marsh & Craven, 2006). Anindividual’s self-perceptions in specific domains of learning (e.g., mathematics, language) combine to form anacademic self-concept, which, alongside self-concepts developed in other spheres of functioning (e.g., physical,social), form the individual’s overall composite self-concept (Marsh, Trautwein, Lüdtke, & Köller, 2008). Inschool-based research, researchers tend to measure self-concept as students’ cognitive self-assessment of theirscholastic ability or competence (e.g., Harter, 2012). Researchers who have assessed students’ domain-specificself-concept have reported that self-concept in one domain may be unrelated to self-concept in another(Marsh et al., 2012). For example, students’ perceptions about their mathematics competence may bedissimilar to their views of their reading competence.

A wealth of research on self-concept over the past three decades has targeted school-aged populations (for acomprehensive review of self-concept research in education, see Marsh et al., 2012). Domain-generalmeasures of self-concept appear only weakly related (and sometimes unrelated) to academic performance andmotivation (Baumeister, Campbell, Krueger, & Vohs, 2003; Hattie, 1992). Likewise, self-esteem, whichrefers to an overall feeling of one’s worth as a person, is similarly unrelated to performance. When assessed ata domain-specific level, however, one’s self-concept of ability is a good predictor of these same outcomes(Marsh & Craven, 2006).

In the past decade, researchers have examined two explanatory views regarding the direction of influencebetween academic self-concept and academic achievement. The self-enhancement view posits that when theirself-concept is robust, learners will experience increased academic achievement. In this view, enhancing self-concept in turn increases achievement. The skill development view suggests that, by building requisite skillsand performance, students will develop a positive self-concept. Evidence from reciprocal effects models hasshown a bidirectional pattern of influence: academic self-concept and academic achievement are mutuallyreinforcing (Marsh & O’Mara, 2008). The implication of this finding is that teachers and parents shouldattend to the development of students’ academic skills and academic self-views.

Social Cognitive Theory

From a social cognitive theoretical perspective, personal, behavioral, and environmental factors influence eachother in a dynamic process of triadic reciprocality (Bandura, 1986). Personal factors include the beliefsindividuals hold about what they can do, or self-efficacy. Self-efficacy influences and is influenced by actualperformances. Bandura (1997) contended that individuals with robust beliefs of personal efficacy are moremotivated, tend to engage tasks in novel ways, take more risks, and persist when they encounter challenges.

Self-efficacy is distinct from other personal capability beliefs, such as self-concept. Academic self-efficacy

316

refers to one’s perceived capability to accomplish given academic tasks and can be thought of in terms of cando statements (e.g., “I can solve algebraic equations with one variable”). Self-concept of ability, on the otherhand, refers to a broader self-evaluation of competence within an academic domain (e.g., “Work inmathematics classes is easy for me”). Although both types of self-judgments are related to competence, theyare typically assessed at different levels of specificity (Bong & Skaalvik, 2003). Self-efficacy questions revealhow confident a learner feels that she can accomplish given tasks or succeed at academic activities. Self-concept questions reveal how learners evaluate their own competence in a particular academic domain.

Students’ efficacy judgments are also related to, but distinct from, their expectations about the outcomesthat their actions will bring about. Outcome expectations refer to a belief that a certain outcome (whetherphysical, social, or self-evaluative) will result from one’s actions (Bandura, 1997). The academic outcomes thatstudents expect are largely dependent on beliefs in their personal efficacy to perform and learn the skillsrequired to bring about those outcomes. For example, a student must earn a particular grade point average(performance) to receive an award (outcome). Because the award is directly contingent on her performance,her efficacy beliefs relative to her performance will likely be consistent with her expectations of receiving theaward. As this example illustrates, self-efficacy and outcome expectations are often correlated, particularly inactivities where outcomes depend directly on the quality of one’s performance. Some exceptions to this patterncan be noted, however. When outcomes are not directly dependent on one’s performance, such as inenvironments rife with prejudice or bias, efficacy beliefs and outcome expectations may operate independently(Bandura, 1997). If a student faces a prejudiced or oppressive teacher, he may not expect a favorable outcome,even if he believes himself academically capable. A student with high self-efficacy for physics may neverthelesshold a reserved expectation about the outcome of her physics test because of her instructor’s harsh gradingpractices. The less personal control a learner has over the outcomes of her actions, the less correlated herefficacy beliefs will be to her outcome expectations. Both self-efficacy and outcomes expectations can beimportant aspects of learners’ motivation and performance.

The influence of personal efficacy beliefs on subsequent academic achievement has been consistentlydocumented (for reviews, see Klassen & Usher, 2010; Pajares, 1997). Students with higher self-efficacy forparticular tasks attain higher levels of success (e.g., higher scores). Self-efficacy has also been shown toinfluence self-regulatory processes such as goal setting and self-monitoring (Zimmerman & Cleary, 2006).Students with higher self-efficacy for self-regulation organize their work and manage their time moreeffectively (Klassen et al., 2009); they also report lower anxiety, higher mastery goals, and higher achievement(Usher & Pajares, 2008a). As was the case with self-concept, reciprocal effects models have shown aconsistent bidirectional relationship between self-efficacy and achievement. Williams and Williams (2010)found support for a reciprocal relationship between high-school students’ mathematics self-efficacy andachievement in 26 national contexts.

Self-Determination Theory

From the perspective of self-determination theory, individuals are said to be self-determining (i.e., intrinsicallymotivated) in an activity to the extent that they are meeting three primary psychological needs: the need tofeel competent, autonomous, and related to others (Deci & Ryan, 1985). The need for competence, or havingthe skills necessary to succeed in a given pursuit, is considered a partial source of intrinsic motivation.

317

Students feel a sense of competence when they master their environment. They feel a sense of autonomy whenthey fully endorse or get behind their own actions. To become intrinsically motivated, “people must not onlyexperience perceived competence (or self-efficacy), they must also experience their behavior to be self-determined” or “accompanied by a sense of autonomy” (Ryan & Deci, 2000, p. 58). In this respect, self-determination theory, like social cognitive theory, emphasizes the importance of students’ need for a sense ofcompetence and control in learning activities.

The bulk of the research conducted within the self-determination framework and applied to educationalsettings has focused on the degree to which students feel and are permitted to be autonomous in the learningprocess (see Reeve, 2004). However, some scholars have included some measure of perceived competence. Forexample, De Naeghel, Van Keer, Vansteenkiste, and Rosseel (2012) examined the influence of autonomousmotivation, controlled motivation, and self-concept on elementary-school students’ engagement, frequency,and comprehension in reading. Reading self-concept was related to all three reading outcomes; autonomousmotivation was only related to reading frequency. Similarly, competence beliefs, relatedness, and perceivedautonomy were all three related to changes in early adolescents’ behavioral and emotional engagement anddisaffection (Skinner, Furrer, Marchand, & Kindermann, 2008).

Expectancy-Value Theory

Expectancy-value theory posits that academic motivation is the result of learners’ expectations of success andthe degree to which they value academic activities. Expectancy refers to a learner’s belief that he or she will dowell on a given academic task in the future. Value offers information about why a learner engages in an activity(e.g., intrinsic value, attainment value, utility value, perceived cost; see Chapter 7, this volume, for a moredetailed description of the value component of this theory). The theory holds that students’ expectancy beliefsand the degree to which they value achievement-related tasks are related to school performance, academicchoices, and persistence (Eccles & Wigfield, 2002). Unlike the expectancy-value theoretical frameworks putforth by earlier scholars, contemporary expectancy-value frameworks, based largely on research conducted indiverse school settings, offer a broader perspective and emphasize how the social and cultural environmentinfluences the development of students’ motivation (Wigfield, Tonks, & Klauda, 2009). Rather than focusingnarrowly on whether a student expects to perform successfully, this broader view defines expectancy in termsof students’ beliefs about how well they will perform on academic tasks. These beliefs are conceptually distinctfrom students’ competence or ability beliefs, which refer to judgments of one’s current ability (Wigfield &Eccles, 2002). Both expectancies and ability beliefs jointly contribute to student motivation and learning.Evidence from research across a variety of academic domains has shown that students who hold higherexpectations for success and more positive ability beliefs experience more school success and show greaterpersistence for academic tasks (Wigfield et al., 2009).

A sizeable body of research within the expectancy-value framework has been devoted to understanding howstudents’ subject-specific ability beliefs and values develop and change throughout childhood and adolescence(e.g., Fredricks & Eccles, 2002; Gniewosz, Eccles, & Noack, 2012; Jacobs, Lanza, Osgood, Eccles, &Wigfield, 2002). Findings generally show that students report lower self-concepts of ability as they progressthrough school, particularly during school transitions (e.g., elementary to middle school). Eccles andcolleagues have suggested that systematic differences in school- and classroom-level structures, such as an

318

increasing orientation toward performance and relative ability in secondary schools, are at odds with thedevelopmental needs of learners. However, the relationship between ability beliefs and performance outcomesgrows stronger as students move from the elementary- to middle-school years, indicating that greaterattention should be paid to constructing learning environments that match the developmental needs ofstudents whose beliefs are still nascent (see Eccles & Roeser, 2011).

Implicit Theory of Intelligence

Researchers have contended that the view of intelligence that students hold—whether as a fixed trait or anincremental quantity that can change with effort—influences how they approach their academic work andrespond to setbacks. This mindset has been referred to as a learner’s implicit theory of intelligence (Dweck,2006). Students are typically asked to respond to general statements about their intelligence (e.g., “You have acertain amount of intelligence, and you really can’t do much to change it”; Dweck, 1999). Beliefs about thefixed or malleable nature of intelligence are then compared with students’ motivation and academic outcomes.Blackwell, Trzesniewski, and Dweck (2007) found that students who held an incremental theory ofintelligence (i.e., that one’s intelligence can improve with effort) at the beginning of junior high schoolreported more positive motivation beliefs (e.g., a learning goal orientation, belief that effort was necessary forimprovement) and achieved higher scores in mathematics two years later than did students who held a morefixed view.

Assessing one’s implicit theory of intelligence in a domain-general manner may not evoke the samepsychological response as assessing one’s domain-specific views of ability. For example, a learner may view heroverall intelligence as relatively fixed but believe herself capable of improving as a writer. Few researchers haveinvestigated implicit theories of ability in different academic domains. For example, Chen and Usher (2013)found that adolescents who held a more malleable view of science ability were more likely to draw frommultiple sources of information when forming their science self-efficacy than were students who viewedscience ability as fixed. These and other studies suggest that one’s implicit theories of ability and intelligencemay be considered as framing belief systems that influence how learners attend to and select information,experience phenomena, and come to understand themselves (e.g., Mangels, Butterfield, Lamb, Good, &Dweck, 2006).

Personal Capability Beliefs Related to Self-Regulation

Researchers have increasingly noted the importance of the underlying self-regulatory skillset that guides thelearning process in different areas. To be successful, students must plan, organize, implement study strategies,manage their time, set goals, and monitor their progress. Just as students form beliefs about their capabilitiesto meet subject-area objectives (e.g., to understand the symbolism in a Hemingway novel), they also developbeliefs about their self-regulatory capabilities (e.g., to maintain focus while reading Hemingway). Knowledgealone does not ensure that a learner puts skills to effective use; learners must also possess a belief in theircapability to use self-regulatory skills effectively (Zimmerman, 2011; Zimmerman & Cleary, 2006).

Just as they form beliefs about their academic competencies, learners hold beliefs about their capability touse appropriate self-regulatory strategies and to exercise self-control (Schunk & Usher, 2011). Students’ self-efficacy for self-regulated learning has been shown to predict academic motivation, achievement, successful

319

strategy use, and school completion (Caprara et al., 2008; Usher & Pajares, 2008a). For example, junior high-school students who reported higher self-efficacy for self-regulated learning earned higher achievement scores,even when prior achievement, gender, socioeconomic status, personality, and intelligence—covariates thathave been shown to be closely related to achievement—were controlled (Zuffiano et al., 2013). Students whobelieved themselves capable of regulating their academic work also achieved more academically. Researcherswho have investigated students’ actual self-disciplined behavior have similarly found that more disciplinedstudents reap an achievement advantage at school, and that girls are more self-disciplined than boys(Duckworth & Seligman, 2006). Linking personal capability beliefs about self-regulation to actual self-regulatory behavior (e.g., third-person reports of self-regulation, -control, -discipline, grit; see Duckworth &Kern, 2011) could shed light on the benefit of such beliefs.

320

Relationship Between Personal Capability Beliefs and Other Outcomes

In numerous studies involving tens of thousands of students of different ages, researchers have shown that thebeliefs students hold about their own capabilities (i.e., academic self-concept, self-efficacy, perceivedcompetence) predict student achievement on numerous measures of academic competence ranging fromstandardized tests, school-assigned grades, subject-area tests, and other school assignments (e.g., Klassen &Usher, 2010; Marsh & Craven, 2006). Personal capability beliefs often remain predictive of achievement evenwhen past achievement is controlled, which attests to the powerful influence of beliefs on future performances.Students who doubt their academic capabilities perform less well at school and are at greater risk for academicdifficulty.

Personal capability beliefs have also been shown to predict choice behavior such as course selection, field ofstudy in high school or college, and career choice. Data from a large-scale longitudinal study that followedstudents from 805 high schools in England and Germany showed that high-school students’ self-concept ofability in mathematics and English was related to their university entry and choice of university major (Parkeret al., 2012). Students with higher mathematics self-concept were more likely to pursue mathematics study,whereas those with higher self-concept in English were more likely to enter more verbal majors such ashumanities. Meta-analysis results from 45 studies conducted primarily with college students have similarlyshown that the influence of environmental supports and barriers on students’ career goals depend in part onstudents’ level of self-efficacy (Sheu et al., 2010). Students with higher self-efficacy in a given domain alsoreport higher outcome expectations and greater interest in their domain-related career goals.

Learners’ cognitive and metacognitive experiences are also influenced by their beliefs about their owncapabilities. Those with a high sense of personal efficacy foresee their own success. They rehearse successfulscenarios and anticipate favorable outcomes (Bandura, 1997). These rehearsals make students more likely toimplement successful strategies during their academic performances (Zimmerman, 2011). Students whobelieve in their academic capabilities are more likely to handle setbacks and challenges effectively. They reportpursuing their academic activities to improve their own skills and learning. Conversely, learners beset withself-doubt are more likely to engage in self-handicapping strategies such as procrastination (Klassen et al.,2009). They dwell on the possibility of their own failure and imagine worst-case scenarios. They employpreventive strategies to avoid appearing incompetent (Usher & Pajares, 2008a).

Capability beliefs, which are cognitive judgments, can affect and be affected by learners’ emotion andmotivation (see Chapters 6 and 7). For example, Wolters, Fan, and Daugherty (2013) found that learners whoare sure of their own capabilities make attributions that uphold those beliefs. They attribute their success totheir own ability rather than to their effort; they tend to attribute failure to internal, unstable causes such asinsufficient effort. On the other hand, a self-doubting student attributes failure to stable causes such as poorability (Dweck, 2006). Adolescents who tend to engage in work for learning and content mastery also reporthigher self-efficacy (Pajares, Britner, & Valiante, 2000). Those who try to avoid looking incompetent tend todoubt what they can do, and in turn undermine their own potential for success. Low self-efficacy is associatedwith a host of negative emotions, including stress and depression (Bandura, 1997). Emotions related toachieving are linked to learners’ competence perceptions and their sense of control in determining theiracademic outcomes; a high sense of perceived efficacy and control brings about joy and hope; a low sense of

321

efficacy and control is often accompanied by anxiety or hopelessness (Pekrun, 2006).

322

Development of Personal Capability Beliefs

A lifetime of experience interacting with one’s environment and one’s own inner stirrings offer a wealth ofinformation about one’s present and future competencies (Bandura, 1997; Wigfield & Eccles, 2002).Experience is filtered through multiple frames of reference—both internal and external—that channellearners’ interpretations of the events that happen to them (Marsh et al., 2012). The learning context providesan important and often influential backdrop for capability beliefs to be examined and modified.

Enactive Experience

Direct, or enactive, experience of success or failure can have a profound influence on a young learner’s self-view. Seeing oneself overcome extreme odds can fundamentally alter one’s sense of efficacy; repeated failurescan reinforce a sense of inadequacy and self-doubt (Bandura, 1997). Learners evaluate their performances inrelation to others or to the internal or external expectations set for and by them. Completing 75% of a testcorrectly may be a roaring success for a certain student in a certain context and an utter devastation to another.One’s own performances are typically a strong source of one’s perceived capability. In general, perceivedmastery experiences have been shown to outweigh other types of information when predicting students’efficacy beliefs (Usher & Pajares, 2008b). However, this is not the case for all students and likely variesaccording to situational factors.

Social Comparison and Modeling

Another means by which learners assess their own capabilities is social comparison. Students revise theiracademic self-concept in terms of how their own accomplishments compare to those of their peers (Schunk,1987). Two students with the same record of academic accomplishment may develop different conceptionsabout their capability according to the social standards by which they evaluate their own performances (Marshet al., 2008). Researchers have shown that a student’s academic self-concept depends not only on personalaccomplishment but also on the accomplishments of others in the student’s nearby environment (e.g.,classroom, program of study, or school). Being a “big fish in a little pond,” that is, achieving higher than one’slower-ability classmates, raises one’s academic self-concept (Marsh et al., 2008). On the other hand, beingsurrounded by “big fish”—high-achieving classmates—has been shown to lower students’ academic self-concept. Even when students are themselves high achievers, if the average ability level of their class or schoolis high, their academic self-concept tends to suffer. Students also revise their beliefs about what they can do asthey see others accomplish similar tasks. For example, female engineers reported that they became convincedof their own efficacy for engineering by watching close family members who were engineers (Zeldin, Britner,& Pajares, 2008).

Social Messages

The messages students receive from others also influence how they will come to view their own capabilities.Bandura (1997) noted that “it is more difficult to instill enduringly high beliefs of personal efficacy bypersuasory means alone than it is to undermine such beliefs” (p. 104). Harsh evaluations of one’s capabilitiesare not easily forgotten (Pajares, 2006). Researchers have found that certain types of teacher feedback (e.g.,

323

positive, specific, effort, and ability feedback) can have beneficial effects on students’ beliefs in their owncapabilities, depending on an individual’s stage of learning and level of success (see Hattie & Timperley, 2007,for a review). The differential effects of evaluative feedback from others can be attributed to the various waysthe feedback is framed, the context in which it is provided, characteristics of the evaluator, the developmentalstage of the learner, and how the messenger is perceived by the recipient (Bandura, 1997). Some researchershave investigated the relative influence of effort feedback versus ability feedback on student outcomes. Dweckand her colleagues have reported that effort feedback (e.g., “You worked hard on that problem”) is moreeffective than ability feedback (e.g., “You’re very good at this”) for improving students’ persistence, enjoyment,and performance, particularly when students experience failure or setback (see Dweck, 2006). However,Schunk (1984) found that providing students with ability feedback led to greater gains in efficacy beliefs andperformance than did providing effort feedback. Even so, feedback that emphasizes ability may be detrimentalto self-efficacy when learners encounter difficulty, even if it does boost self-efficacy when learning is relativelyeasy (Pajares, 2006).

Emotional and Physiological Arousal

Individuals learn to read their own emotional and physiological states when they approach tasks as signs ofwhat they can and cannot do. A student whose palms drip with sweat as he stands before his classmates todeliver a speech might interpret his physiological state as a sure sign that he cannot possibly perform well.Another student may step before the class and interpret his rush of excitement as an indicator that he can acehis delivery. Researchers have consistently found that students who experience high levels of anxiety inmathematics, science, writing, and general academics report lower self-efficacy in these same areas (Usher &Pajares, 2008b). The influence of learners’ emotion is covered in greater depth in Chapter 6 of this volume.

Use of Capability-Related Information

Assessing the influence of any source of capability-related information requires attention to a number offactors. First, experiences become influential only when cognitively processed. An external observer may viewa student’s creative essay as a written work of art, but unless the student interprets the work as a success, hisbeliefs in his writing capability might not improve. Second, students integrate information across multiplesources when forming their capability beliefs (Bandura, 1997). Some research evidence has suggested that thecombined and individual influence of these sources of information varies as a function of developmental andcontextual differences (see Usher & Pajares, 2008b, for a review). For instance, Spinath and Spinath (2005)found that, as students progressed through elementary school, their ability perceptions were better explainedby their teachers’ evaluations of them than by their parents’. Girls may be more attentive to informationtransmitted socially (e.g., messages from others, exposure to social models) than are boys (Usher & Pajares,2006). Learners from different cultural backgrounds may also attend to ability-related information differently.For example, middle-school students who have been historically marginalized by the cultures in which theylearn were more attuned to social messages than were those from privileged cultural backgrounds (Usher,2009). High-school students whose outlook on ability was more malleable interpreted their experiences inunique ways from those who viewed ability as primarily unchangeable (Chen & Usher, 2013). Capabilitybeliefs might even be partly explained by genetic factors. In an investigation of 3,785 pairs of twins,

324

researchers found that considerable variation in self-perceived abilities could be explained by heritable factors(Greven, Harlaar, Kovas, Chamorro-Premuzic, & Plomin, 2009).

Developmental Trajectories

A number of longitudinal investigations have shed light on how students’ personal capability beliefs developand change over the course of schooling. As noted previously, research shows that academic self-conceptdeclines as students progress through school (e.g., Archambault, Eccles, & Vida, 2010; Wigfield, Eccles, &Rodriguez, 1998). Self-efficacy for self-regulated learning has also been shown to decline as studentstransition from junior to senior high school (Caprara et al., 2008). Conversely, students with more stableefficacy beliefs earn higher school grades and are more likely to remain in school.

Belief patterns over time vary according to context (e.g., age, domain, ability level, gender). Someresearchers have reported a curvilinear pattern in self-efficacy change in the upper-elementary grades such thatacademic self-efficacy declines between Grades 3 and 4 and increases at the end of primary school in Grades 5and 6 (Hornstra, Van der Veen, Peetsma, & Volman, 2013). Jacobs et al. (2002) found that patterns instudents’ competence beliefs across Grades 1 to 12 differed as a function of academic domain. Whereasstudents’ language arts competence beliefs declined during elementary school, these beliefs stabilized asstudents completed high school. In mathematics and sports, however, students’ competence beliefs declinedsteadily over time. Watt (2004) found that adolescent boys’ perceptions of their talent in English remainedstable from middle school to high school, but girls’ talent perceptions declined. In the studies of both Jacobs etal. and Watt, boys and girls reported lower talent perceptions in mathematics over time.

Despite the fact that longitudinal investigations show considerable individual differences in self-belieftrajectories (e.g., Hornstra et al., 2013), most trajectories still show a decline (Archambault et al., 2010). Thegeneral decline in students’ capability perceptions over time has been attributed to a number of factors,including the increasing cognitive demand of advanced academic content, change in the composition ofstudent peer groups, and changes in schooling practices (Eccles & Roeser, 2011). When learners move fromthe more communal and mastery-oriented environment of elementary school to the more formal, competitive-oriented environment of middle school, they begin to shift their frame of reference to external sources. As aresult of school structures that do not match the developmental needs of the learner (i.e., a poor fit betweenthe learner’s developmental stage and learning environment), many begin to doubt their own capabilities(Wigfield & Eccles, 2002).

325

Group Differences in Personal Capability Beliefs

Pajares (2007) wisely cautioned that “research findings and generalizations drawn from educationalpsychology broadly, and motivation theory and research in particular, cannot be taken as general rules that areindependent of contextual variation” (p. 19). The general findings reported above should be viewed in concertwith findings from more contextualized research that has aimed at examining patterns in students’ beliefs as afunction of context and of group identification, such as gender, ethnicity, nationality, and economic class.

Gender differences in students’ capability perceptions have been documented in numerous studies. In arecent meta-analysis of 187 studies of academic self-efficacy, Huang (2012) found that male students reportslightly higher self-efficacy than do female students overall, but that the direction and magnitude of genderdifferences depend on the academic domain under investigation. Female students report higher self-efficacy inlanguage arts and music, whereas male students report higher self-efficacy in mathematics, computer science,and social sciences. In their analysis of large-scale data collected in 30 countries, Williams and Williams(2010) similarly found that, when mathematics achievement is held constant, high-school girls report lowermathematics skills self-efficacy than do boys. Gender differences in self-efficacy appear to increase as studentsage (Huang, 2012). However, not all studies have observed this trend (e.g., Jacobs et al., 2002). Someresearchers have found no gender differences in students’ self-efficacy, but have shown that boys and girlsreport different exposure to sources of self-efficacy. For example, sixth-grade girls reported receiving morepositive persuasory messages from others and having been exposed more often to capable models than didboys (Usher & Pajares, 2006). For girls but not for boys, social persuasions were a significant predictor ofacademic self-efficacy. Several explanations are possible. Girls may indeed receive (or perceive) more messagesabout their capabilities than boys. Girls may also rely more on the messages they receive when judging theirown capabilities.

Some researchers have examined differences in self-beliefs among students from various ethnic groups.Students in ethnic minorities have reported stronger beliefs in their personal capabilities than other students(Hornstra et al., 2013). And yet, African American students’ capability beliefs are generally equal to or greaterthan those of their White counterparts, even in the face of lower achievement (Graham, 1994). AfricanAmerican students and Indo Canadian students have also been shown to differ from European-originstudents in the import they place on social messages when forming their beliefs about what they can doacademically (Klassen, 2004a; and see Usher, 2009; Usher & Pajares, 2008b). Some have posited that higherself-concept and self-efficacy reported by certain ethnic-minority students may reflect an ego-protectivetendency among groups that have been historically oppressed (Steele, 2010). Investigations that take a widersociocultural approach may provide a more accurate picture of students’ self-beliefs. For example, Eccles,Wong, and Peck (2006) found that the degree to which African American middle-school students perceivedracial discrimination from their teachers and peers was related to a decline in students’ ability self-conceptsacross two school years. The influence of social class has been much less examined with regard to thedevelopment of capability beliefs.

Cross-cultural research on personal capability beliefs has grown in the past two decades. Large-scaledatasets have permitted cross-national comparisons of students’ beliefs and the sources and outcomes of thosebeliefs. Research based on data from the Program for International Student Assessment (PISA), which uses

326

nationally representative samples of 15-year-olds from dozens of countries, has shown that students whoattend academically selective schools with high average achievement suffer a negative effect on their academicself-concept (Nagengast & Marsh, 2012). Self-concept has also been shown to mediate the relationship ofhigh-school students’ prior achievement and career aspirations across 34 countries. In a review of cross-cultural studies of self-efficacy, Klassen (2004b) found that non-Western cultural groups tend to report lowerefficacy beliefs than do Western groups. Although self-efficacy has been shown to be a consistent predictor ofachievement across cultures, self-efficacy research has primarily been conducted in culturally Western settingsand with students from middle- or upper-class families (Klassen & Usher, 2010), thus more work is needed inother cultural contexts.

327

Methodological Considerations

This section addresses the prominent methodological approaches that have been used to investigate personalcapability beliefs and suggests ways in which beliefs might be alternatively operationalized, assessed, andmodeled. Most researchers have relied on quantitative methods to investigate capability beliefs. This sectiondescribes some of the limitations of using this approach exclusively.

Measurement

Personal capability beliefs are typically assessed using self-report measures with Likert-type response scales.Students evaluate the strength of their beliefs by endorsing each statement at a certain level. The content ofthese statements reflects conceptual distinctions of various capability-related judgments. The discriminatingreader will look to the items used to assess a construct for a clear notion of what set of beliefs the itemsactually assess, whether items are conceptually consistent, and whether they are true to the theoretical homesfrom which they are purportedly derived. This is no small task. Marsh et al. (2012) pointed to the jinglefallacy that plagues self-belief research: “two scales with similar names might measure different constructs,whereas two scales with apparently dissimilar labels might measure similar constructs” (p. 432). Adding to thecomplication, researchers have often included self-beliefs from different theoretical traditions in the samestudy or in complex models (Murphy & Alexander, 2000). A review of self-efficacy research conductedbetween 2000 and 2009 revealed that 51% of the self-efficacy measures used were incongruent with themeasurement criteria outlined by Bandura (2006; Klassen & Usher, 2010). As Wigfield et al. (2009) aptlyobserved, “One of the measurement challenges for researchers is matching theoretical constructs toappropriate measurement tools” (p. 59).

Because personal capability beliefs vary in scope, the items used to assess them should be carefully worded.If one intends to measure self-concept, for instance, then items should be worded differently than if one’sintent is to measure self-efficacy (Bong, 2006). Measures of self-concept typically include items related toperceived competence in reference to others (e.g., “Compared to others my age I am good at math”) and pastperformances (e.g., “I have always done well in math classes”; Marsh, 1992). Such items would not beconsidered measures of self-efficacy, which refers to a can-do judgment of one’s capability to successfullyperform or learn to do a given task (e.g., “How confident are you that you can . . . successfully dividefractions?” or “ . . . learn to do complex mathematics problems?”). Self-efficacy items should also reflectvarying degrees of task difficulty (see Bandura, 2006).

As noted earlier, competence beliefs refer to one’s capability beliefs in an area, and control beliefs refer toone’s certainty that one’s actions will bring about desired outcomes. Schunk and Zimmerman (2006) pointedout that many researchers have used measures that assess both types of beliefs simultaneously. The item “Ibelieve I can score well enough on the test to make an A” contains both a judgment of competence (scoringwell enough) and of control (making an A). Measures that combine items reflecting conceptually distinctconstructs mask the relative contribution of each type of belief in prediction models.

Despite their theoretical perspective, researchers who have investigated personal capability beliefs generallyagree that domain-specific measures offer the best predictive utility when explaining domain-specificacademic functioning (Bandura, 1997; Marsh et al., 2012). For example, meta-analysis has revealed that the

328

strength of the relationship between academic self-concept and achievement varies as a function of thespecificity of the self-concept measure (Huang, 2011). Vague or global measures of self-concept or self-efficacy, although perhaps appropriate for predicting general educational outcomes, are not useful forpredicting specific outcomes. On the other hand, “judgments of competence need not be so microscopicallyoperationalized that their assessment loses all sense of practical utility” (Pajares, 1997, p. 13). Determining theappropriate level of generality/specificity requires careful consideration of the particular aims of the research(Schunk & Pajares, 2005).

If measures to assess students’ perceived capability are to predict behavioral or performance outcomes, theymust correspond in content and specificity with the outcomes they are intended to predict. Lack ofcorrespondence between belief and outcome can weaken observed effects (Bandura, 2006). The greater thecorrespondence, the greater will be the belief’s predictive power. A middle-school teacher who wants to knowwhether her students’ beliefs about their writing capabilities are related to their writing quality must firstdetermine how quality will be defined given the learning objectives (e.g., Is the writing expository, descriptive,or narrative?). The teacher can then assess beliefs germane to the writing task. Researchers who wish toinvestigate the predictive validity of perceived capability must first seek to assess beliefs that are most relevantto the course of action and skill set that will be required of the learner. The criterion task should be the pointof departure for crafting appropriate belief measures.

Design and Analysis

Students’ capability perceptions are typically collected via a survey or questionnaire and examined in relationto students’ reported perceptions about other aspects of school, such as their interest level or academic goals,and their achievement. For instance, of 210 articles published on academic self-efficacy between 2000 and2009, 89% used quantitative analyses to examine data (Klassen & Usher, 2010). Although most of the studieswere cross-sectional and correlational in design, about one-fourth included data from more than one timepoint.

Investigations that invoke a temporal lag in data collection are better suited for modeling hypothesizedrelationships between beliefs and outcomes (Valentine, DuBois, & Cooper, 2004). Longitudinal designspermit researchers to monitor changes in students’ beliefs over time and therefore offer a promising avenue forexamining inter- and intraindividual changes (Schunk & Pajares, 2005). Multilevel model-ing techniques canhelp researchers parse out the variance in self-beliefs explained by contextual factors at multiple levels (i.e.,teachers, classrooms, schools). For example, Joët, Usher, and Bressoux (2011) found that variation inchildren’s academic self-efficacy was partly explained by the average self-efficacy level of students in astudent’s class.

Personal capability beliefs have historically been investigated from a variable-centered perspective, whichinvestigates how specific variables predict the academic outcomes of most or all students. Some educationalpsychologists have advocated for a person-centered approach, which accounts for the unique and complexpsychological processes that shape individual patterns of belief and action (Snyder & Linnenbrink-Garcia,2013). For example, using latent profile analysis, Chen and Usher (2013) found that individual high-schoolstudents relied on different combinations of efficacy-relevant information when forming their beliefs abouttheir science capabilities. Modeling the hypothesized sources of self-efficacy in this way enabled the

329

researchers to examine not only the additive effects that traditional measures such as multiple linear regressioncan model, but combinatory effects as well. Similarly, a person-centered analysis revealed a more nuancedpicture of the relationship among self-efficacy, interest, gender, and high-school students’ conceptual changethan would have been obtained from a variable-centered approach (Linnenbrink-Garcia, Pugh, Koskey, &Stewart, 2012).

Theoretical questions must guide method. Including a measure of perceived capability in a complex modelof academic motivation without providing a rationale risks obscuring what is known about how those beliefsfunction. The utility of testing complex analytic models should be weighed against their potential benefit totheory and practice. Qualitative methods, such as interviews, open-ended response techniques, and thought-listing procedures, can permit researchers to gain an in-depth picture of how learners’ beliefs develop andoperate (Usher, 2009). Mobile technologies can also permit investigators to obtain moment-to-momentglimpses of how learners’ beliefs function in everyday settings.

330

Directions for Future Research

Pursuing new avenues of research can help elucidate the role of personal capability beliefs in academicfunctioning and human development. As has been shown, the fact that capability beliefs serve as guides forbehavior is well documented. Perhaps the field would be served by research that can clarify how and underwhat conditions learners come to form and alter their beliefs and how those beliefs change in magnitude andinfluence over time. Researchers have pointed to the need for additional investigations on the sources ofcapability beliefs such as self-efficacy (Usher & Pajares, 2008b). Most investigations have focused onexperiences that build perceived competence (e.g., successful performances, positive evaluative messages,exposure to competent models). Investigating the experiences that undermine self-efficacy might provide amore complete picture of how some students become convinced of their inefficacy. Such investigations couldinclude measures that reflect failure experiences (e.g., “I have performed poorly on mathematics tests”) orcould target learners who have been identified as at risk for academic failure.

Researchers should also consider influences beyond those typically examined. For example, how might theavailability of assistance or the method of content delivery affect students’ beliefs? How might autonomy-supportive or restrictive environments affect them? Investigating whether students tend to rely on the same ordifferent sources of capability-related information (e.g., social messages, exposure to models) over time andacross contexts would also prove informative. Some experiences or accomplishments might be consideredtransformative such that they affect perceived ability in other areas. For example, might athleticaccomplishment transfer to the academic realm? Might an intense experience such as a study-abroad course oran outdoors education retreat offer transformative personal insights that change one’s perspective andperceived capabilities? These questions point to how individuals construe their experiences. Formulas,heuristics, biases—the lens through which individuals interpret what happens to them—are central to howexperiences shape beliefs. Following corrective feedback from the teacher, one student finds himself amidst acatastrophe, another shrugs it off without a second thought, and a third rallies to the challenge by changinghis approach. Research should investigate the ways in which students frame their academic experiences (andperhaps their lives) and in turn alter their capability perceptions.

Intervention studies permit researchers to test the mechanisms by which capability beliefs change. Mostintervention studies in education have been aimed at assessing the influence of instructional, programmatic, orcurricular changes on specific learning objectives such as skill development and achievement. The effectivenessof innovative teaching approaches or novel curricula can be evaluated not only in terms of learning andbehavioral outcomes but also in terms of changes in learners’ perceived capabilities. The inclusion of perceivedcapability measures would enable researchers to view whether school-based changes have correspondingpsychological effects (see Cleary, 2009).

Other interventions directly target students’ motivation and capability beliefs (see Wentzel & Wigfield,2007, and Chapter 12, this volume). Such interventions are becoming more common and hold promise fordemonstrating how learners view themselves and their capabilities under varying conditions. Experimentaldesigns are particularly useful. For example, Blackwell et al. (2007) randomly assigned some middle-schoolstudents to attend an 8-week workshop that emphasized brain growth as a result of persistence on challengingproblems. Others in a control group attended a study skills workshop. Students who received the intervention

331

not only reported more malleable views of intelligence but also earned higher mathematics grades thanstudents in the control group.

Short-term interventions that require low-cost participatory involvement from students may be effective atbringing about lasting attitudinal and behavioral change (Yeager & Walton, 2011). An example comes from aburgeoning area of research on mindfulness, which suggests that individuals who engage in contemplativepractices such as meditation or mindful movement show improvements in their emotion regulation, cognition,attention, motivation, and self-representation (Davidson et al., 2012). Engaging in contemplative practicemight lead to changes in a learner’s self-construal. Through the cultivation of awareness, learners practicevolitional ways of thinking about, experiencing, and making sense of their experiences (Roeser & Peck, 2009).These practices differ from automatic ways of processing information and may offer “numerous opportunitiesfor the development of new ways of understanding oneself and one’s attempts to learn and be resilient duringthe process of learning” (Roeser & Peck, 2009, p. 129; see Chapter 8, this volume). Using an experimentalintervention design, researchers can address whether brief classroom-based contemplative practices influencelearners’ beliefs about their academic and self-regulatory capabilities (Greenberg & Harris, 2012). Researcherswho wish to implement such motivation interventions in schools still face many challenges, however, such asremaining true to an intervention’s theoretical underpinnings, ensuring systematic delivery in school settingsthat are much “noisier” than laboratories, and maintaining sensitivity to diverse needs.

Novel methodological approaches might shed light on how contextual factors influence students’ beliefs.Observational measures are useful for assessing classroom practices and can be examined in correlation withstudents’ beliefs in different learning contexts. For example, capability beliefs might be formed in distinct waysin various classroom configurations: competitive or cooperative, teacher- or student-centered, or technology-heavy (e.g., one device per student) vs. technology-light (few or no digital devices). Similar methods can beused to show the relationship between teacher-level variables (e.g., teacher self-efficacy, teacher competence,pedagogical approach, teacher background) and student beliefs. What teacher practices lead to positivechanges in personal capability beliefs? What practices undermine students’ self-perceptions? Multilevel modelscan allow researchers to parse out variation in self-beliefs due to teachers, classrooms, or schools. Using resultsobtained from sophisticated quantitative analyses to select participants for targeted follow-up interviews couldcreate a rich description of how beliefs develop in different contexts. Qualitative inquiry would enableresearchers to explore the complex ways in which particular students make sense of their environment andexperience.

Research designs should take into account not only the immediate but also the long-term consequences ofthe beliefs students hold about their own capabilities. Although the bulk of the longitudinal research on self-perceived capabilities has examined self-concept, additional work is needed to show how other beliefs evolveas students progress through school. What factors are associated with increasing or decreasing self-beliefs overtime? Are growth trajectories similar for boys and girls or for students of different socioeconomicbackgrounds, for instance? As noted earlier, most studies of students’ self-efficacy have been correlational andcross-sectional in nature. More evidence of the causal relationship between self-efficacy and various outcomesis also needed and will require that data be temporally spaced.

The bulk of the research on personal capability beliefs has been conducted in the area of mathematics. Lessis known about how learners’ beliefs develop and are related to outcomes in other academic disciplines. For

332

example, social studies, civics, music, art, engineering, history, science, and language learning have seen lessresearch in this area and warrant further exploration. Researchers could also compare sources and effects ofpersonal capability beliefs across disciplinary contexts. Such efforts might also address the extent to whichpersonal capability beliefs generalize from one domain to another.

Advancing digital technologies have placed personalized content at one’s fingertips and brought remotesocial environments into one’s own home. People can now select their social environments in unprecedentedways. Some evidence has shown that personal capability beliefs (i.e., self-efficacy) influence learning outcomesin computer-based learning environments (Moos & Azevedo, 2009). More research is needed to examine howpersonal capability beliefs affect and are affected by different virtual environments. Because educational unitshave relied increasingly on technologies to deliver content, researchers should examine learners’ technology-related competence beliefs and how they develop and function. How do beliefs influence technology adoptionand implementation? How do they affect learning? One can only imagine how students who doubt theircapabilities with technology might fare in high-tech learning environments. The ubiquity of technology alsoplaces a higher demand on learners to regulate their own activities. Those with high self-efficacy for self-regulated learning may benefit from technology-rich learning modalities; those who do not believe themselvescapable of modulating electronic distraction may not fare as well (Schunk & Usher, 2011). Learners likely varyin their perceptions about how well they can regulate their use of and exposure to social and academictechnologies; what might these variations predict?

Despite the assumption made at the start of this chapter that beliefs are rules for action, in some cases,beliefs and actions are misaligned. Consider the question, “What shall we do with overconfident students?”Albert Bandura said in an American Psychological Association address in 1998 that, “One cannot afford to bea realist.” By this he meant that beliefs that slightly exceed one’s ability may in fact provide the incentive forone to take a risk, and eventually, to grow. But this is a fair question, particularly at a time when comedysketches poke fun at a society that tells kids, “You can do anything!” Although most students’ perceivedcapabilities are linearly related to their achievement, there are those individuals for whom a large discrepancybetween the two exists. These are so-called “poorly calibrated” students, whose beliefs do not correspond withtheir actions, and they may be at considerable academic risk. Overconfidence can lead to complacency whenremedial action is needed (Zimmerman, 2011). Underconfidence might put students at even greater risk;otherwise able students who are beset by self-doubt shortchange themselves and squander their own potential.Although researchers have addressed this conceptually (e.g., Pajares, 2006), few have empirically examinedpoor calibration between self-perceived capability and actual skill (see Alexander, 2013). A first step is todetermine how best to identify poorly calibrated students. A host of research questions can then be addressed.For instance, are certain students more likely to overestimate or underestimate what they can do? Do thesestudents use unique approaches for making sense of capability-related information? Identifying those withpoor calibration between beliefs and outcome might become a useful diagnostic tool for targetedinterventions.

Last but not least, researchers should seek to expand the cultural contexts in which personal capabilitybeliefs are examined (Pajares, 2007). An overwhelming number of studies in social and educationalpsychology have been conducted in so-called “WEIRD” (Western, educated, industrialized, rich, anddemocratic) societies (Henrich, Heine, & Norenzayan, 2010). Cross-national comparative research offers

333

excellent information, but most has focused on high-school students. Less is known about how youngerstudents, particularly those in elementary school, develop their self-beliefs. Research documenting thepsychological landscape of students living in poverty or in rural settings is also scarce, as is research in urbanareas and in schools predominantly attended by students from historically marginalized ethnic groups.

334

Conclusion and Implications

In 2006, an internet search generated over half a million web pages for the term “self-efficacy” (Pajares, 2006).At the time of this chapter’s writing, a Google search generated 9.6 million results; “self-concept of ability”generated 28.2 million. It would seem that interest in self-beliefs has grown by leaps and bounds. Severalpertinent implications are worth emphasis in closing. First, beliefs related to one’s academic capabilities mustbe developed and nurtured in conjunction with the development of academic skills. Attempts at ego or self-esteem boosting will have short-lived and perhaps even harmful results (see Baumeister et al., 2003). Invitinglearners to experience failure as a normal part of growth can help ground their beliefs on their own perseverantaccomplishments. Second, most students live in a culture where rewards and recognition programs, abilitygrouping, and other overt (and covert) means of publicizing ability-related information are readily available.Such information can shake students’ beliefs in their own capabilities (Pajares, 2006). As noted above,researchers have shown that self-concept suffers when students perceive that they are surrounded by morecapable peers. One way of addressing this is by changing school structures and systems. However,investigating dispositions within the individual that might mitigate the potentially harmful effects of socialcomparative information would be also useful. Teachers and parents can also help convey the message that thepond is big enough for all fish to thrive. Third, teachers should monitor and attend to students’ changing self-beliefs and their association with changing learning structures. To do this will require the cultivation ofcompassionate awareness so that teachers can “be totally and nonselectively present to the student—to eachstudent” (Noddings, 1984, p. 180). Teachers must also critically examine school structures that, thoughmainstream and longstanding, may be unsupportive of or detrimental to students’ developing self-conceptions.Calming the busyness and stress that interfere with the capacity for openness and awareness will enablepractitioners to attend to how students interpret their own experiences and feelings and alter their self-beliefs.

It seems fair to assume that most parents want their children to develop a healthy self-view. Nothingfacilitates this more than unconditional love and positive regard. These support the child’s naturaldevelopment into a being unique to the world. As young people grow and ego develops, they naturally beginto look to external sources for validation. But this hardly marks the end of the road to so-called self-knowledge. The way of knowing oneself is not permanent but evolving. This evolution might be towardunderstanding that which James (1892/2001) called the I, a Knower whose vantage point transcends theexternal and habitual ways in which we come to know ourselves. What would happen to the Self if theindividual suspended or dropped the ego involvement of the many Mes (i.e., social, material) that receive somuch attention? What then would be the role of external sources in changing self-beliefs? Would a Selfcontinue to exist? Japanese Zen Master Dogen once wrote, “To study the Way is to study the self. To studythe self is to forget the self. To forget the self is to be enlightened by all things” (Kim, 2004, p. xxiv). Perhapsbeyond self-concepts lies a Knower who understands that our fullest capabilities are yet to be found. Thisknowledge gives beliefs their motivating power. Perhaps, then, the best advice parents and teachers can givelearners is what Christopher Robin shared with Pooh (Geurs, 1997): “Promise me you’ll always remember:You’re braver than you believe, and stronger than you seem, and smarter than you think.”

335

336

References

Alexander, P. A. (2013). Calibration: What is it and why it matters? An introduction to the special issue on calibrating calibration. Learning andInstruction, 24, 1–3.

Archambault, I., Eccles, J. S., & Vida, M. N. (2010). Ability self-concepts and subjective value in literacy: Joint trajectories from grades 1through 12. Journal of Educational Psychology, 102, 804–816.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.Bandura, A. (1998). Exercise of agency in accenting the positive. Invited address delivered at the meeting of the American Psychological

Association, San Francisco.Bandura, A. (2006). Guide for constructing self-efficacy scales. In F. Pajares & T. Urdan (Eds.), Adolescence and education, Vol. 5: Self-efficacy

and adolescence (pp. 307–337). Greenwich, CT: Information Age Publishing.Baumeister, R. F., Campbell, J. D., Krueger, J. I., & Vohs, K. D. (2003). Does high self-esteem cause better performance, interpersonal success,

happiness, or healthier lifestyles? Psychological Science in the Public Interest, 4, 1–44.Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent

transition: A longitudinal study and an intervention. Child Development, 78, 246–263.Bong, M. (2006). Asking the right question: How confident are you that you could successfully perform these tasks? In F. Pajares & T. C.

Urdan (Eds.), Adolescence and education: Vol. 5. Self-efficacy beliefs of adolescents (pp. 287–305). Greenwich, CT: Information Age.Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15,

1–40.Caprara, G. V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G. M., Barbaranelli, C., & Bandura, A. (2008). Longitudinal analysis of the

role of perceived self-efficacy for self-regulated learning in academic continuance and achievement. Journal of Educational Psychology, 100(3),525–534.

Chen, J. A., & Usher, E. L. (2013). Profiles of the sources of science self-efficacy. Learning and Individual Differences, 24, 11–21.Cleary, T. J. (2009). Monitoring trends and accuracy of self-efficacy beliefs during interventions: Advantages and potential applications to

school-based settings. Psychology in the Schools, 46(2), 154–171.Davidson, R. J., Dunne, J., Eccles, J. S., Engle, A., Greenberg, M., Jennings, P., . . . & Vago, D. (2012). Contemplative practices and mental

training: Prospects for American education. Child Development Perspectives, 6, 146–153.Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum.De Naeghel, J., Van Keer, H., Vansteenkiste, M., & Rosseel, Y. (2012). The relation between elementary students’ recreational and academic

reading motivation, reading frequency, engagement, and comprehension: A self-determination theory perspective. Journal of EducationalPsychology, 104(4), 1006–1021.

Dewey, J. (1933). How we think. Boston: D. C. Heath.Duckworth, A. L., & Kern, M. L. (2011). A meta-analysis of the convergent validity of self-control measures. Journal of Research in Personality,

35, 259–268.Duckworth, A. L., & Seligman, M. E. P. (2006). Self-discipline gives girls the edge: Gender in self-discipline, grades, and achievement test

scores. Journal of Educational Psychology, 98, 198–208.Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. Philadelphia, PA: Taylor and Francis.Dweck, C. S. (2006). Mindset. New York: Random House.Eccles, J. S., & Roeser, R. W. (2011). Schools as developmental contexts during adolescence. Journal of Research on Adolescence, 21, 225–241.Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132.Eccles, J. S., Wong, C. A., & Peck, S. C. (2006). Ethnicity as a social context for the development of African American adolescents. Journal of

School Psychology, 44(5), 407–426.Fredricks, J., & Eccles, J. S. (2002). Children’s competence and value beliefs from childhood through adolescence: Growth trajectories in two

male-sex-typed domains. Developmental Psychology, 38, 519–533.Geurs, K. (Producer and Director). (1997). Pooh’s grand adventure: The search for Christopher Robin [Motion picture]. USA: Walt Disney

Television Animation.Gniewosz, B., Eccles, J. S., & Noack, P. (2012). Secondary school transition and the use of different sources of information for the construction

of the academic self-concept. Social Development, 21(3), 537–557.Graham, S. (1994). Motivation in African Americans. Review of Educational Research, 64, 55–118.Graham, S., & Weiner, B. (1996). Theories and principles of motivation. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational

psychology (pp. 63–84). New York: Simon & Schuster Macmillan.

337

Greenberg, M. T., & Harris, A. R. (2012). Nurturing mindfulness in children and youth: Current state of research. Child DevelopmentPerspectives, 6(2), 161–166.

Greven, C. U., Harlaar, N., Kovas, Y., Chamorro-Premuzic, T., & Plomin, R. (2009). More than just IQ: School achievement is predicted byself-perceived abilities—but for genetic rather than environmental reasons. Psychological Science, 20(6), 753–762.

Harter, S. (2012). The construction of the self: Developmental and sociocultural foundations (2nd ed.). New York: The Guilford Press.Hattie, J. (1992). Self-concept. Hillsdale, NJ: Lawrence Erlbaum.Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112.Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? The Behavioral and Brain Sciences, 33(2–3), 61–83.Hornstra, L., Van der Veen, I., Peetsma, T., & Volman, M. (2013). Developments in motivation and achievement during primary school: A

longitudinal study on group-specific differences. Learning and Individual Differences, 23, 195–204.Huang, C. J. (2011). Self-concept and academic achievement: A meta-analysis of longitudinal relations. Journal of School Psychology, 49(5), 505–

528.Huang, C. (2012). Gender differences in academic self-efficacy: A meta-analysis. European Journal of Psychology of Education, 28(1), 1–35.Jacobs, J. E., Lanza, S., Osgood, D. W., Eccles, J. S., & Wigfield, A. (2002). Changes in children’s self-competence and values: Gender and

domain differences across grades one through twelve. Child Development, 73, 509–527.James, W. (1892/2001). Psychology: The briefer course. New York: Dover.Joët, G., Usher, E. L., & Bressoux, P. (2011). Sources of self-efficacy: An investigation of elementary school students in France. Journal of

Educational Psychology, 103(3), 649–663.Kim, H.-J. (2004). Eihei Dōgen: Mystical realist. Somerville, MA: Wisdom Publications.Klassen, R. (2004a). A cross-cultural investigation of the efficacy beliefs of South Asian immigrant and Anglo non-immigrant early adolescents.

Journal of Educational Psychology. 96, 731–742.Klassen, R. M. (2004b). Optimism and realism: A review of self-efficacy from a cross-cultural perspective. International Journal of Psychology, 39,

205–230.Klassen, R. M., Ang, R., Chong, W. H., Krawchuk, L. L., Huan, V., Wong, I., & Yeo, L. S. (2009). A cross-cultural study of adolescent

procrastination. Journal of Research on Adolescence, 19, 799–811.Klassen, R. M., & Usher, E. L. (2010). Self-efficacy in educational settings: Recent research and emerging directions. In T. C. Urdan & S. A.

Karabenick (Eds.), Advances in motivation and achievement: Vol. 16A. The decade ahead: Theoretical perspectives on motivation and achievement(pp. 1–33). Bingley, UK: Emerald Publishing Group.

Linnenbrink-Garcia, L., Pugh, K., Koskey, K. L. K., & Stewart, V. (2012). Developing conceptual understanding of natural selection: The roleof interest, efficacy, and basic prior knowledge. Journal of Experimental Education, 80, 45–68.

Mangels, J. A., Butterfield, B., Lamb, J., Good, C., & Dweck, C. S. (2006). Why do beliefs about intelligence influence learning success? Asocial cognitive neuroscience model. Social Cognitive and Affective Neuroscience, 1(2), 75–86.

Marsh, H. W. (1992). Self-description questionnaire III: Manual. Sydney: University of Western Sydney, SELF Research Centre.Marsh, H. W., & Craven, R. G. (2006). Reciprocal effects of self-concept and performance from a multidimensional perspective: Beyond

seductive pleasure and unidimensional perspectives. Perspectives on Psychological Science, 1, 133–163.Marsh, H. W., & O’Mara, A. (2008). Reciprocal effects between academic self-concept, self-esteem, achievement and attainment over seven

adolescent years: Unidimensional and multidimensional perspectives of self-concept. Personality and Social Psychology Bulletin, 34, 542–552.Marsh, H. W., Trautwein, U., Lüdtke, O., & Köller, O. (2008). Social comparison and big-fish-little-pond effects on self-concept and other

self-belief constructs: Role of generalized and specific others. Journal of Educational Psychology, 100, 510–524.Marsh, H. W., Xu, M., & Martin, A. J. (2012). Self-concept: A synergy of theory, method, and application. In K. R. Harris, S. Graham, & T.

C. Urdan (Eds.) APA educational psychology handbook, Vol. 1: Theories, constructs, and critical issues (pp. 427–458). Washington, DC: AmericanPsychological Association.

Maslow, A. H. (1968). Toward a psychology of being. New York: Van Nostrand Reinhold.Moos, D. C., & Azevedo, R. (2009). Learning with computer-based learning environments: A literature review of computer self-efficacy.

Review of Educational Research, 79(2), 576–600.Murphy, P. K., & Alexander, P. A. (2000). A motivated exploration of motivation terminology. Contemporary Educational Psychology, 25, 3–53.Nagengast, B., & Marsh, H. W. (2012). Big fish in little ponds aspire more: Mediation and cross-cultural generalizability of school-average

ability effects on self-concept and career aspirations in science. Journal of Educational Psychology, 104(4), 1033–1053.Noddings, N. (1984). Caring: A feminine approach to ethics and moral education. Berkeley, CA: University of California Press.Pajares, F. (1997). Current directions in self-efficacy research. In M. Maehr & P. R. Pintrich (Eds.). Advances in motivation and achievement.

(Vol. 10, pp. 1–49). Greenwich, CT: JAI Press.Pajares, F. (2006). Self-efficacy during childhood and adolescence: Implications for teachers and parents. In F. Pajares & T. Urdan (Eds.),

338

Adolescence and education, Vol. 5: Self-efficacy beliefs of adolescents (pp. 339–367). Greenwich, CT: Information Age Publishing.Pajares, F. (2007). Culturalizing educational psychology. In F. Salili & R. Hoosain (Eds.), Culture, motivation, and learning (pp. 19–42).

Charlotte, NC: Information Age.Pajares, F., Britner, S. L., & Valiante, G. (2000). Relation between achievement goals and self-beliefs of middle school students in writing and

science. Contemporary Educational Psychology, 25, 406–422.Pajares, F., & Schunk, D. H. (2002). Self-efficacy and self-belief in psychology and education: A historical perspective. In J. Aronson & D.

Cordova (Eds.), Psychology of education: Personal and interpersonal forces (pp. 5–25). New York: Academic Press.Parker, P. D., Schoon, I., Tsai, Y.-M., Nagy, G., Trautwein, U., & Eccles, J. S. (2012). Achievement, agency, gender, and socioeconomic

background as predictors of post-school choices: A multicontext study. Developmental Psychology, 48, 1629–1642.Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and

practice. Educational Psychology Review, 18, 315–341.Reeve, J. (2004). Self-determination theory applied to educational settings. In E. L. Deci and R. M. Ryan (Eds.), Handbook of self-determination

research (pp. 183–203). Rochester, NY: University of Rochester Press.Roeser, R. W., & Peck, S. C. (2009). An education in awareness: Self, motivation, and self-regulated learning in contemplative perspective.

Educational Psychologist, 44, 119–136.Roeser, R. W., Peck, S. C., & Nasir, N. S. (2006). Self and identity processes in school motivation, learning, and achievement. In P. A.

Alexander & P. H. Winne (Eds.), Handbook of educational psychology (pp. 391–424). Mahwah, NJ: Lawrence Erlbaum Associates.Rogers, C. R. (1947). Some observations on the organization of personality. American Psychologist, 2, 358–368.Rokeach, M. (1968). Beliefs, attitudes, and values: A theory of organization and change. San Francisco: Jossey-Bass.Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational

Psychology, 25, 54–67.Schunk, D. H. (1984). Sequential attributional feedback and children’s achievement behaviors. Journal of Educational Psychology, 76, 1159–1169.Schunk, D. H. (1987). Peer models and children’s behavioral change. Review of Educational Research, 57, 149–174.Schunk, D. H., & Pajares, F. (2005). Competence perceptions and academic functioning. In A. J. Elliot & C. Dweck (Eds.), Handbook of

competence and motivation (pp. 85–104). New York: Guilford Press.Schunk, D. H., & Usher, E. L. (2011). Assessing self-efficacy for self-regulated learning. In B. J. Zimmerman, & D. H. Schunk (Eds.),

Handbook of self-regulation of learning and performance (pp. 282–297). New York, NY: Routledge.Schunk, D. H., & Zimmerman, B. J. (2006). Competence and control beliefs: Distinguishing the means and ends. In P. A. Alexander, & P. H.

Winne (Eds.), Handbook of educational psychology, 2nd ed. (pp. 349–367). New York, NY: Lawrence Erlbaum Associates.Sheu, H.-B., Lent, R. W., Brown, S. D., Miller, M. J., Hennessy, K. D., & Duffy, R. D. (2010). Testing the choice model of social cognitive

career theory across Holland themes: A meta-analytic path analysis. Journal of Vocational Behavior, 76, 252–264.Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008). Engagement and disaffection in the classroom: Part of a larger motivational

dynamic? Journal of Educational Psychology, 100, 765–781.Snyder, K. E., & Linnenbrink-Garcia, L. (2013). A developmental, person-centered approach to exploring multiple motivational pathways in

gifted underachievement. Educational Psychologist, 48(4), 209–228.Spinath, B., & Spinath, F. M. (2005). Longitudinal analysis of the link between learning motivation and competence beliefs among elementary

school children. Learning and Instruction, 15(2), 87–102.Steele, C. M. (2010). Whistling Vivaldi and other clues to how stereotypes affect us. New York, NY: W. H. Norton.Watt, H. M. G. (2004). Development of adolescents’ self-perceptions, values, and task perceptions according to gender and domain in 7th-

through 11th-grade Australian students. Child Development, 75, 1556–1574.Wentzel, K. R., & Wigfield, A. (2007). Motivational interventions that work: Themes and remaining issues. Educational Psychologist, 42(4),

261–271.Wigfield, A., & Eccles, J. S. (2002). The development of competence beliefs, expectancies for success, and achievement values from childhood

through adolescence. In A. Wigfield & J. S. Eccles (Eds.), Development of achievement motivation (pp. 91–120). San Diego, CA: AcademicPress.

Wigfield, A., Eccles, J. S., & Rodriguez, D. (1998). The development of children’s motivation in school contexts. Review of Research inEducation, 23, 73–118.

Wigfield, A., Tonks, S., & Klauda, S. L. (2009). Expectancy-value theory. In K. R. Wentzel & A. Wigfield (Eds.), Handbook of motivation atschool (pp. 55–76). Mahwah, NJ: Lawrence Erlbaum Associates.

Williams, T., & Williams, K. (2010). Self-efficacy and performance in mathematics: Reciprocal determinism in 33 nations. Journal ofEducational Psychology, 102(2), 453–466.

Wolters, C. A., Fan, W., & Daugherty, S. G. (2013). Examining achievement goals and causal attributions together as predictors of academic

339

functioning. Journal of Experimental Education, 81, 295–321.Wylie, R. C. (1989). Measures of self-concept. Lincoln, NE: University of Nebraska Press.Usher, E. L. (2009). Sources of middle school students’ self-efficacy in mathematics: A qualitative investigation of student, teacher, and parent

perspectives. American Educational Research Journal, 46, 275-314.Usher, E. L., & Pajares, F. (2006). Sources of academic and self-regulatory efficacy beliefs of entering middle school students. Contemporary

Educational Psychology, 31, 125–141.Usher, E. L., & Pajares, F. (2008a). Self-efficacy for self-regulated learning: A validation study. Educational and Psychological Measurement, 68,

443–463.Usher, E. L., & Pajares, F. (2008b). Sources of self-efficacy in school: Critical review of the literature and future directions. Review of

Educational Research, 78, 751–796.Valentine, J. C., DuBois, D. L., & Cooper, H. (2004). The relation between self-beliefs and academic achievement: A meta-analytic review.

Educational Psychologist, 39, 111–133.Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research,

81(2), 267–301.Zeldin, A. L., Britner, S. L., & Pajares, F. (2008). A comparative study of the self-efficacy beliefs of successful men and women in

mathematics, science, and technology careers. Journal of Research in Science Teaching, 45, 1036–1058.Zimmerman, B. J. (2011). Motivational sources and outcomes of self-regulated learning and performance. In B. J. Zimmerman & D. H.

Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 49–64). New York, NY: Taylor & Francis.Zimmerman, B. J., & Cleary, T. J. (2006). Adolescents’ development of personal agency. In F. Pajares & T. Urdan (Eds.), Adolescence and

education, Vol. 5: Self-efficacy beliefs of adolescents (pp. 45–69). Greenwich, CT: Information Age.Zuffiano, A., Alessandri, G., Gerbino, M., Kanacri, B. P. L., Di Giunta, L., Milioni, M., & Caprara, G. V. (2013). Academic achievement:

The unique contribution of self-efficacy beliefs in self-regulated learning beyond intelligence, personality traits, and self-esteem. Learning andIndividual Differences, 23, 158–162.

340

12Motivation Interventions in Education

Bridging Theory, Research, and Practice

CHRIS S. HULLEMAN

University of Virginia

KENN E. BARRON

James Madison University

[T]he job of the educational psychologist is to psychologize about authentic educational problems and issues, and not simply to bringpsychology to education, as if we were missionaries carrying out the Lord’s work. (Berliner, 2006, p. 23)

To the extent that psychologists [are] willing to walk down a two-way street with educators, there is increased hope for realizing the long-held goal of applying the science of learning to education. Striving to achieve this goal is a worthwhile adventure that offers advantagesboth to educational practice and psychological theory. (Mayer, 2012, p. 250)

The field of motivation research within educational psychology has been especially generative over the lastseveral decades, in particular by producing theories, constructs, and tests thereof. However, this researchproductivity has not resulted in comparable benefits for educational practice (Berliner, 2006; Kaplan, Katz, &Flum, 2012). Our current methods have been unbalanced in favor of observational, correlational, andlaboratory studies that often have implications for practice but do not end up changing practice. In otherwords, we have “brought psychology to education” by developing theories and constructs without regard forsolving the practical problems of educators. Although helpful in advancing theory, this test-theory-first, solve-problems-second approach has served to exacerbate gaps between research and practice. Fortunately, there isan alternative.

In this chapter, we consider how intervention studies can help educational psychologists walk withpractitioners and bridge the research–practice divide, particularly in the area of student motivation. In Part 1,we consider the case for intervention research as a bridge between motivation theory and research, on the onehand, and practice, on the other. In Part 2, we review two different intervention approaches for conductingmotivational interventions: targeted interventions and multicomponent interventions. In Part 3, we provide adetailed case study of each intervention approach. In Part 4, we offer conclusions and recommendations fornext steps.

341

Part 1: Motivation Research in Education: The Case for Interventions

As psychologists, we are trained in our earliest methodology classes that research is motivated by three majorgoals: to describe, predict, or explain human behavior. In addition, we are exposed to the major researchmethodologies that allow us to answer each of these goals, by using observational methods for description,correlational methods for prediction, and experimental methods for causation (or quasi-experimental methodsfor limited causation). Whereas each methodology has merit, many of our earliest undergraduate psychologycourses in methodology are entitled experimental psychology, thus implying a preference or bias for oneparticular methodological approach over others (Perlman & McCann, 2005). But non-experimental methodshold an important place for initiating and advancing research. When little is known about a topic, one of thebest starting points is to simply observe the phenomenon of interest to answer the question: “What is X?” Forexample, how many students feel self-efficacy and value for their math class? This in turn can quickly lead toexplorations of how two or more observed phenomena are related to each other to answer a moresophisticated question: “Does X predict Y?” For example, what short-term and long-term educationaloutcomes are predicted by students’ feelings of self-efficacy and value?

As a field, we have learned a great deal about important motivational variables through observational andcorrelational methods, and how those variables are linked to key adaptive student outcomes. Research on self-efficacy and value reveals that both generally decline as students progress through school (e.g., Jacobs, Lanza,Osgood, Eccles, & Wigfield, 2002), and that self-efficacy and value predict unique educational outcomes.Self-efficacy is generally a stronger predictor of performance outcome such as grades and standardized testscores, whereas value is a stronger predictor of continued course taking and interest in that subject (for reviewssee Wigfield & Eccles, 2000; Wigfield & Cambria, 2010). Another benefit of adopting non-experimentalmethods, especially in real-world educational contexts, is when it is unethical or unrealistic to manipulate thebehavior of interest (Harackiewicz & Barron, 2004).

However, the ability to establish causation and answer the question “Does X cause Y?” requires anexperimental or interventionist approach, where a researcher can formally manipulate and introduce anindependent variable to see the effect it has on an observable outcome (cf. Shavelson, Phillips, Towne, &Feuer, 2003). Finding the underlying cause is often seen as the pinnacle of research and the culmination of aresearch continuum that may start out as non-experimental but end with a clear causal test (Shadish, Cook, &Campbell, 2002). This also allows us to best answer to teachers and school administrators who ask us whatthey can change to increase the self-efficacy and value of their students. We want to be able to say, if you doX, it will cause your students to have more of Y. Unfortunately, even though our earliest methodology coursesemphasize experimental methods, as a motivational field, we have conducted far less work that would fallunder either an experimental or interventionist approach.

In this chapter we address this shortcoming by reviewing the current state of intervention work that hasbeen conducted in the field of motivation. We define an intervention as a manipulation implemented by anexternal agent (i.e., teacher, researcher) that was intended to change students’ cognitions, emotions, and/orbehaviors (Lazowski & Hulleman, 2013). As such, utilizing an intervention methodology does not require arandomized experiment. Instead, we consider intervention to be an umbrella term that includes a variety ofmethodological approaches, including randomized experiments and design-based research (Kaplan et al.,

342

2012), both of which are reviewed by Penuel and Frank in Chapter 2 of this handbook.In a randomized field experiment (i.e., randomized control trial), the effectiveness of an intervention is

tested in the field on a particular population of interest by randomly assigning individuals to either a treatmentgroup or a control group. It is held as a gold standard for validating the effectiveness of the treatment freefrom biases that occur when participation is self-selected or non-random (Schneider, Carnoy, Kilpatrick,Schmidt, & Shavelson, 2005; Shadish et al., 2002). Design-based research is a general label for an emergingbody of approaches in which researchers and practitioners work in iterative cycles in naturalistic settings totest and refine interventions to improve learning and instruction (e.g., Brown, 1992; Design-based ResearchCollective, 2003). Penuel and Frank (Chapter 2, this volume) liken this methodology to engineering, whereprototype testing is at the center of learning what does and does not work. Qualitative and/or quantitativeevidence is collected to test the success or failure of an intervention, and then a team reflects on that data toinform changes to the intervention for the next round of testing.

Although formal tests of randomized interventions are a great way to establish causality, they’re not theonly reason to conduct intervention research (Shavelson et al., 2003). In fact, an arguably more importantreason to conduct intervention research is to operationalize our theoretical constructs as potential educationalpractices that boost motivation and learning. It is one thing, for example, to observe that students with higherself-efficacy or perceived value for math at the beginning of a course perform better and learn more at the endof the course. It is another thing, entirely, to recommend changes in teaching practices based on thisobservation. What should teachers do differently to increase students’ self-efficacy and help them find value inmath? Should they change how they talk about student successes and failures? The grading structure? Thecontent of what they teach? The types of learning activities they provide for students? The only way to makeclear recommendations about what practitioners should actually do based on our theories and research is todevelop recommendations for practice, and then systematically engage in intervention research to test theireffects on student learning outcomes. It is in precisely these situations, when potential intervention ideas arebeing developed, that design-based studies and other types of quasi-experiments provide importantinformation (see Chapter 2, this volume; Kaplan et al., 2012; Shavelson et al., 2003). For example, how wellreceived are the suggested learning activities by students? How easy are they to implement by teachers? Howdifferent are the recommendations from current practice? All these questions may best be answered outside ofa randomized experiment.

The research methods selected have clear implications for the conclusions that can be drawn from the work(Barron, Brown, Egan, Gesualdi, & Marchuk, 2008; Harackiewicz & Barron, 2004). Most notable is thetradeoff between cause and effect (i.e., internal validity) and generalizability (i.e., external validity; Shadish etal., 2002). One reason researchers choose to study phenomena in laboratory settings is to gain experimentalcontrol to isolate the variables of interest, while holding extraneous variables constant. Although helpful inestablishing internal validity, a laboratory setting is subject to artificiality that threatens generalizability.Similar challenges exist even for randomized field experiments, which are often forced to sacrifice externalvalidity in order to establish cause and effect (Schneider et al., 2005).

Researchers in the field of social psychology have referred to this tradeoff as the social psychologist’sdilemma (Aronson, Wilson, & Brewer, 1998). Partially because we were both trained as experimental socialpsychologists, we have been inspired by Cialdini’s (1980; Mortensen & Cialdini, 2010) challenge to the field.

343

1.2.3.

He argued that far too many social psychologists worked exclusively in laboratory research settings to testtheoretical ideas. He noted this was especially problematic when the laboratory setting was artificial and overlycontrolled. Instead of trying to capture effects in the lab, he suggested that research efforts were better spentstudying effects that already appear to be powerful in naturalistic settings. As an alternative approach, Cialdiniproposed the concept of full-cycle social psychology. Hypotheses about phenomena should first be derivedfrom observing those phenomena naturalistically in the real world. Then, research should be conducted in acontrolled laboratory setting to determine the causes for why it might occur. Finally, verification should becontinued back in a field setting, which often generates new hypotheses to start the cycle anew. By startingand ending in naturalistic field settings, Cialdini argued we would have a better model for theory building andtheory testing, integrating both basic and applied research to solve real-world problems.

Our colleague, David Daniel, makes a similar plea when writing to learning scientists in his field ofcognitive psychology (Daniel, 2012). In response to the call for widespread adoption of laboratory-basedresearch findings (Roediger & Pyc, 2012), Daniel argued that a systematic “vetting” process is needed to verifyif laboratory-based findings translate to actual, real-world classroom settings. Daniel proposed five steps toevaluate whether we can take a laboratory finding and apply it successfully to change practice in the classroom:

Begin with exploration in the lab to find promising findings.Move to careful experimentation in select classrooms to yield a promising principle.Develop and design classroom/teacher-friendly methods integrating the promising principle into an everydaypromising practice and to help ensure the fidelity of the intervention.Continue coordinated experimentation in more diverse and complex classroom settings to yield a teachingbest practice.Disseminate and continue refining the best practice.

Others have raised similar concerns about the broad application of laboratory findings to the field(Dunlosky & Rawson, 2012; Mayer, 2012; Pellegrino, 2012). In motivation research, Pintrich (2003) made acall for a motivational science approach that focused on use-inspired research; that is, research inspired bypractical questions, grounded in theory, and guided by systematic inquiry. Writing over a decade ago, Pintrichidentified motivation research that had established the first two steps of Daniel’s (2012) process. Pintrichidentified five motivational generalizations (step 1) that yielded 14 design principles for classroom instruction(step 2, see Pintrich, 2003, Table 2). However, motivation researchers have been less fruitful on the last threesteps of Daniel’s process, and have been unable to develop promising and best practices.

It is within this context that we highlight intervention research as a crucial methodological tool in bridgingthe research–practice gap. We do not propose that interventions should be the only focus of motivationalresearchers, nor do we argue that all interventions must be tested using randomized field experiments(Shavelson et al., 2003). Rather, as noted in the beginning of our chapter, research methods in motivationhave been unbalanced, heavily favoring observational, correlational, and laboratory designs. Further, in orderfor interventions to provide effective direction for educational practice, interventions need to be use-inspired(Pintrich, 2003) and focused on solving practical problems of educators (Berliner, 2006; Kaplan et al., 2012).They also need to be guided by theory, of which the field of motivation has plenty. In the following sections,we review the current state of motivation interventions in education, and highlight two different intervention

344

approaches that hold promise for impacting practice.

345

Part 2: Motivation Interventions in Education: An Overview

Two main approaches to interventions grounded in motivation theory exist in the research literature. First,there are targeted interventions that leverage precise psychological mechanisms to enhance subsequentlearning outcomes (for reviews, see Lazowski & Hulleman, 2013; Yeager & Walton, 2011). Targetedinterventions tend to be briefer in duration and focus on one or two components of motivation. Second, thereare more comprehensive interventions that integrate multiple motivation components and often leveragemotivation alongside specific curricular content (e.g., literacy) or pedagogical practices (e.g., cooperativelearning) to enhance specific academic knowledge and skills (e.g., Guthrie, Wigfield, & VonSecker, 2000;Martin, 2008).

Targeted Motivation Interventions

We have organized targeted student motivation interventions into four main areas that synthesize theconstructs that motivate students in classrooms. Three of the areas are adapted from Pintrich’s (2003, Table 2)review of motivation research in education: expectancy and control beliefs, interest and value, and goals. Tothese we add the fourth area of research that has re-emerged in the decade since Pintrich’s article waspublished: the psychological costs of engaging in academic tasks (Barron & Hulleman, in press; Eccles(Parsons) et al., 1983), such as the anxiety and stress that students face when they experience fear of failure orstereotype threat (e.g., Steele, 1997). Below, we offer some exemplar intervention studies within each of thefour areas, with an emphasis on interventions implemented in the field as opposed to those tested within thelaboratory.

Expectancy and Control Beliefs Interventions. In general, this category of interventions helps students feel moreconfident to learn and achieve in a specific academic context and to be in control of producing theirachievement outcomes. Within the research literature, there are a number of associated constructs that havebeen investigated under this umbrella, including the perceived competence to perform specific academic tasks(e.g., self-efficacy, competence beliefs), to obtain a specific performance level (e.g., expectancies, outcomeexpectations), perceptions of the reasons students succeed or fail on academic tasks (e.g., attributions), andhow much control they have to create a positive outcome. Although numerous theoretical approaches exist(Bandura, 1997; Eccles (Parsons) et al., 1983; Pekrun, 2006; Skinner, 1996; Weiner, 2010), the general idea isthat students who believe they have more expectancy and control over their behavior and learning are moresuccessful. For example, Weiner (1972) proposed that students attribute success and failure on academic tasksto ability, effort, perceived task difficulty, or luck. Adaptive attributions involve ascribing success to morestable factors (e.g., ability) and failure to less stable factors (e.g., effort, task difficulty). If individuals attributesuccess to a less stable factor (e.g., good luck) or failure to a more stable factor (e.g., lack of ability), then theywill be uncertain about future success.

For example, Dweck’s (1999) theory about the malleability of intelligence posits that students who have agrowth mindset (i.e., belief that intelligence increases over time by engaging in challenging learning activities)are more confident and learn more than students who have a fixed mindset (i.e., belief that intelligence doesnot change over time, regardless of effort or experiences). By helping students understand that being