09 July 2008
New Report Teaches Teachers to Evaluate Online Learning
Rapid growth in online education requires culture of evaluation
Washington -- About 1 million U.S. elementary and high school students were taking online courses in 2007 -- 22 times the number enrolled in such courses in 2000. But if two researchers writing in the summer issue of Education Next are correct, the growth in online education to date is nothing compared to what is about to happen: by the year 2014, some 10 percent of all U.S. kindergarten to grade 12 courses will be computer-based, say Harvard Business School’s Clayton M. Christensen and his co-author Michael B. Horn, and by 2019 a full 50 percent of all K-12 courses will be online.
These changes in American education will be driven by the low cost of delivering online courses and the advantages that accrue from individualized instruction, say Christensen and Horn. Online courses also can be geared toward the special needs many school systems currently cannot fill, such as foreign language courses and Advanced Placement courses, or the needs of growing numbers of children schooled at home.
But if online education is to fulfill its potential, accountability will be key. To that end, the U.S. Department of Education July 2 released a new report, “Evaluating Online Learning: Challenges and Strategies for Success.”
“Evaluating Online Learning” is essentially a guide for educators, some of whom are seeking to design distance learning courses, virtual schools and educational Web sites, and all of whom need to be able to use such resources since every U.S. public school is connected to the Internet.
Part I of the report focuses on seven exemplary online programs that represent a range of options, from online courses to Web sites that feature education resources.
Each of the seven programs surmounted challenges to evaluating online learning and offer potentially useful case studies for other programs.
The seven featured programs and resources are: Alabama Connecting Classrooms, Educators, & Students -- Statewide Distance Learning, operated by the Alabama Department of Education; Algebra I Online, operated by the Louisiana Department of Education; Appleton eSchool, operated by Wisconsin’s Appleton Area School District; Arizona Virtual Academy, a public charter school; Chicago Public Schools’ Virtual High School; Digital Learning Commons in Washington state; and Thinkport, a Web site operated by Maryland Public Television and the Johns Hopkins Center for Technology in Education.
The analyses of these programs illustrate some of the traps and opportunities of which evaluators must be aware.
The online field trips offered by Maryland’s Thinkport Web site, for example, turned out in the final randomized controlled trial to have taught students more successfully than they would have learned by traditional instruction. But the initial trial indicated the online trips did not have a significant effect on student learning. Only after teachers had gained experience with the online field trips and were using them a second time did the learning tool achieve its potential.
Students participating in Louisiana’s Algebra I Online program outperformed students taught by traditional methods, but the online students were more likely to report not having a good experience and were less likely to report feeling confident about their algebra skills. The evaluators speculated that online students need to get more immediate reassurance from a teacher or mentor that they in fact are learning and understanding the material.
Chicago’s Virtual High School found it had to use evaluation findings to persuade reluctant administrators and principals to provide mentors for students. Compelling data showed “students need help with pacing, study skills, and troubleshooting the technology; without this help many were failing.”
In other words, online learning is not simply a matter of plunking kids down in front of computers and telling them to get to it. The report raises a number of important questions for researchers. These questions range from the factors that may increase online course success rates and the characteristics of successful online learners to the training, mentoring and support systems necessary for teachers to be effective.
Part II of “Evaluating Online Learning” offers recommendations to educators -- recommendations for everything from evaluating the multifaceted online resources to translating evaluation findings into action.
One of the key recommendations is to “develop a program culture that supports evaluation.” In most of the case studies, early data were used to improve the program in specific ways.
Already 42 states have significant online learning programs and some 57 percent of public high schools in the U.S. provide online learning, according to the North American Council for Online Learning, an international nonprofit organization. There are also 26 statewide or state-led virtual schools in the United States and 173 virtual charter schools in 18 states.
“Evaluating Online Learning: Challenges and Strategies for Success” (PDF, 80 pages) is available on the Web site of the U.S. Department of Education.
