HOME Star School Information General Information
Distance Learning Home Creating Connections K-12 Students Adult Learners Educators Our Library
Our Library / BACK
Evaluation of Star Schools Projects: Part 2
(4 parts)
Use of Technology
For the Star School projects funded in the first two cycles, the dominant distance learning technology was via satellite. Districts or schools used satellite dishes as downlinks to relay live studio broadcasts into the classroom(s) Project staff members do not know the extent to which satellite broadcasts were taped and not viewed live, although estimates range from 50 percent to 90 percent.
To enhance interactivity, each project provided an "audio bridge" --a single or series of telephone lines to connect classrooms directly with the studio teacher during the broadcast. Most (but not all) participating classrooms had a telephone. Some projects used the audio bridge on a first-come-first-served basis, while others rotated systematically among schools so that each school or classroom would be guaranteed "on-air" time. The extent of interaction often depended on the initiative of the on-site facilitator. The amount of interaction also was a function of the total number of schools enrolled in the course or module; the larger the enrollment, the less access to on-air time.
Some projects also used the audio bridge immediately after a program aired, during specific "office hours" for the studio teacher and/or during prearranged tutorials with other staff. Audio-bridge time also was enhanced during special event broadcasts so that incoming calls were directed to several professionals in addition to the studio teacher.
The extent to which other technologies were used is in part related to program objectives. Whole- course instruction always used satellite technology that was usually enhanced with an audio bridge, computer networking (particularly electronic mail), compressed data transmission equipment, or fax machines. Supplemental instruction designed to enhance classroom teaching typically used more varied technologies.
MCET provided a menu of multiple technologies from which participating middle school teachers chose that was most applicable to their school, and all schools were provided with the needed equipment to use the technologies. With a variety of technologies available and ample on-site technical assistance provided, the patterns of use by MCET teachers may inform future educational applications of technology.
The MCET report on teachers'' use of technology indicates heavy reliance on those tehnologies that teachers themselves could schedule into their classroom day - computers telephone and television . Satellite programming, althought a major focus of the MCET Star Schools project, was used far less intensively than any computer-related technology. It outranked only the fax machines and the laser disc player in frequency of use. The uses of the laser disc player may be an underestimate because some teacher indicated that with so much to learn during the first year of operation, tjey had postponed learning how to use the laser disc player until after the grant period ended.
TERC offered a variety of computer-based technologies and was the only Star Schools project during either Cycle One or Two that did not use satellite programming. TEAMS' technology more closely resembles that of whole-course instruction; it provides satellite programming with an audio bridge and extensive classroom materials for its supplemental math and science modules.
Satellite-based and computer-based technology applications differ in two fundamental ways. First, the satellite-based distance learning activities are generated from a central point and sent out to receiving sites while computer-based applications frequently included information sharing that begins at the school site Second, satellite-based distance learning activities are scheduled by the provider. When school personnel wish to use such programming live, they must schedule other courses and activities around the broadcast. In contrast, the use of computer-based technology can be scheduled within a classroom. These differences seem to influence the choice of technology at the project level and how Star Schools distance learning is implemented at school sites.
Three student-level effects of distance learning are of interest. First is the effect of distance instruction on learning. This can be demonstrated through outcome data such as passing grades in the courses, gains in achievement, and good performance on tests. Second is the effect of distance learning on student interest and motivation, for which only indirect outcome indicators are available. Third is the effect on student work as evidenced by special products.
Four projects did not collect outcome data of any kind. Project staff referred to the need for longer treatment periods before one could expect to perceive impact on students. Two projects gathered information on student outcomes using a single method of assessment and the other two used two methods.
The project reporting grade distributions offers whole course high school instruction in AP level subjects, foreign languages, and specialized math topics. In this project, 88 percent of the distance learners earned a C or better in the broadcast course. Almost half (42 percent) of the students earned As. No comparison data are provided.
In both projects that used comparison tests, project staff members remain dissatisfied with the test. In one case they believed although offering courses in the same content area, traditional classroom instruction and the distance learning instruction were not actually comparable on a specific level of subjects covered. Students in distance- earning classes demonstrated lower levels of performance in all areas tested. In the second case, distance learning groups performed better than their traditional classroom counterparts. However, items were changed between the pre- and post-test period, and 40 percent of the students did not finish the tests in the time intended, but returned to complete them . A third project used pre- and post-tests, demonstrating a 45 percent gain. Comparison groups were not used. One site looked at norm-referenced test scores of the participating Chapter 1 students and found gains for 16 of the 24 middle school students.
Teacher feedback was collected through formal interviews and/or surveys by two projects. In both instances, teacher reports were favorable. Teachers in one project reported that academically disadvantaged students were better served distance learning than by traditional classes. They also reported that minorities and learning disabled students had higher rtes of attendance during distance learning module days. Teachers in the other project also reported high rtes of attendance, but did not differentiate among types of students most affected. Teachers also reported that the hands-on approach resulted in observably higher levels of student motivation during broadcast instruction.
Student feedback also was elicited by two projects. Students in one project reported no significant differences between distance learning and traditional classroom learning: class cohesiveness, goal direction, teacher characteristics and skills, and teacher support. The other project had students complete course-specific evaluations. Across courses, 43 percent of the students said they believed they learned as much in Star Schools classes as they would have learned in traditional classes.
In addition, anecdotal illustrations of successful distance-learner performance were offered by a number of teaches, principals, and the project staff across all eight projects. For example, in two projects, AP studio teachers reported that they believed AP distance learners were passing AP tests at the same rate as those taking traditional classroom AP courses. The staff of one project pointed to the fact that three distance learning foreign language students were among the top 10 finalists in a statewide Japanese competition. Another foreign language project reported project-taught students "sweeping" the site language competition for five years. Another project reported spontaneous student use of languages learned via broadcast.
Anecdotal information also is used to indicate strength of motivation among distance learners. One site reports that students drove up to 35 miles each way to receive broadcast instruction in algebra during the summer. Another site reports that students voluntarily gave up 15 minutes of their daily lunch period to receive broadcast instruction. Classroom teachers at several sites also stated that student attendance rates were better on broadcast days than otherwise. They attributed the increase in attendance to the teaching methods used during the broadcast module.
Outcome data are sketchy. However, what data are available suggest that for at least some courses, learning takes place, students are motivated, and work is accomplished. This also is the general opinion offered by project staff, principals, and classroom teachers and facilitators.
Those programs offering supplemental instruction, most notably TEAMS and MCET, were intended to change teacher' attitudes and behaviors in the classroom. Both TEAMS and MCET conducted surveys of participating classroom teachers on changes in instructional practices. In TEAMS, a strong majority (86 percent) of teacher responding to the internal evaluation survey reported using different instructional materials as a result of participating in the distance learning instruction.
The internal evaluation surveys of MCET middle school teachers do not reveal appreciable changes in their views on science teaching and learning. They already supported hands-on science, interdisciplinary teaching, and cooperative learning prior to participation in project activities. Teachers appeared philosophically to have already taken a cognitive or constructivist approach to science education. However, teachers reported changes in their classroom behavior through participation in the Star School activities. They indicated an increase in the use of multiple technologies in the classroom (95 percent of teachers), innovative ways of teaching science (85 percent), interdisciplinary teaching (71 percent), cooperative learning (68 percent), hands-on science (57 percent) and team teaching (47 percent). Teachers also reported changes in classroom management strategies, with more emphasis given to organizing classes with small group activities (85 percent of teachers) and less given to lecturing to the whole class (69 percent). Classroom observers noted that the multiple technologies and hands-on activities created interest among students and that students were more responsible for their own learning. Observers also noted that teachers still had a long way to go in asking students for explanations, having students use previous experiences to explain concepts, and planning how to group student to ensure maximum learning (Drexler & Kapitan, 1993).
Although there is self-report of changes in attitude and behavior, in neither project were changes reported in the staffing patterns in the schools.
Those distance learning projects providing whole-course instruction did not explicitly intend to change individual teacher attitudes or behavior. The adults in the classroom often were non-teaching staff whose primary responsibilities in the distance learning classroom were for managing the classroom and distributing materials. In a few schools, there was anecdotal information that some facilitators who were already certified teachers improved their teaching skills, and there were reports that certified math teachers who supervised calculus courses became comfortable enough with the material to teach independent of the distance learning course.
Star Schools funded activities had two types of effects on curriculum and instruction within receiving school. First, it broadened curriculum offerings and, in specific instances, built local capacity to provide instruction in a greater array of courses. Second, Star Schools projects sponsored activities designed to improve instruction by providing models and an array of curriculum materials, and these had different levels of impact. Following is a discussion of each type of school-level impact.
Although the major purpose of activities that increased the number and types of courses offered was to provide students with increased educational opportunity, instances exist in which the courses increased local capacity for curriculum breadth. For example, the Midlands project required a certified mathematics teacher to serve as the classroom facilitator for the calculus course. The project reports approximately 10 cases in which the classroom teacher became comfortable enough with the subject to offer the course independent of distance learning.
The more significant impacts on school curriculum and instruction were intended from activities designed to influence how teachers and students interact during regular classroom instruction. The actual effects, however, varied according to the design of the activities and how well the met local needs. For example, a major "success story" came from a site that had been searching for ways to integrate curriculum and revitalize long-term teachers. The school had little money for materials (less than $2 per pupil per year for science materials), but had a group of teachers who were looking for ways to stimulate themselves and their students. The teachers wrote the required proposal to the Star Schools project for participation and took "everything we could get our hands on." During the project years, they implemented most of the offered activities, thereby changing science instruction to a "hands-on" approach. After funding ceased, they m modified some activities and continued others. Perhaps most strikingly, one teacher had postponed implementing some activities during the grant years, felling unprepared to tackle the full array of project activities, but planned to implement them subsequently.
School staff in other projects reported that non-participating teachers borrowed materials and approached curriculum changes in ways that were congruent with the Star Schools activities. Because the schools also were participating in other site and district programs designed to increase cooperative learning and problem-solving approaches to teaching, it is impossible to determine how much the existence of Star Schools activities contributed to teachers' interest in the materials. Nonetheless, the availability of materials was helpful to local efforts.
Activities directed at changing instruction in regular classrooms can provide models of instruction and supplement existing curricula. However, they face a dilemma. On the one hand, what is known about teacher development (Loucks-Horsley & Associates, 1990; Joyce & Showers, 1988) requires close interaction between the staff development provider and the classroom teacher. On the other hand, the strong argument made for using distance learning technologies for staff development focuses on its low cost per teacher. That is through distance learning, large numbers of teachers can be reached at a single time. However, if the studio teachers, who provide model of instruction, also are expected to coach the classroom teacher, they must have knowledge of how each teacher understands and uses the preferred modes of instruction . The needed intimacy, then, may require a limited number of participating teacher, unless the distance learning is accompanied by on-site support. But either limiting the number or providing on-site assistance means that the per-teacher cost may be no lower than other approaches to staff development. As a result, using distance learning as a vehicle for improving instruction may not be cost effective when compared to other ways to accomplish the same objective
1. How effective are telecommunications partnerships and programs after federal funding ceases?
Three projects funded by the Star Schools Program currently receive no Star Schools funding. Two offer primarily whole-course programs and are dependent on enrollment for continuation. One has sufficient demand, although not for the courses developed with Star Schools funds, to continue a strong presence in the distance learning community; the other project has shrunk considerably. The third project, which relied on computer-based technologies and developed materials to supplement the curriculum, no longer provides distance learning activities. However the computer-based curriculum materials are used by an other project, and the organization developed a more advanced computer networking program a result of the problems experienced in Star Schools. In all three instances, the partnerships did not survive the loss of funding.
Currently, funded Star Schools projects are working toward developing a diversified funding base so they could continue if Star Schools funds were no longer available to them. The most common form of diversification is requiring dues from participating states and school districts. Projects also actively seek additional grants and contracts. Although one project encourages schools to use federal and state funds to support Star Schools activities, project staff members express concern about the longevity of such funding.
2. What non-federal funds are leveraged by Star Schools projects? How stable is such funding?
Star Schools projects derive funds from a variety of sources. The most stable sources seem to be dues paid at the state level. When projects rely on local school or district contributions, they are quite susceptible to changes in the fiscal environment. Most projects actively seek funds from federal agencies in addition to the OERI, and in fact, most non-Star Schools funding comes from other federal sources and is short term in nature.
3. How did Star Schools grantees spend funds generated by the Star Schools Assistance Program?
Across the projects from which information was obtained, the largest portion of funds (35 percent) was spent on equipment. Fiscal agents distributed an average of 28 percent of their grants to partners, including state, local, and regional educational agencies, institutions of higher education , not-for-profit organizations, and profit-making companies. On average, 21 percent of funds were spent to develop and disseminate programming, and 17 percent was spent on administration.
The investment in equipment helped build an infrastructure for distance learning activities. Consequently, it might be possible to devote more funds to programming than to equipment.
4. What subject matter was offered through Star Schools/ With what effectiveness? What student learning outcomes occurred?
Star Schools projects offered two types of programs. The first were whole courses, mainly in high school subject. The second were supplementary courses or modules offered at all levels of elementary and secondary school.
Foreign languages (including French, German, Japanese, Latin, Russian, and Spanish) comprise the largest number of full-course offerings (31), and 11,630 students currently are enrolled in distance learning foreign language courses. Science course serve 1,552 students; and mathematics courses, 939 students. Over 1,583 students currently are enrolled in AP courses, including math and science, through Star Schools distance learning.
Projects do not keep good data on enrollment in modules and supplementary courses. However, a conservative estimate is that a total of 140,000 students currently are involved in Star Schools distance learning activities. This report relies on data gathered by projects, which were not consistent in how they assessed effectiveness. What little evidence, from two projects, exists seems to indicate that Star Schools distance learning is as effective as other means of instruction. Further, it provides opportunities for students particularly in rural areas, that would otherwise not exist.
5. What general staff development was offered by Star Schools projects? How effective was it?
General staff development was perhaps the weakest component of Star Schools projects. During the 1992-93 academic year, Star Schools projects sponsored 130 in-service "events," ranging from one-half hour workshops to training programs lasting 10 sessions. However, a few projects collected data about participation, and none collected data on effects. Most projects believed the general staff development activities were severely under used.
Two Star Schools project identified staff development as the focal point of their activities. In them, programs and materials delivered to students were designed to change teachers'[ attitudes and behaviors in teaching science, and in one case, math as well. The purpose was to demonstrate hands-on, constructivist approaches to instruction and provide materials that supported such pedagogy. One project included extensive staff development, including in-person workshops, one-week intensive seminars, and teleconferences. The project also provided substantial on-site technical assistance. Teachers who participated in the project did not change their attitudes as many came with a philosophical commitment in line with project goals. They did, however, report changes in how they actually worked with students and the materials they used. The other project provided fewer supports. Nonetheless, teachers reported changes in their own attitudes and behavior.
6. How did issues related to teacher certification affect Star Schools projects?
In general, issues related to teacher certification were not salient in Star Schools projects. The issue only arose in full-course programs. Most projects used certified teachers as studio teachers and worked with participating states to gain certification for them. One project relied heavily on university professors for whole-course teaching but required that a receiving site classrooms be supervised by a certified teacher. It recommended, but did not require, that the teacher be certified in a related field. The project ran into problems in one participating state.
Star Schools-sponsored supplementary instruction was received in regular classrooms that were staffed by certified teachers. Therefore, there were no concerns about the certification status of studio teachers.
7. What were the effects of distance learning on curricula and staffing patterns at participating schools?
In general, Star Schools activities did not affect curricula and staffing patterns at participating schools. However, two outcomes occurred in a limited number of school. first, a few schools reported that math teachers who supervised distance learning calculus classes became confident enough to teach calculus at the site. Second d, one Star Schools project fostered interdisciplinary instruction. In its most successful sites, team teaching and other forms of faculty collaboration were reported to increase.
The major effect of Star Schools activities was to broaden opportunities for students to take a variety of courses, particularly in math, science, foreign language, and in all AP subjects.
8. How many students participated in Star Schools activities? What were their characteristics?
Based on project documents, supplemented by information gathered in site visits and follow-up telephone interviews, a conservative estimate is that 140,000 students currently are participating in Star Schools distance learning activities. The students live in 48 states.
Less clear is the degree to which disadvantaged and traditionally underserved populations are being reached through the Star Schools Program. Projects do not tend to collect data concerning students' gender, ethnicity, or poverty levels. From the limited information available, it appears that 58 percent of students served are minorities, 40 percent are educationally disadvantaged, and 77 percent are in Chapter 1-eligible schools. (However, there is little information about how many students participating in Star School are actually receiving Chapter 1 services.) The mail survey to schools that is planned for Phase 2 of the study will systematically collect information on these issues.
9. What are the socioeconomic and geographic characteristics of schools participating in Star Schools projects? What types of programs are provided to different schools?
Again , project-generated data are incomplete regarding the types of schools participating and the types of Star Schools-sponsored activities they receive. In general, it seems that urban schools are more likely to receive programs designed to supplement regular instruction than they are to receive whole courses. Rural schools seem equally likely to receive both types of programs. In absolute numbers, there are fewer rural than urban school, largely because two projects involve numerous schools in the largest school district in the nation.
Of all policy decisions, the one that gave projects and receiving schools the most problems was the two-year funding cycles. For most projects, the first year of the grant was spent acquiring and installing equipment, recruiting sites, and developing programs. Consequently, schools did not begin active participation until the second year, and often experienced normal problems of implementation. The problem was most difficult for projects that used a variety of technologies or engaged in a great deal of curriculum development, but it affected all projects.
Even projects offering whole-course instruction that was fairly traditional (AP course, foreign language courses, and high school math and science courses) found they could not solidify their market with a two-year funding cycle. One reported that it as willing to use Star Schools funding as seed money, but it needed more time to convince potential "buyers" of the value of the product.
The two-year funding cycle had particularly negative effects on projects that viewed Star Schools as an R&D opportunity. The time limitation pushed projects to deliver services to participants as early as possible, without adequate time for field-testing or limited market tests. The short funding period provides a perverse incentive for projects to work with schools that need only equipment, not facility changes; courses that are readily developed and therefore fairly traditional; and with staff that accept technology. The incentive, in sum, works against the goal of equalizing educational opportunity.
Schools visited during the 1992-93 academic year were selected by projects as examples of "best- case" implementations of Star School activities. Staff within these sites, while praising Star Schools project staff, raised the question of ongoing support.
In part, the two-year funding cycle fits a view of Star School as a seed money program, particularly if the seed money is allocated to equipment. However, all projects engaged in some program development. Further, even funding on equipment required projects to train staff at the receiving site, and the two years were inadequate to the task.
reposted 6/26/98