|
|
FIPSE-Sponsored Assessment:
Wright Community College Great Books Curriculum 2003 - 2004
Prof. Bruce Gans
Wilbur Wright College This report concerns the progress made during the first year of a three-year FIPSE Great Books Project. The original Great Books grant proposal called for a three-year collaborative effort which at its conclusion would create Great Books Curricula at five partner community college institutions. These Great Books Curricula would be fully integrated into the core curriculum in such a way as to enjoy ongoing and indefinite institutional support. The curricula would function to raise and enhance the learning and learning skills of the colleges’ large populations of nontraditional, underserved, and minority students. Additionally, the grant planned for the dissemination of the project’s Great Books pedagogy through a consortium-created National Great Books web site, thereby achieving a wide and ongoing impact beyond the life of the FIPSE project itself. This web site would present the project’s research and resources in order to educate, encourage, and serve similar institutions across the United States. Assessment activities were assigned a useful but supporting role in this effort. FIPSE mandated that the first year of the grant was to focus entirely on assessment of the Great Books Curriculum at Wright College. The objective for the first year thus became to investigate and document the original Great Books Project grant’s assertion that Great Books pedagogy enhanced learning and learning skills among underserved, nontraditional, and minority students. The objective was to verify through neutral measurement instruments the special value of Great Books pedagogy. This would in turn serve to validate the value of implementing Great Books Curricula at partner institutions and elsewhere in the future. At the same time, this assessment work had the objective of evolving portable assessment tools for institutions to use as they implement their own Great Books Curricula over the next two years and beyond. The assessment activities were also to produce some self-study that would tell the Great Books project staff, as well as Great Books faculty at Wright and elsewhere, how a Great Books Curriculum like the one at Wright Community College might improve and upon what it ought to build. These goals have been accomplished, as will be detailed below. Over the first year of the project, then, several means were used to assess the Wright College Great Books Curriculum. One was the administration of the College Assessment of Academic Proficiency test (CAAP test), devised by ACT, to a sampling of Wright Great Books students and non-Great Books students alike. The test was given by Wright College as part of its own internal NCA assessment efforts. The CAAP test was in fact given in partnership with the FIPSE Great Books Project, which underwrote a portion of the expense in exchange for having access to special aspects of the data collected, some of which came at this project’s request. The five thousand dollars contributed by the Great Books Project itself represented something of an accomplishment because it was a significant cost savings. The purchase price for such an instrument for the Great Books Project would otherwise have been three times that sum, i.e., fifteen thousand dollars, not including the extra fees and obstacles in arranging for the administration of such a test. The purpose of the CAAP test is to measure student proficiency in general education areas such as reading, writing, and critical thinking, among others. The Great Books Project staff set themselves the task of comparing student Compass test scores, which students achieved prior to entering Wright College, with the CAAP scores. In addition, the Great Books Project staff compared the reading and critical thinking scores of those who had taken one, two, three, four, and more Great Books courses to those who have taken none. The Great Books Project staff also investigated a comparison of grade point averages between those who had taken four or more Great Books courses and those who had not participated in the program. This number of four Great Books courses was chosen not only because Great Books pedagogy, like education itself, is incremental and cumulative in its impact (and is not a magical, instantaneously transformative pill), but also because under the Great Books Curriculum, students earn a Great Books certification on their transcripts if they have taken at least four Great Books Curriculum courses—the equivalent of a full semester’s academic work, or 25% of the courses necessary for graduation with an associate’s degree. The CAAP test was administered in spring of 2004. We are pleased to report that the data demonstrates a significant increase in reading skills in minority, nontraditional, and underserved students who took at least three Great Books courses over those who did not. Interestingly, the CAAP test also showed that Great Books minority students who had taken at least three Great Books courses improved significantly over Great Books students, both minority and white, who had taken two or fewer such courses. Clearly, the more engaged a minority student is in this pedagogy, the more likely the student is to benefit from a series of steady, incremental increases in skill and knowledge. Professor Roger Connor, who was in charge of the data analysis, reports that "the students who had taken 3 or more GB courses averaged 63.2 against 59.9 for those who had taken exactly 2 GB courses." After running the significance tests, Professor Connor determined "that is a significant difference at the 95% criterion. (i.e., there is only 5% chance that such a result could occur by chance.)" Professor Connor "used a t-test, although with samples this large a z test comes out virtually the same . . . The average number of hours for the 3+ group was indeed higher (39.0 to 35.5). But because the standard deviation was very high in both groups (about 17), the difference in number of hours was not significant (p = .26, whereas it needs to be less than .02 to be significant)." Gratifyingly, the analysis of Great Books courses’ effect on the General Education grade-point averages of Great Books students and non-Great Books minority students revealed that minority students who took four Great Books core curriculum courses saw a significant increase of nearly half a grade in their General Education GPA. Dr. Barshis’s report is worth quoting at length here:
GBC students in general achieved higher average GPAs in their General Education courses than did the non-GBC students in the control group (2.87 vs. 2.45, nearly half a letter grade higher). Minorities as well as whites scored at a higher level if they were part of the GBC (2.73 for minority GBC students vs. 2.38 for minority non-GBC students; 3.03 for white GBC students vs. 2.58 for white non-GBC students). Hispanics (who made up 44% of the GBC group) also scored at a higher level (2.70) when compared to Hispanics in the non-GBC group (42% of the sample), who scored at a 2.25 level. The hypothesis that participation in the Great Books Curriculum at a significant level (4 or more courses) should help students perform more effectively in their General Education courses where the GBC-emphasized skills of critical reading and thinking, effective writing, and cultural literacy are useful seems validated by this comparison. And the data on minority students, especially Wright’s large population of Hispanics, shows that involvement in the GBC allowed them to match or exceed the performance of their white non-GBC counterparts in General Education curricula, by far the largest component of required course work in community college degree programs.
Counter to expectations, Professor Connor preliminarily found no significant differences in critical thinking scores among Great Books and non-Great Books students. It should be noted that the CAAP data was not returned to Wright College until mid-May, and other necessary data arrived only in June. Hence, returning in more depth at a later time to this area may show another result. Nevertheless, if this finding stands up, it will constitute an important area where faculty must devise new strategies to reach a common definition of what constitutes critical thinking, and revise their pedagogical repertoire to achieve gains in this vital area of Great Books pedagogy. One possible approach may be a greater reliance on the "shared inquiry" form of approaching core texts. Collaboration with dedicated Great Books institutions such as the Great Books Foundation and Shimer College, already a consulting institution on this project, is another seemingly highly useful resource in this area for improving teaching and learning. While much good work has been done, the Great Books Project is eager to pursue further elements of this research on an ongoing basis as part of the continuing work on evaluation and assessment to which the project is committed. Another form of inquiry concerned self-reporting by a sampling of Great Books Curriculum students, both minority and white, on their own experience of the impact the Great Books courses had on their reading, writing, critical thinking skills, and other forms of learning. This survey was developed by Dr. James Palmer, a Professor of Sociology at Illinois State University. Dr. Palmer is a nationally recognized expert on community college education, and he conducted his study with his colleague Professor Dianne Gardner, also of Illinois State University. Professors Palmer and Gardner designed the questionnaire by interviewing a cross-section of Great Books Curriculum students about what impact they believed the pedagogy had on them, and then selecting the most commonly occurring responses to constitute the questionnaire. There was also a section at the end of the questionnaire for students to write whatever reflections on the pedagogy they felt like sharing. A third form of assessment was designed to improve the teaching of Great Books. It was conducted by Palmer and Gardner and consisted of in-depth interviews with Great Books faculty about their experience with Great Books pedagogy. Palmer and Gardner’s study validated the practical direct value of the Great Books Curriculum for minority students, as well as its long-term indirect value. Their study supported the Great Books Project’s assertion that minority students experience greater academic self-confidence and feel greater comfort and interest in taking more complex and challenging curricula; and they do so with the intention of pursuing a four-year degree, instead of being temporary, intermittent course-takers. While the report is included as an appendix to this report, it is worth quoting those sections that pertain directly to the objectives of the first year of the grant. Students responded to survey statements as follows, rank-ordered by the percentage of respondents who "strongly agreed" with the following statements:
[Editor’s note: Significantly, when these percentages were combined with those of students who expressed the next nearest strength of agreement, the numbers were overwhelmingly positive.] In addition, the survey instrument included an open-ended question: Consider all of the Great Books courses you have taken (including this one). What have you gained most from your experience with these Great Books courses? Please describe below.
Responses . . . noted perceived gains in critical thinking, reading, writing, speaking in front of groups, discussing complex ideas, and listening to and learning from others . . . They indicated that they gained more than an increased level of skill; they also noted changes in the ways they used their skills. Reading is an example. Responses to the open-ended question indicated enhanced reading ability in terms of:
The responses also indicated that the students perceived that they were:
The Great Books Project staff is also excited by the opportunities for increasing the impact of Great Books pedagogy that has emerged as a result of Professor Palmer and Professor Gardner’s findings as well as that of the CAAP test. In the former case, Palmer and Gardner made an important and entirely unexpected discovery. Their investigation indicated that roughly half the students taking a Great Books course were not aware of that fact. What makes this finding so surprising is that each Wright College Great Books course is clearly indicated as such in every course schedule used by students to sign up for classes. The Wright College Great Books Curriculum also works with the College Advisors, who work closely with innumerable students every semester in formulating individual student programs. This process includes distributing leaflets and brochures to individual students explaining the Great Books Curriculum and highlighting not only the Great Books courses themselves but their reading lists as well. Large and elaborate posters listing each Great Books Curriculum course and its reading list are posted throughout the school, at all the elevators, on all the bulletin boards, and at the entrance to the Dean of Instruction’s office. Moreover, the school library dedicates two walls inside its entrance to permanent framed displays of Great Books Curriculum faculty and to the national press coverage and grants earned by the Wright College Great Books Curriculum. And of course, Great Books Curriculum courses are taught by a twenty-member faculty as well as by a half-dozen adjuncts. The reason this finding presents such positive chances for the improvement of teaching and learning are as follows. First, the CAAP tests, as well as the survey Professors Palmer and Professor Gardner conducted, have shown that student learning and skills increase whether or not students are aware they are in a Great Books Curriculum class. As Dr. Barshis expressed it to the FIPSE project staff, Palmer and Gardner’s findings show the value of Great Books pedagogy in achieving the objectives of the grant. Barshis writes of the study that "ordinary community college students, many of whom are minorities, who encounter Great Books courses - often without realizing they're in them - achieve the program outcomes built into the curriculum - better reading, realization of their entry into the ranks of the educated, improved critical and cultural literacy, and greater self-confidence. Also to be noted is the realization by faculty of the program’s uniqueness as a curriculum of a higher order, even though the faculty are not always able to sell the students the full realization of what that means." Since the assessment instruments used during the first year of the FIPSE project have shown that students who actively pursue Great Books Curriculum courses demonstrate greater gains in reading skills, General Education grade-point average, and academic satisfaction and self-confidence than those who do not, it seems fair to assume that if the Great Books Curriculum faculty can improve upon their ability to inform their own students about the program and what it offers them throughout the semester, a higher percentage of students will be motivated to actively pursue this pedagogy. Plans under way to achieve this goal include a meeting of the Great Books Curriculum faculty with Professor Palmer to discuss these results, explore possible reasons for the lack of student awareness, and consider ways in which more effective communication and reinforcement can be developed. Plans are also under way to distribute a packet of materials to students during the first day of Great Books classes, along with instructions on the importance of reviewing them in class. Professor Palmer has also made several valuable recommendations for the improvement of teaching and learning that will be enthusiastically taken up in a series of Great Books faculty meetings in the upcoming semester. Professor Palmer’s discussion is worth quoting here at length:
The first [recommendation] is the establishment of an internal assessment plan that begins by articulating and then measuring shared learning outcomes across the courses. The four stated program outcomes for the Great Books Curriculum can be a starting point. The assessment plan can refine the definitions of those outcomes in ways that help faculty members develop teaching strategies that will lead to those outcomes. For example, if critical thinking is a desired outcome, the plan should offer a common definition of critical thinking that will be applied across all Great Books courses and help faculty members understand where they should be leading students.
The second step, taken in tandem with the first, entails collaborative faculty work on the identification and description of pedagogical practices that lead Great Books students to desired ends. The focused group discussions clearly indicated that those teaching Great Books courses have a wealth of knowledge and experience about instructional practices that can engage students. These practices should be shared internally and externally and then refined as part of the ongoing effort to assess learning outcomes and evaluate the program. This second step complements the first, helping to define the curriculum not only in terms of desired outcomes but also in terms of the processes and experiences that will lead to those outcomes. The impact on educational practice of the FIPSE Great Books Curriculum can be measured in the progress of the five partner institutions involved with Wright College. Today all of these community colleges have an identifiable set of staff prepared to form a Great Books Curriculum. In every case but one the faculty is drawn from across the curriculum. The administrations at these institutions are committed to identifying Great Books courses in their course schedules, supporting student recruitment into these classes, and identifying the students so that their progress may be tracked over time for purposes of assessment. At all of these institutions, core courses are being revised to contain at least half of their primary readings from a Great Books core list and will be coordinated with a cross-curricular theme. This is being done in consultation with the FIPSE Project Director, who is also supplying materials used to form and grow the pedagogy and programs from Wright College’s Great Books Curriculum. These materials range from sample faculty recruitment letters to DVDs of best practices in classroom teaching and student Great Books public presentations. Moreover, the goal of creating portable assessment tools has also been accomplished. The CAAP test, of course, has its own independent existence, and it can be employed by the partner institutions just as was done by the Wright College Great Books Project. Moreover, by coordinating the administration of the exam with the NCA needs of the larger institution, as happened at Wright, this evaluation can be done (as we have shown) at a comparatively minimal cost. In addition, the survey devised by Professors Palmer and Gardner for purposes of self-study, as well as the methodology for the faculty survey, will be made readily available to partner institutions. Before closing, though, it is worth mentioning some data that reflects on the Great Books Curriculum model’s effectiveness with minorities and faculty development, and its appropriateness as a model for dissemination among institutions that serve nontraditional, underserved student populations:
Appendixes
Appendix A: Comparison of Grade Point Averages between Great Books Students and Non-Great Books Students The Study This study involved a comparison of Great Books Students (4 or more GBC courses, at least 8 General Education courses) with non-Great Books students (no more than 1 GBC course, at least 8 General Education courses) on General Education grade point averages (GPAs). Assumptions for the Study If participation in Wright College’s Great Books Curriculum involves GBC students in a more intensive and focused set of reading, writing, and critical thinking activities as they encounter Great Books authors in original texts (supplemented by a rich array of extracurricular enhancements), then it is reasonable to assume that these students will develop the kind of collegiate academic skills that will enhance their performance in other courses in their General Education core of 12–15 classes. Wright students who do not participate in the Great Books Curriculum but who have completed a substantial portion of their General Education requirements should offer a fair comparison group to help assess the effectiveness of the Great Books Curriculum on those of its students who have completed a comparable number of General Education required courses. General Education grade point averages in each group will be the point of comparison, and the study’s hypothesis is that the GBC students should perform at a higher level across their General Education courses than the non-GBC students. Since Hispanics comprise the largest ethnic category of Wright students, the performance of Hispanics in both the GBC and non-GBC samples will also be compared. Study Methodology Great Books Curriculum staff have compiled lists of currently enrolled GBC students who have completed multiple GBC courses. In addition, the Office of the Dean of Instruction at the college receives an annual report of currently enrolled students who have completed at least 45 hours of transferable credit (62 hours being the minimum required for most associate’s degrees at the college). The GBC researcher for this study used the two lists to randomly select: (a) a control group of 50 non-GBC students (having no more than 1 GBC course) who had completed at least 8 of the required 12–15 General Education courses; and (b) a comparison group of 50 GBC students (those having completed at least 4 GBC courses) who also had completed at least 8 of their General Education required courses. Social Security numbers were used to randomly identify students for each group (the original pool for each group numbered between 90 and 100), and the first 50 in each group who met the requirement of having completed at least 8 General Education courses (based on a review of student academic histories) were used for the study. Grades achieved in General Education courses were recorded, and General Education grade point averages (on a 4-point scale) were computed for each student. Student ethnic categories were then added to each record so that further comparisons between the control and GBC groups could be made. Results GBC students in general achieved higher average GPAs in their General Education courses than did the non-GBC students in the control group (2.87 vs. 2.45, nearly half a letter grade higher). Minorities as well as whites scored at a higher level if they were part of the GBC (2.73 for minority GBC students vs. 2.38 for minority non-GBC students; 3.03 for white GBC students vs. 2.58 for white non-GBC students). Hispanics (who made up 44% of the GBC group) also scored at a higher level (2.70) when compared to Hispanics in the non-GBC group (42% of the sample), who scored at a 2.25 level. The hypothesis that participation in the Great Books Curriculum at a significant level (4 or more courses) should help students perform more effectively in their General Education courses where the GBC-emphasized skills of critical reading and thinking, effective writing, and cultural literacy are useful seems validated by this comparison. And the data on minority students, especially Wright’s large population of Hispanics, shows that involvement in the GBC allowed them to match or exceed the performance of their white non-GBC counterparts in General Education curricula, by far the largest component of required course work in community college degree programs.
Appendix B: Report on the CAAP Test
Roger Connor Design If we need a reminder of the centrality of reading to a college education, we need look no farther than Carlyle’s epigram written in stone above the entrance to Deering Library at Northwestern University: "The true university . . . is a collection of books." The design of the Wright College Great Books Curriculum gives an unusual opportunity to study the difference in improvement of reading ability between students who take courses that have Great Books content and students who take the same courses with conventional content. By a fortunate coincidence, Wright College was planning to give a battery of CAAP tests (College Assessment of Academic Proficiency) just in time for us to participate. We used this data to conduct the two studies outlined below. Our expectation was that students who elected to take Great Books Curriculum sections would improve their reading abilities more than those who chose parallel sections of the same courses. The random nature of the sample was guaranteed by the fact that the college chose which courses and sections would be tested without reference to our study, and by the fact that the college has been inconsistent in reporting COMPASS scores to ACT, so that such comparisons can be made. This latter factor accounts for the relatively limited size of our sample. Study 1 We requested ACT to provide us with a "Linkage Study" which would indicate broadly the degree of improvement in reading ability from the time students took the related Compass tests as part of their entrance requirement to the college. Although the data collection from this study did not permit us to make a section-by-section comparison, the only systematic difference between the samples of reading improvement in our study is the number of courses taken in the Great Books Curriculum. There is no way to make a direct correlation between the Compass and CAAP tests, since they are always taken at different times. ACT has, however, worked out a reporting system that divides students into three groups: those that show less improvement than expected, those that show expected improvement, and those that show more improvement than expected. In our total sample of 67 students, 12 fell in the below-average group, 31 in the average group, and 23 in the above-average group, A comparison of the groups shows a clear positive correlation between the number of Great Books Curriculum courses taken and improvement in reading ability. The average number of Great Books Curriculum courses taken by the below-average group was 1.25; by the average group, 1.77; and by the above-average group, 2.34. Moreover, the last two groups had an average accumulated number of credit hours that was virtually the same (52.6 for the average group, 51.1 for the above-average group. Only the below-average group had spent a significantly shorter time in school: 46.7 credit hours.) So the reading improvement of the last group cannot be seen as a result simply of more college classes taken, but must be seen as a direct result of more classes taken with Great Books content. The achievements of minority students were equivalent to those of whites. Study 2 A second study directly compared the CAAP reading scores of two groups from classes of the same kind: second semester English composition, one of which was designated as a part of the Great Books Curriculum (with at least half of its readings from the Great Books list), and the other of which comprised normally taught sections. Although the separate sections of English 102 served as a starting point for this study, the paucity of data necessitated pooling the students and then separating them into groups, one of which comprised students who had taken two courses in the Great Books Curriculum (n = 22), and the other of which comprised students who had taken 3 or more courses in the Great Books Curriculum (n = 26). Our expectation was that the group that had taken 3 or more Great Books Curriculum courses would have a higher average reading score on the CAAP test than those who had taken only 2 Great Books Curriculum courses. The average CAAP reading score for the 2-GB group was 59.9 (standard deviation 5.1); the average for the 3-plus-GB group was 63.2 (standard deviation 5.6). Application of a t-test to this data shows that the difference in average scores is significant at the 95% criterion level (t = 2.13). Again, the two groups did not show any significant difference in the total number of credit hours taken at the college (39.0 average hours for the 2-GB group; 35.5 hours for the 3-plus-GB group - an insignificant difference with a z score of only 0.69, confirming the null hypothesis). Therefore it appears highly probable that students who choose to take at least 3 courses in the Great Books Curriculum will improve their reading ability significantly over those who choose to take fewer GB courses. Again, the gains showed by minorities were equivalent to those shown by whites. Explanation The ability to read well is at the heart of any college education, and so most college courses require reading. Why does taking three or four courses in the Great Books Curriculum result in greater improvement in reading ability than taking similar courses which are not in that curriculum? It is our view that it is not the quantity of reading that matters, but the quality. Works that have achieved status in our culture have done so because they provide a broad intellectual perspective that spans generations - even eras. Contemporary writers, consciously and unconsciously, write within and draw on this treasury of perspectives. The extent of a modern reader’s grasp of this humane heritage determines, in large measure, how fully he or she can see the significance of the work of any writer (and this includes most writers of merit) who makes express or implicit reference to this vast body of well-considered ideas. Our conviction that students who take courses in the Great Books Curriculum will show more improvement in reading ability than those who do not seems to be well supported by these studies.
Appendix C: A Formative Evaluation of the Wilbur Wright Great Books Curriculum
Professor James Palmer and Professor Dianne Gardner Introduction The Great Books Curriculum at Wilbur Wright College offers courses in a variety of academic disciplines. Each course devotes at least half of its readings to works from a list of "the best that has been thought and said." The Great Books faculty is committed to helping students understand and enjoy these works. The courses are also intended to enhance students’ "self-image and sense of achievement," improve students’ overall academic performance, and improve students’ abilities in reading, analytical thinking, and "cultural literacy." The curriculum offers a certificate after 12 hours of Great Books courses. Students also have the opportunity to participate in a variety of co-curricular activities, including internal publications, student and faculty symposia, and fine arts performances. The Wright Great Books Curriculum has attracted national attention as other community colleges seek to understand the program and duplicate its successes. This report provides an initial snapshot of the program as it is experienced and understood by students and faculty members. Three steps were taken in the Spring 2004 semester to develop this snapshot. First, we recruited and interviewed 17 students with experience in at least two Great Books courses. Second, the interview results were used to develop a two-page questionnaire that was distributed to students in 36 Great Books course sections in Biology, English, Ethics, History, Humanities, Literature, and Theatre. Finally, we conducted three focus group interviews with 9 Great Books faculty members. The intent of this three-part review was not to determine whether the Great Books curriculum is a success or a failure. Rather, the intent was to provide insights that will help Great Books faculty members as they work toward the development of a summative evaluation of program effects. Students’ Interviews In order to understand student perceptions, we conducted interviews with 17 students who had each completed 2 to 7 Great Books courses. We recruited these students by visiting Great Books classes, explaining the purpose of the study, and inviting those who had completed 3 or more Great Books courses to participate in an interview. Our original intent was to interview 15 students who met this 3-course criterion. But because very few students who had completed 3 courses volunteered, we expanded the pool to include students who had completed at least 2 Great Books courses, including the classes from which they were recruited. Of the 17 interviewees, 6 were Hispanic, 2 were African-American, and 10 were women. The students were paid a fifty-dollar honorarium for their time. Each interview was taped and transcribed, although only half of the transcripts were completed in time to contribute to the development of the survey and this report. An analysis of the transcripts and our interview notes shows that student comments fell under two broad themes: descriptions of what students experienced in Great Books classes, and perceptions of what students gained from their participation in the classes. Themes raised in the interviews cannot be interpreted as definitive outcomes, because the interviewees are not representative of the total student population taking Great Books courses. But the interview findings may suggest potential indicators of student learning and development that could guide the development of a detailed evaluation plan for the Great Books Curriculum. Learning Experiences in Great Books Courses The students who volunteered for interviews described these classes as stimulating and engaging. They felt they were being exposed to material that was unambiguously at a "college level" and that would be widely recognized as a body of writing about which educated persons should be acquainted. As one student put it, he wanted to know "great" literature so that he would "get the joke" when conversing with employers and others. In short, many of the interviewees felt they were being introduced into the world of educated people. In addition, the students described a range of teaching and learning activities that made the Great Books classes valuable. These experiences were of two types (though there is no way to determine if these experiences coexisted in the same classes or were distributed among them). The first entailed experiences in which faculty members worked students through a series of short readings, papers, quizzes, or lectures that helped students understand difficult texts. The students appreciated their professors’ ability to help them untangle the complex readings they were confronted with. The second category of experiences entailed class discussions in which faculty members drew out student voices and provided both the challenge to "go deeper" and the support to get there. Students often credited these discussions with helping them appreciate cultural perspectives that might be different from their own. Learning Gains in Great Books Courses When asked about what they gained from the courses, the students noted development in reading ability, analytical thinking skills, and appreciation of cultural differences. Students also reported gains in writing and in speaking and expressing ideas in front of others. Another perceived outcome was an enhanced ability to make connections between different academic disciplines (such as art and literature), compare themes between the works in different classes, or appreciate how the past influences the present. In some cases, a significant outcome was in "realizing what I don’t know" and learning from "listening to someone else." For many students, that "someone else" included other students as well as classroom teachers and the course texts themselves. Many students were excited by what they could learn through the Great Books courses and noted that they would share class books with others (including family members) or seek out additional works by the authors they read in the Great Books courses. In short, students reported that they were acquiring discipline-based knowledge and were honing their reading, writing, and speaking skills. But their comments also suggested more than knowledge and skills acquisition; from their perspective, the students were learning to do things differently or better. Reading is an example. In addition to better reading skills, the students indicated that they were reading more than they had before and that they were reading more intentionally for understanding and "meaning." Some indicated that they were attending performances based on their reading, choosing more Great Books for personal reading, and encouraging others to read them too. These themes of "learning to do better" and "learning to do differently" were verified by the results of the student survey. A Related Finding One significant early finding emerged when we visited Great Books classes to recruit students for the interviews. It became apparent that many students sitting in the recruitment sections did not know they were in a Great Books course; they were not aware of the Great Books Curriculum and did not associate their class with it. Those who volunteered for interviews, of course, were aware of the Great Books Curriculum, and their views cannot be interpreted as the typical perspective of students in Great Books courses. This serendipitous finding is significant. Although some students did not know they were in a Great Books course, those who were aware of the Great Books Curriculum and actively sought out Great Books classes held distinctive views about them in terms of both learning experiences and learning outcomes. This became very clear in the student survey results. Student Survey A survey, developed on the basis of the student and faculty interviews, was administered in class to students in all Great Books courses at the end of the Spring 2004 semester. Biology, English, Ethics, History, Humanities, Literature, and Theatre classes were represented, and 633 completed surveys were gathered. The results offer insights into student demographic characteristics and the reasons students have for enrolling at Wright College and taking Great Books courses. The survey data also confirm interview findings about what students perceive are the benefits of participating in Great Books classes, both in terms of course experiences and outcomes. Finally, the data help us understand the characteristics of students who have the most experience with the Great Books Curriculum, both in terms of the number of courses completed and the tendency to actively seek out Great Books courses. Demographics and Reasons for Enrolling The survey findings suggest that students in Great Books courses are relatively young (average age = 23) and represent diverse ethnic backgrounds. Most (59%) are female, and the majority of respondents (76%) enrolled at Wright to prepare for transfer to a four-year college. When asked why they enrolled in the course in which they were completing the questionnaire, most (67%) indicated that the course “was required for my degree”; only 13% indicated that they enrolled because it was a Great Books course. In addition, only 47% of the students knew that the class was a Great Books course when they first enrolled. When asked how many other Great Books courses they had completed, 33% indicated that they didn’t know. Clearly, many students are not aware of the Great Books Curriculum. Perceptions of Experiences and Learning Gains The questionnaire listed 13 statements (in Question 11) that represent a distillation of the course experiences and potential learning gains mentioned in the student interviews. The respondents were asked to indicate their level of agreement (“strongly agree” to “strongly disagree”) with each of the statements. Majorities either agreed or agreed strongly with each of the 13 statements, but some items garnered stronger levels of agreement than others. The statements, rank-ordered by the percentage of respondents who "strongly agreed," are as follows:
3 course, this was much higher in English 102, which is a composition course. The percentage of students in English 102 who agreed with this statement was 60%. 4 62% of the students in the Theatre classes agreed with this statement. In addition, the survey instrument included an open-ended question:
Consider all of the Great Books courses you have taken (including this one). What have you gained most from your experience with these Great Books courses? Please describe below.
Responses were provided by 324 students, and they identified the same outcomes that the interviewees mentioned, noting perceived gains in critical thinking, reading, writing, speaking in front of groups, discussing complex ideas, and listening to and learning from others. Like the interviewees, students responding to the open-ended question indicated that they gained more than an increased level of skill; they also noted changes in the ways they used their skills. Reading is an example. Responses to the open-ended question indicated enhanced reading ability in terms of:
The responses also indicated that the students perceived that they were:
Students Who Have the Most Experience with the Great Books Courses Of the 633 students responding to the survey, only 67 (11%) indicated that they had completed 3 or more Great Books courses, and only 85 (13%) indicated that they enrolled in the sampled class (i.e., the class in which they completed the survey) because it was a Great Books course. Their responses to the questionnaire differed in many ways from the responses of the total student sample. A review of the data reveals that they were more likely to
Thus, in contrast to Great Books course-takers as a whole, those who have completed 3 or more Great Books courses or who actively seek out Great Books classes appear to be more intrinsically motivated, to have higher academic self-confidence (except in their ability to solve mathematical problems), and to hold the strong perception that their participation in the Great Books classes was beneficial. In addition, women appear to have more experience with the Great Books Curriculum in terms of the number of courses completed and the tendency to seek out Great Books courses. This may reflect the use of class discussion and the premium placed on learning from others. These findings should not be interpreted as indicators of the effect of the Great Books Curriculum. Indeed, of those who indicated that they enrolled in their class because it was a Great Books course, only 25% had completed 3 or more Great Books classes. Yet the percentages of those who enrolled in college for "personal enrichment," who rated their academic skills as "excellent," or who "strongly agreed" with the 13 statements concerning course experiences and outcomes were higher in this group than in the group of 67 students who had completed three or more Great Books courses. Future evaluations will have to assess the extent to which perceived outcomes at the end of a Great Books class or at the end of a sequence of Great Books classes reflect the actual impact of the curriculum rather than the skills and predispositions that students bring with them to the Great Books classes. Faculty Focus Group Interviews A third component of the study involved focus group interviews with a total of nine Great Books faculty members. Two of the meetings were conducted on the Wright College campus. The third was conducted as a conference call. The faculty members were asked for their perceptions of what distinguished Great Books courses from other courses at Wright. They were also asked for perceptions of what students gain through their participation in the Great Books classes. Not surprisingly, the faculty had a more coherent idea of what constituted the Great Books Curriculum than did the students. First, each focus group indicated that the program had both in- and out-of-class learning experiences and that thematic units linked them together. (Students never mentioned the unifying themes of the program during the interviews, and few mentioned the co-curricular activities.) In contrast to survey findings showing the small proportion of students who complete 3 or more Great Books courses or who self-select into Great Books courses, some of the faculty members also thought that students acquired a "special identity" through the Great Books Curriculum. However, other faculty members noted that students who were not well-suited to "the pedagogy" in these courses often dropped out. Second, the faculty members had a more sophisticated view of learning experiences in the Great Books courses. They stressed the importance of shared inquiry, the Socratic method, performance assessments (such as projects and presentations), interdisciplinary understanding from the connections made between and within courses, and use of primary texts in the best available edition or translation. In-class and online discussions were seen as valued learning experiences in which students were exposed to challenging material "that is not watered down" and were then supported to "engage the text." While the faculty members did not mention students learning from each other (a theme mentioned by some of the student interviewees), their comments placed a high value on allowing students to use their own voices in discussion and to listen to the perspectives of others. Faculty members noted how they placed difficult, controversial issues before students, helping them to understand how these issues developed over time and using the texts to hone analytic skills. In addition, the faculty also worked to include diverse voices in course readings. They saw this as a benefit to students, helping them to learn about other cultures and making sure that their own cultures were represented as well. Finally, the faculty members, like their students, noted learning gains that go beyond enhanced analytic skill. They mentioned an "openness" to new ideas and the development of writing skills that "mov[e students] from high school report writing to researched, documented argumentation." They also stressed the importance of acquainting students with time-tested writings and scholarship that they might not have otherwise been exposed to. Thus, the faculty seemed to share values for high-level learning built around rigorous texts. But a defining pedagogy tying all courses together remains elusive. In one of the focus groups, a question about “common pedagogy” across the Great Books Curriculum yielded the response that it was “not yet explicit.” Others said that they were not sure if there was a common pedagogy. A summative evaluation of the Great Books Curriculum awaits the full development of the curriculum itself as a cohesive group of courses that is tied together by explicitly stated learning goals and a shared understanding of the pedagogical approaches that will lead to those goals. Discussion The intent of this report is formative, describing the Great Books Curriculum as a lived experience for students and faculty. It provides a snapshot view of what students perceive to be the key learning experiences in the curriculum (“what we did”) and the key outcomes (“what we gained”). Results show that the student perspectives are generally complemented by those of core Great Books faculty who participated in face-to-face and telephone focus-group interviews. As such, this evaluation and its descriptions form a baseline for future work to develop and refine the educational practices and most cherished learning outcomes of the Great Books Curriculum. Data collected in this evaluation support claims that the Great Books courses provide some distinctive learning experiences and lead to important learning gains. Many of the perceived gains appear to be related to at least two of the intended outcomes of the Great Books courses: improved thinking, reading, and writing skills; and improved cultural literacy. Furthermore, the evaluation suggests that there are students and faculty members who are devoted to Great Books courses. But the extent to which these experiences and gains apply across courses is unknown, as is the exact nature of the learning gains achieved by students. And beyond the few self-selected students and faculty members who commit themselves to the courses, a coherent, systemic curriculum has yet to emerge. The most telling evidence of this is the fact that many students did not know they were in Great Books courses, and few enrolled in Great Books courses explicitly because they are Great Books courses. The student survey suggests that the courses are beneficial, but many students - perhaps most - did not appear to be interfacing with these courses as a curriculum. The focal criterion of the program - that 50% of course readings must be from a list of "great books" - has proven to be a useful starting point. It has provided at least the beginnings of a unifying theme. But it is not sufficient to make the collective efforts of the talented and committed faculty members who teach Great Books courses into a coherent whole. The program stakes its identity on Great Books and their timeless qualities. Yet the strength of the program is not in the books per se but in the faculty members and their clear intention to offer Wright students a challenging curriculum and the necessary pedagogical support to help them succeed with it. There is great potential here given the commitment shown by faculty and the enthusiastic responses of the students who were interviewed. Two steps can be taken to realize this potential. The first is the establishment of an internal assessment plan that begins by articulating and then measuring shared learning outcomes across the courses. The four stated program outcomes for the Great Books Curriculum can be a starting point. The assessment plan can refine the definitions of those outcomes in ways that help faculty members develop teaching strategies that will lead to those outcomes. For example, if critical thinking is a desired outcome, the plan should offer a common definition of critical thinking that will be applied across all Great Books courses and help faculty members understand where they should be leading students. The second step, taken in tandem with the first, entails collaborative faculty work on the identification and description of pedagogical practices that lead Great Books students to desired ends. The focus group discussions clearly indicated that those teaching Great Books courses have a wealth of knowledge and experience about instructional practices that can engage students. These practices should be shared internally and externally and then refined as part of the ongoing effort to assess learning outcomes and evaluate the program. This second step complements the first, helping to define the curriculum not only in terms of desired outcomes but also in terms of the processes and experiences that will lead to those outcomes. The articulation of both will be necessary if Wright’s Great Books Curriculum is to serve as a model for other community colleges. |
|
E-mail a Friend
Print This Page