Georgia Southern University

First Monitoring Report

(1) Restatement of Comprehensive Standard, Recommendation Number, and the Recommendation as Articulated in the Visiting Committee Report

Comprehensive Standard 3.5.1
The institution identifies college-level competencies within the general education core and provides evidence that graduates have attained these competencies.

Recommendation 6
The Committee recommends that the University expand its evaluation system to assess all stated general education core competencies in order to provide evidence that graduates have attained those competencies.

The Committee found evidence that Georgia Southern University has conducted a series of reviews and surveys to examine the appropriateness of the general education core curriculum. The University has identified a set of general education outcomes; however, the Committee was able to find only limited evidence of the attainment of general education core competencies by graduates.

An example of this limited evidence was the Regents’ Test, which measures proficiency in reading comprehension and writing and is required by the time a student has earned 45 credit hours. The University has also attempted to use the College Assessment of Academic Proficiency (critical thinking, reading, and mathematics) but with very limited success with respect to the number of students participating in taking the tests. As stated in the Focused Report, the University acknowledges “the need to more extensively assess the stated general education outcomes.” Assessment measures for all of the general education competencies are needed.

(2) History of Institutional Response If More than a First Response
In its response to the Visiting Committee’s Report, the University noted the derivation of its general education outcomes—emanating from the University System of Georgia in 1998. The University further noted the strong likelihood that these System general education outcomes may be revised, the result of which places Georgia Southern in the awkward position of having to develop an assessment plan for general education outcomes that may soon be outdated. Indeed this has come to pass, as the University System proceeded during 2005-06 with its re-evaluation of the System general education outcomes. Nonetheless, Georgia Southern indicated that it wished to be responsive to the Visiting Committee’s concerns and therefore would proceed with developing an effective assessment strategy.

The result was the General Education Outcomes Assessment Plan which mapped out direct linkages between planning, assessment, and implementation activities. The full plan is contained in Georgia Southern University’s “Response Report to the Visiting Committee” as attachment to Comprehensive Standard 3.5.1 (http://sacs.georgiasouthern.edu/revisedqepresponse/attachments/GenEd_LOAPlan_12Aug05.xls). The University further noted that the plan would undergo “further refinement and delineation of assessment methods and implementation strategy” during Fall 2005.

Table 3.5.1 in the response below, addresses each of the specific components of the General Education Outcomes Assessment Plan and notes the progress made to date.

(3) Current Request of the Commission
CS 3.5.1 (Undergraduate Programs), Recommendation 6
Provide evidence that graduates achieve the general education competencies. In its last report, the university presented a well-conceived plan; however, numerous elements had not yet been implemented.

(4) Georgia Southern’s Response
Background
As noted above, Georgia Southern University finds itself it in the awkward position of refining and implementing a general education assessment plan at the same time that the University System of Georgia is reconsidering the general education outcomes. In light of the University System’s actions, Georgia Southern approached the refinement and implementation of the General Education Outcomes Assessment Plan critically, carefully evaluating each component to be sure the action would net the University useful and timely information. (For greater detail on the University’s efforts, please see Attachment 3.5.1A.)

For example, in its response to the Visiting Committee, Georgia Southern noted that the team responsible for implementing the General Education Outcomes Assessment Plan (the General Education Assessment Council – GEAC) planned two initial surveys to gather additional information prior to the full implementation of the plan. These surveys were the General Education Achievement and Importance surveys — one version which would be administered to faculty teaching core general education courses in Fall 2005; and a second version which would be administered to students in selected general education courses. The results of these surveys would be used to refine the core curriculum outcomes assessment matrix completed in an earlier survey in March 2004 (pp. 23-28). ¹ On closer review, however, it was determined that there were no tangible benefits to administering either of these surveys. The University opted instead to administer a direct assessment of student learning versus these indirect assessments of faculty and student perceptions.

Progress: College Assessment of Academic Proficiency (CAAP)
Consequently, Georgia Southern administered the College Assessment of Academic Proficiency (CAAP) to measure critical thinking skills in 349 students in March and April of 2006. This assessment was administered in classes where the majority of students were sophomores and juniors who had completed the general education core. Preliminary results indicate that Georgia Southern students demonstrated critical thinking skills slightly above the national norm average (63.7 versus 62.7). The University is now focused on how to improve these scores. Best practices and a content analysis of the report has been requested which will enable the University to link critical thinking skills to the general education outcomes. Pending analysis of this report in Fall 2006, the University plans to investigate more closely those cases where the scores fell below the national norm, seeking to identify specific deficiencies that need to be addressed. Additionally, the University plans to expand its future administrations of CAAP to include writing skills, mathematics, reading, and science.

Progress: College of Liberal Arts and Social Sciences (CLASS)—General Education Syllabi Review
In the interim, the College of Liberal Arts and Social Sciences (CLASS), which served 56% of students in the general education core curriculum during 2005-06² , conducted an intensive syllabi review of their general education curriculum during the 2005-06 academic year. The reasons for this intensive review included the following.

√ Provide an accurate understanding of the quality of the present general education syllabi.

√ Facilitate a cultural shift from teacher-centeredness to learner-centeredness.

√ Ensure quality and consistency between the same courses.

√ Provide clarity in course design (e.g., ensure that learning outcomes correlate with learning activities and assessments).

√ Provide clearly articulated expectations for learning outcomes correlated to learning activities and assessments.

√ Engage students and give them ownership and responsibility for the learning process.

Progress: Mathematical Modeling Initiative
One area of the general education core curriculum that continues to be a major concern to the University is the high “drop, failure, withdraw” (DFW) rate evidenced in College Algebra (MATH 1111). The DFW rate for this course exceeds national norms. Established benchmarks are a DFW rate of less than 40% and a success rate of 60% or greater. The Allen E. Paulson College of Science and Technology collected data on two major assessments to inform decisions on how to improve student learning outcomes in mathematics. One assessment was an analysis of the frequency of course (and section) offerings and enrollment projections for general education courses. The second assessment was a review of student performance in College Algebra (MATH 1111) from Fall 1998 through Spring 2005 (see Attachment 3.5.1C). Based upon the data collected, the College plans to shift selected enrollments from College Algebra to Introduction to Mathematical Modeling, beginning in Fall 2006. This initiative is part of an overall effort to decrease the DFW rates in all core mathematics courses by 25% each academic year, commencing 2006-07.

The main premise behind this initiative is that for many program majors Introduction to Mathematical Modeling⁴ (MATH 1101) provides adequate mathematical preparation consistent with the requirements of the major (and Area A of the general education core curriculum). Alternatively, College Algebra⁵ (MATH 1111) includes concepts specifically required of business or science majors. (For a cross-listing of the student learning outcomes associated with each course and their connection to the general education outcomes, see Attachment 3.5.1D.) The Department will be establishing a prerequisite check in the BANNER student information system to ensure that all currently enrolled students in both courses have met the required prerequisites. Currently, as many as 20% of the courses are comprised of students lacking this required preparation. Secondly, students will be advised to register for the Optional Mathematics Placement Test (administered via the web) before attending university orientation. Both of these actions will help to ensure that students are placed in the appropriate mathematics course that meets their program needs and facilitates their success.

Section offerings in both courses will be adjusted to compensate. For instance, rather than offer an average of 49 sections of College Algebra in the fall semester (2006), 37 sections will be offered, and 15 sections of Mathematical Modeling will be offered. Likewise, for the spring semester (2007), the average number of sections for College Algebra will be reduced from 26 to 14, and 10 sections of Mathematical Modeling will be offered. Additionally, the department is continuing its review of appropriate student learning outcomes in core math classes and will be revising course syllabi based upon this review.

Finally, to ensure that MATH 1101 instructors are adequately prepared to teach Mathematical Modeling and that student learning outcomes are achieved, the College held three separate MATH 1101 Discussion Groups in Spring 2006. A common course outline was developed, and MATH 1101 classes taught by experienced faculty held observation sessions for new instructors. The DFW rates for both College Algebra and Mathematical Modeling (as for all core mathematics courses) will continue to be monitored, and the initiative adjusted accordingly.

Progress: National Science Foundation—Science, Technology, Engineering, and Mathematics (STEM) Grant
Another initiative targeted at decreasing the DFW rates in science and mathematics courses, but related to assessing some of the general education outcomes, is a project funded through a $1 million National Science Foundation (NSF) grant for Science, Technology, Engineering, and Mathematics (STEM) students. This project, Advisement and Scholarship, Promoting Inquiry-Based Research Experiences in STEM (ASPIRES), seeks to decrease DFW rates in introductory science and mathematics courses by 3-5% each year, achieving DFW rates of 20-25% within five years. To reach this goal, ASPIRES will initiate supplemental instruction through Peer-Led Team Learning (PLTL) and will incorporate guided inquiry into introductory courses. Corresponding assessment strategies have been built into the model to measure the success of each initiative. Some of the assessment strategies measure student learning as it relates to selective general education outcomes.

For example, the table below shows how one of the assessment strategies for the initiative, Initiate supplemental instruction through Peer-Led Team Learning (PLTL) to promote student academic achievement, can be mapped to assessing (indirectly) student learning in two of the general education outcomes.

Similarly, the second goal of ASPIRES, to increase retention of STEM students, also correlates to assessing some of the general education outcomes. This goal seeks to achieve a first-year retention rate of 82% within three years and 85% within five years. Within STEM-specific courses, a first-year retention rate increase of 3% is sought per year, achieving 75% within five years. A graduation rate of 40% within two years of the project’s completion is sought. To reach these benchmarks, the project will implement three initiatives, one of which is to target undergraduate research opportunities for freshmen. Again, presented in the table below is an example of how two of the assessment strategies can be mapped to assessing one of the general education outcomes.

Progress: Regents’ Exam—Writing Assessment
All students who graduate from a University System of Georgia institution are required to pass the Regents’ Exam, which assesses the student’s reading and writing ability. Any student who has not passed both portions of the exam by the time he or she accumulates 45 credit hours must enroll in a review course for the Regents’ Exam section(s) failed. In 2000-01, Georgia Southern University’s pass rates for students who had earned 45 credit hours were 74.2% on the essay portion of the exam, 74.5% on the reading portion of the exam, and 68.4% for both sections, requiring 21 sections of the review courses each semester.

Not satisfied with the students’ performance, the University initiated the following actions to improve these pass rates. Students were registered for the Regents’ Exam in their first semester and were advised to continue registering for the exam each semester until both sections were passed. Sitting for the exam was also made a prerequisite of ENGL 1102 (Composition II and a core curriculum requirement in Area A). Email reminders were sent to students from the Registrar’s Office, Testing Office, and the Academic Success Center (ASC) reminding them of their exam appointments. Students were also encouraged to attend a review workshop sponsored by the ASC the week before the exam. Signs were posted around campus to further remind students of this important appointment, and the exam date was published in the George-Anne (campus newspaper). These marketing strategies successfully increased the pass rates, resulting in a reduction of the number of review sections needed from 21 to 8 each semester. The 2004-05 pass rates for Georgia Southern were 95.8% for the essay portion of the exam, 96.1% for the reading portion of the exam, and 93.3% for both sections.

Progress: National Survey of Student Engagement (NSSE)
Georgia Southern University continues to administer the National Survey of Student Engagement (NSSE) to provide longitudinal data on how Georgia Southern students perceive their level of engagement which can then be related to the general education outcomes. This national assessment was administered in the Spring 2002, 2004, and 2005. For example, the second item on the survey instrument asks students “During the current school year, how much has your coursework emphasized the following mental activities?” Answers can be correlated to the following general education outcomes as shown below.⁶

Likewise, item 11 on NSSE asks “To what extent has your experience at this institution contributed to your knowledge, skills, and personal development in the following areas?” Virtually all of the items (a-p) [http://nsse.iub.edu/html/survey_instruments_2005.cfm] can be correlated with the general education outcomes. Data collected is analyzed and compared to institutional peers.⁷ Benchmark data (http://services.georgiasouthern.edu/osra/download/NSSE, pp. 15-17), particularly the Level of Academic Challenge, provide additional normative information. Analysis of this benchmark for the most recent survey administration suggests that Georgia Southern students would welcome greater academic challenge. This information will form part of the 2006-07 academic blueprint currently under discussion (at the deans/Provost level) as well as guide faculty discussions at the departmental and college levels.

Upcoming Year
The syllabi review in the College of Liberal Arts and Social Sciences helped to link the general education outcomes with specific courses. Faculty are now using the results of this review to ascertain whether their assessments measure the general education outcome(s). For instance, in the Department of Literature and Philosophy, one instructor is reviewing his assigned readings to determine if they facilitate achievement of the desired outcome. Syllabi revisions are already being made as a result of this review, and refinement of assessment instruments will likely follow.

In addition, the College of Liberal Arts and Social Sciences has identified a 2006-07 Faculty Fellow who will be tasked primarily with overseeing the implementation of the action steps which emanated from the syllabi review. In a similar vein, another faculty member has been identified to coordinate the first year writing program and work with the chair of the Department of Writing and Linguistics to develop a plan to address the high DFW rates in the composition courses which are a part of the general education curriculum.

Preliminary results from the Mathematical Modeling Initiative show some improvement. The average grade point average for the past two years in College Algebra was 2.17 with an average DFW rate of 46.10% across 48 sections. In Mathematical Modeling, the equivalent course (which covers the same topics, but is taught using real world applications and problem solving) had an average grade point average of 2.73 with a DFW rate of 30.44% across three sections. As a result of this evidence, the number of Mathematical Modeling sections will be increased in 2006-07 from three to 15. Both DFW rates and grade point averages will continue to be monitored as well as the success of students from each course in their other upper-division coursework. National studies, including some funded by the National Science Foundation, indicate that math modeling leads to deeper and longer term understanding of mathematics by students. Depending on the results from ongoing assessments, the percentage of students taking each of the two foundational mathematical options will be expanded, maintained, or reduced.

The Advisement and Scholarship Promoting Inquiry-Based Research Experiences in STEM (ASPIRES) grant is a new award; therefore, efforts in the forthcoming year will focus mainly on establishing the support structure necessary to implement the action plan. Implementation of the action plan for the first freshmen cohort will begin in August 2007, and over the five-year award period will directly impact 650+ incoming first-year STEM students for each year of the grant, resulting in an additional 140 STEM graduates per year. The initiatives will offer student support through tutoring and advising to ensure academic achievement and career development; promote student/faculty engagement; and offer faculty professional development opportunities. A carefully designed oversight and management plan will be utilized to ensure that ASPIRES achieves its goals and outcomes, including collecting baseline data at the beginning of the project, end of each project semester, and end of each project year for each action step.

Collection of evidence on student learning has stimulated conversation and action regarding learning outcomes in all majors. As noted previously, major areas differ in their progress of evidence gathering and continuous improvement. Continued analysis of data—from NSSE, CAAP, and other sources—will serve to sustain the momentum required to improve both retention and learning.

Attachments
Table 3.5.1
Attachment 3.5.1A
Attachment 3.5.1B
Attachment 3.5.1C
Attachment 3.5.1D