Georgia Southern University

First Monitoring Report

September 6 2006

(1) Restatement of Comprehensive Standard, Recommendation Number, and the Recommendation as Articulated in the Visiting Committee Report

Comprehensive Standard 3.4.1
The institution demonstrates that each educational program for which academic credit is awarded (a) is approved by the faculty and the administration, and (b) establishes and evaluates program and learning outcomes.

Recommendation 5
The Committee recommends that the University be consistent in establishing learning outcomes across all educational programs and in developing and implementing assessments to evaluate these outcomes.
Georgia Southern University prepares several reports having a direct or indirect relationship to the establishment and evaluation of educational program and learning outcomes:

  • Program Reviews (mandated by the Board of Regents) for the purpose of “evaluating academic programs in terms of their quality, viability, and productivity of efforts in teaching and learning, scholarship, and service as they related to the institutional mission.”
  • Annual Reports for the purpose of highlighting accomplishments that occurred during the year of record.
  • Level III (academic and support unit) Strategic Plans.
  • Institutional Effectiveness Plans (IEP) for the purpose of formal assessment, on a 3-year cyclical basis, of student learning outcomes and programmatic outcomes and connection of the outcomes with the University’s mission and strategic plan.
Although there is some mention of program outcomes in the Program Reviews, Annual Reports, and Level III Strategic Plans, none of these documents was designed to focus particularly on student learning outcomes. The first use of IEPs for assessing and reporting on student learning outcomes occurred over the period of 2002-2005. As acknowledged by the University, based on the results of this first year cycle of use of the IEPs, understanding of measurement and assessment tools in evaluating student learning outcomes has been inconsistent across academic units. The University is taking several steps to ensure a more consistent identification of student learning outcomes and the evaluation of learning outcomes across all academic units:
  • Engaging faculty in workshops to broaden understanding of the development of student learning outcomes and their assessment and evaluation.
  • Initiating a new cycle of IEPs
  • Proposing the elimination of Level III Strategic Plans for academic departments.
  • Using findings from the IEPs in Program Reviews and Annual Reports.
The Committee acknowledges the University’s ongoing review of the IEP process and its relationship to other planning and evaluation activities and also acknowledges the University’s plans to conduct another IEP cycle with modifications based on the findings of the review.

(2) History of Institutional Response If More than a First Response
In its response to the Visiting Committee’s Report, Georgia Southern indicated a plan to initiate a new round of data collection using the Institutional Effectiveness Plans (IEP). The University also outlined additional steps to ensure a more consistent identification of student learning outcomes and more consistent evaluation of learning outcomes across all academic units. These steps included engaging faculty in workshops to broaden understanding of the development of student learning outcomes and their assessment and evaluation, refining institutional master planning processes, sending campus representatives to external training related to assessment and learning outcomes, and eliminating the Level III Strategic Plans for academic departments.

(3) Current Request of the Commission
CS 3.4.1 (All Educational Programs), Recommendation 5
Document implementation of the university’s refined planning and evaluation processes during 2005/06. Include evidence of the assessment of expected outcomes for educational programs (including program and learning outcomes), administrative, and educational support services, and provide evidence of improvement based on analysis of those results. In its last report, the university indicated that it had chosen to implement refinements during 2005/06 rather than to rework the 2004/05 documentation.

(4) Georgia Southern’s Response
As indicated above, Georgia Southern launched another cycle of data collection in Fall 2005, using the Evidence-Based Decision-Making (EBDM) process to gather and analyze data. EBDM has been widely embraced on campus to the extent that many people now refer to the Institutional Effectiveness Plans (IEPs) as EBDMs. To introduce EBDM to the campus community, Dr. Marilee J. Bresciani, Assistant Vice President for Institutional Assessment and Visiting Associate Professor Office of Institutional Assessment and Diversity formerly at Texas A&M University, was invited to Georgia Southern to present a two day workshop in September 2005. All division representatives and corresponding unit heads were asked to attend. Information was presented explaining the EBDM process and participants had an opportunity to discuss EBDM in smaller interactive group sessions. In addition, EBDM was the main topic at the October 2005 Administrator’s Workshop and at the October 2005 Deans, Directors, and Department Chairs Workshop. Culminating from these sessions, the University developed “common language” for the EBDM process. This document is presented in Attachment 3.3.1A.

The data collection included all academic, administrative, and educational support services. Initial data collection was submitted by October 2005, with review and feedback provided shortly thereafter by a University review team. Units were asked to refine their plans and resubmit full EBDMs by June 1, 2006. The Divisions of Business and Finance, University Advancement, and Student Affairs and Enrollment Management were given until July 15, 2006. All of the University’s EBDM plans are provided in the Appendix.

Since the Division of Academic Affairs is the largest division on campus, the Provost and Vice President for Academic Affairs initiated several additional activities between October 2005 and June 2006 to ensure that all units had a basic understanding of the EBDM process, kept on task, and were able to relate their individual unit’s EBDM to other ongoing campus initiatives. Following focus groups with students and faculty leaders, the academic deans finalized the “Desired Attributes of Graduates” in early January 2006. These outputs were used in a two day, off-campus workshop at Little Ocumulgee State Park. At this venue, deans, department chairs, and directors met in interactive sessions to discuss the common understanding of desired Georgia Southern University outputs and how EBDM should document achievement of these outputs.

The Director of Strategic Research and Analysis worked closely with the Division of Student Affairs and Enrollment Management in the development of that Division’s EBDM plans, providing training, guidance, and feedback as needed. Likewise, the Director of Institutional Effectiveness met with the Divisions of Business and Finance and University Advancement, providing training, guidance, and feedback. Table 3.4.1 documents the progress made in response to Recommendation 5.

Review of the EBDMs indicate that not all of the units have the reached the desired level of understanding of the EBDM process. Within Academic Affairs, it is clear that some colleges are more capable of using assessment results to document improvements than are others. All units, however, have established student learning and/or programmatic outcomes and identified assessment measures. In several units, which did not exist before 2005, these assessment instruments still need to be developed. For example, the Jiann-Ping Hsu College of Public Health was newly established effective January 1, 2006. Therefore, implementation of the EBDM plans for the programs in this college are scheduled for 2006-07.

Nonetheless, the University can document significant progress from the prior round of IEPs. Table 3.4.1A displays a sample of student learning and programmatic outcomes by program along with an explanation of how assessment results were used to improve student learning or program results. In addition, Table 3.4.1B contains a sample of the full EBDM plans, showing the outcome, relevancy to the strategic theme, assessment measure(s), benchmark(s), and how the results were used to improve student and/or program performance. Again, all EBDM plans are provided in the Appendix.

Upcoming Year
Furthermore, the University plans to continue its efforts to broaden understanding of the EBDM process and of student learning outcomes. Beginning in Fall 2006, the University plans to initiate training sessions for the Undergraduate and Graduate Committee members1 to enable them to better identify appropriate student learning outcomes and to work with units in writing measurable student learning outcomes on course curriculum forms. Course curriculum forms should not be approved by these committees without appropriate and measurable student learning outcomes identified.

Throughout all units in the University, campus conversations are continuing based upon the evidence already collected. The Division of Academic Affairs held its annual Deans, Directors, and Department Chairs Workshop on August 10, 2006, where work continued on drafting an academic blueprint for the 2006-07 academic year. This blueprint highlights several targeted areas that the Division plans to purse this forthcoming year. These initiatives were identified, in part, from evidence collected in individual unit EBDM plans, and the EBDM process will be used to implement the academic blueprint once it is finalized.

In the coming year, units are tasked with following through on the actions reported in their EBDM plans and providing a mid-year evaluation at the end of the fall semester and an end-of-year evaluation in the spring semester. These efforts may range from gathering data to analyzing data to actually documenting closure of the feedback loop based on the evidence gathered. For example, the EBDM plan for the College of Liberal Arts and Social Sciences notes that data still needs to be collected, while the EBDM plan for Art—Graphic Design Concentration indicates that faculty need to develop assessment rubrics. In some cases, data have been gathered and analyzed, but the results showed that assessments instruments needed to be refined. Such is the case with the plan submitted for the Finance program in the College of Business Administration.

Mathematical Sciences outlined an improvement plan based upon the evidence already collected. In this case, the department plans to establish a prerequisite check in the Banner Student Information System before the beginning of classes to remove unprepared students from courses. Currently, almost 20% of courses are comprised of students who do not meet the prerequisites. Results of this effort will form part of their mid-year evaluation report. By the end of the year, it is anticipated that many more units will be at the stage of documenting improvements based on the evidence collected.

Attachments
Attachment 3.4.1A
Table 3.4.1
Table 3.4.1A
Table 3.4.1B


1For a full description of the curriculum review process, please refer to http://sacs.georgiasouthern.edu/compliance/comprehensive/cs3-4-1.htm.