Georgia Southern University

First Monitoring Report

(1) Restatement of Comprehensive Standard, Recommendation Number, and the Recommendation as Articulated in the Visiting Committee Report

Comprehensive Standard 3.3.1
The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.

Recommendation 4
The Committee recommends that the University document the use of analyses of outcome assessments in decision making for improvements made in its educational programs and its administrative and education support services.

The Focused Report, the Institutional Effectiveness Reports for 2002-03, and Academic Program Reviews were reviewed with respect to documenting the expected outcomes for educational programs and administrative and educational support services, assessments of those outcomes, and evidence of improvements based on analyses of those results. Also, the Committee interviewed the Director of Strategic Research and Analysis, the Director of Institutional Effectiveness, and the Associate Provost. The interviews yielded further insight into the process that led to the development of the Institutional Effectiveness Plans and the process used to identify strengths and weaknesses of the first iteration of the Institutional Effectiveness Reports. The Committee notes the efforts being made to provide assessment training and the intention to use external experts in assessment. Although the Reaffirmation Committee’s review established the existence of program and service outcome statements, many were not measurable. Further, use of assessment data as the basis for improvements was not well documented. University personnel acknowledged that evidence of data to support the assessment of outcomes was inconsistent in some areas.

(2) History of Institutional Response If More than a First Response
In its response to the Visiting Committe’s Report, Georgia Southern indicated that it gave serious consideration to revising the Institutional Effectiveness Plans (IEPs) reviewed by the Visiting Committee to correct the noted deficiencies. After much deliberation, the University opted to invest its energies in a new refined cycle of data collection. Preceding the new round of data collection, faculty, representing each academic college, the Provost, the Chair of the Strategic Planning Council, the institution’s Chief Information Officer, and two deans attended external training sessions on developing appropriate student learning outcomes and aligning them with assessments. This information was then brought back to campus to be shared with others. Beyond these training sessions, campus groups, led by the Chair of the Strategic Planning Council, met to revise the University’s master planning process based on suggestions from the off-site and on-site SACS review teams. Refining the campus planning process has been an ongoing effort since Spring 2005. These efforts sought to strengthen the linkage between strategic planning, decision making, and assessment. The results of these efforts were encapsulated in the Master Planning & Assessment Schematic shared in the University’s “Response Report to the Visiting Committee” as Attachment (a) for Comprehensive Standard 3.3.1

Coinciding with the refinement of the University’s master planning process, another campus group, led by the Office of Institutional Effectiveness and Strategic Planning, met to carefully and critically re-examine the last round of IEPs. Inconsistencies in the appropriate use of learning and programmatic outcomes and common problems related to the documentation of the use of assessment data as the basis for improvement were identified. Training specifically targeted to these two deficiencies was provided prior to the new round of data collection scheduled for Fall 2005.

(3) Current Request of the Commission
CS 3.3.1 (Institutional Effectiveness), Recommendation 4
Document implementation of the university’s refined planning and evaluation processes during 2005/06. Include evidence of the assessment of expected outcomes for educational programs (including program and learning outcomes), administrative, and educational support services, and provide evidence of improvement based on analysis of those results. In its last report, the university indicated that it had chosen to implement refinements during 2005/06 rather than to rework the 2004/05 documentation.

(4) Georgia Southern’s Response
As indicated above, Georgia Southern launched another cycle of data collection in Fall 2005, using the Evidence-Based Decision-Making (EBDM) process to gather and analyze data. EBDM has been widely embraced on campus to the extent that many people now refer to the Institutional Effectiveness Plans (IEPs) as EBDMs. To introduce EBDM to the campus community, Dr. Marilee J. Bresciani, Assistant Vice President for Institutional Assessment and Visiting Associate Professor Office of Institutional Assessment and Diversity formerly at Texas A&M University, was invited to Georgia Southern to present a two day workshop in September 2005. All division representatives and corresponding unit heads were asked to attend. Information was presented explaining the EBDM process and participants had an opportunity to discuss EBDM in smaller interactive group sessions. In addition, EBDM was the main topic at the October 2005 Administrator’s Workshop and at the October 2005 Deans, Directors, and Department Chairs Workshop. Culminating from these sessions, the University developed “common language” for the EBDM process. This document is presented in Attachment 3.3.1A.

The data collection included all academic, administrative, and educational support services. Initial data collection was submitted by October 2005, with review and feedback provided shortly thereafter by a University review team. Units were asked to refine their plans and resubmit full EBDMs by June 1, 2006. The Divisions of Business and Finance, University Advancement, and Student Affairs and Enrollment Management were given until July 15, 2006. All of the University’s EBDM plans are provided in the Appendix.

Since the Division of Academic Affairs is the largest division on campus, the Provost and Vice President for Academic Affairs initiated several additional activities between October 2005 and June 2006 to ensure that all units had a basic understanding of the EBDM process, kept on task, and were able to relate their individual unit’s EBDM to other ongoing campus initiatives. Following focus groups with students and faculty leaders, the academic deans finalized the “Desired Attributes of Graduates” in early January 2006. These outputs were used in a two day, off­campus workshop at Little Ocumulgee State Park. At this venue, deans, department chairs, and directors met in interactive sessions to reach a common understanding of desired Georgia Southern University outputs as well as to develop a stronger grasp of the EBDM process for their individual units.

The Director of Strategic Research and Analysis worked closely with the Division of Student Affairs and Enrollment Management in the development of that Division’s EBDM plans, providing training, guidance, and feedback as needed. Likewise, the Director of Institutional Effectiveness met with the Divisions of Business and Finance and University Advancement, providing training, guidance, and feedback. Table 3.3.1 documents the progress made in response to Recommendation 4.

Review of the EBDMs indicate that not all of the units have the reached the desired level of understanding of the EBDM process. Within Academic Affairs, it is clear that some colleges are more capable of using assessment results to document improvements than are others. All units, however, have established student learning and/or programmatic outcomes and identified assessment measures. In several units, which did not exist before 2005, these assessment instruments still need to be developed. For example, the Jiann-Ping Hsu College of Public Health was newly established effective January 1, 2006. Therefore, implementation of the EBDM plans for the programs in this college are scheduled for 2006-07.

Nonetheless, the University can document significant progress from the prior round of IEPs. Table 3.3.1A displays a sample of student learning and programmatic outcomes by program along with an explanation of how assessment results were used to improve student learning or program results. In addition, Table 3.3.1B contains a sample of the full EBDM plans, showing the outcome, relevancy to the strategic theme, assessment measure(s), benchmark(s), and how the results were used to improve student and/or program performance. Again, all EBDM plans are provided in the Appendix.

Upcoming Year
In the coming year, work will continue using the EBDM process. Units will be tasked with following through on the actions reported in their EBDM plans and providing a mid-year evaluation at the end of the fall semester and an end-of-year evaluation in the spring semester. These efforts may range from gathering data to analyzing data to actually documenting closure of the feedback loop based on the evidence gathered. For example, the EBDM plan for the College of Liberal Arts and Social Sciences notes that data still needs to be collected, while the EBDM plan for Art—Graphic Design Concentration indicates that faculty need to develop assessment rubrics. In some cases, data have been gathered and analyzed, but the results showed that assessments instruments needed to be refined. Such is the case with the plan submitted for the Finance program in the College of Business Administration.

Mathematical Sciences outlined an improvement plan based upon the evidence already collected. In this case, the department plans to establish a prerequisite check in the Banner Student Information System before the beginning of classes to remove unprepared students from courses. Currently, almost 20% of courses are comprised of students who do not meet the prerequisites. Results of this effort will form part of their mid-year evaluation report. By the end of the year, it is anticipated that many more units will be at the stage of documenting improvements based on the evidence collected.

Besides the general expectation of continuing to implement EBDM plans, campus conversations in all divisions at the University are continuing based upon the evidence already collected. The Division of Academic Affairs held its annual Deans, Directors, and Department Chairs Workshop on August 10, 2006, where work continued on drafting an academic blueprint for the 2006-07 academic year. This blueprint highlights several targeted areas that the Division plans to purse this forthcoming year. These initiatives were identified, in part, from evidence collected in individual unit EBDM plans, and the EBDM process will be used to implement the academic blueprint once it is finalized.

Attachments
Attachment 3.3.1A
Table 3.3.1
Table 3.3.1A
Table 3.3.1B