{Prefer to access the Mid-Cycle Evaluation via the PDF? Click here.}
Table of Contents
2022 Mid-Cycle Evaluation Team
Mission Fulfillment: Executive Summary
Institutional Overview
Oregon Coast Community College (OCCC) is an open-access, publicly funded two-year institution, one of 17 community colleges in Oregon. OCCC was established in 1987 as a “contract college” operating under the regional accreditation of other larger Oregon community colleges. In 2014, OCCC leadership initiated the process to seek independent regional accreditation, which was earned in 2020 through the Northwest Commission of Colleges and Universities. Achieving and sustaining independent accreditation has profoundly impacted all college operations. As is the case for post-secondary education nationally, in the years since 2020 OCCC has also been profoundly impacted by the Coronavirus pandemic, and now seeks to proactively manage the shift to endemic conditions.
OCCC serves Lincoln County, which encompasses nearly 1,000 square miles of land (and 200 square miles of water) with 73 miles of coastline on the Pacific Ocean. This terrain of great beauty and incredible ecological, biological, and geographic diversity is rapidly becoming a nexus for Maritime and STEM-based activity. Lincoln County also faces many of the challenges of isolated rural communities, including sparse population, distance to services, pockets of poverty, dearth of public transportation, and other barriers. The county’s economy varies markedly throughout its different communities. Northern and southern areas are dominated by seasonal tourism and retail services while the central portion of the county, including Newport, sees tourism buttressed by more stable, year-round contributions from government, commercial fishing, port facilities, and a growing science and technology sector. OCCC is the only locally accessible provider of the first two years of a bachelor’s degree. The nearest university, Oregon State University (OSU), is 55 miles and a coastal mountain range away. OCCC’s headcount has varied from 1,100 to 2,000 students annually including credit and non-credit. Full-time equivalent enrollment (FTE) of credit students dipped, along with headcount, in 2020-21 but is now approaching the pre-pandemic level of 465. Students are full-time and part-time, balancing the demands of family, work, and school. Students take lower-division transfer credits, train or retrain for careers, update their skills, learn English, earn a GED, or otherwise enrich their knowledge. Because OCCC is a small college, students experience more direct contact with faculty and support staff, allowing them to achieve their educational goals in a supportive academic environment and caring social atmosphere
Throughout its history, the College has provided courses and programs to meet the educational and economic development needs of Lincoln County. Transfer pathways have become more robust over time, with most transfer students going on to attend Oregon State University or Western Oregon University. Many students also attend OCCC for career preparation. Via the Associate of Applied Science in Nursing more than 250 OCCC students have earned their degrees, helping to meet the healthcare needs of Lincoln County and beyond. OCCC’s Aquarium Science program is one of only two of its kind in the United States. In recent years, OCCC has launched programs in business, teacher-education (K-12 and early childhood education), welding, and computer science. Each of these new programs was carefully selected based on employment projections and community supports, and mission-aligned grants were strategically secured to fund start-up costs. The College works closely with key partners such as the Lincoln County School District, the Port of Toledo, Georgia Pacific, Lincoln County Economic Development Alliance, Oregon State University, Northwest Oregon Works, the Yaquina Bay Economic Foundation, Samaritan Hospitals, Confederated Tribes of Siletz Indians, and Centro de Ayuda.
Major Areas of Focus Since Last Visit
OCCC was last reviewed by NWCCU evaluators in 2019, resulting in the granting of our initial independent accreditation in February of 2020. Work and progress have continued since that time, with certain major focus areas summarized below.
Comprehensive Planning
OCCC engages in strategic planning to identify five-to-seven-year priorities and goals that advance mission and core themes. The Five Big Ideas Strategic Framework guided the work of the college since 2015, with gaining independent regional accreditation as one of the key outcomes (along with student success, commitment to planning, great place to work, and agile organization). In 2021 the OCCC Board of Education completed a review of the College Mission (adopted in 2015). The 2021 review resulted in inclusion of the phrase “a commitment to equitable outcomes.” In 2022, the President led the College community in a strategic planning process to develop the next College Strategic Plan 2023-2028 (CSP). As of the submission of this report, the Board of Education has endorsed five Strategic Priorities that will be the cornerstone of the College Strategic Plan (2023-2028). Through a comprehensive planning process, OCCC strives to operationalize the philosophical principle of being a student-centered college. Centering our work around students is called out in the College mission, the strategic plan, budgeting, assessment mechanisms, and various institutional and operational plans. A more thorough examination of how planning is maturing at OCCC is provided in Addendum 3: Integrated Planning.
Student-Centered College
One important example of this student-centered focus is the evolution of the delivery of services by Student Affairs. Early in 2022 the Division of Student Affairs redesigned OCCC’s recruitment, financial aid, admissions, advising and student success practices to better serve students. The new model provides students with personalized service from the very start, throughout their academic career to the attainment of their educational goals at OCCC. Prior to the redesign, student-support staff specialized by functions such as high-school outreach, advising, and financial aid. These positions have transitioned to a student success coach model. Dedicated coaches now work with students over the entirety of their time with OCCC. Ultimately, the team of four student success coaches will be cross trained to support students from the start of their post-secondary journey to the end, directly assisting with processes such as admissions, registration, applying for financial aid, and career planning. A new position, the student resource navigator, provides students with support to address non-academic issues that may present a barrier to degree completion.
Institutionalizing, Sustaining, and Integrating New Systems
While OCCC has been in operation and offering degrees and certificates since its founding in 1987, many important elements of infrastructure were delivered via a contract for services with other larger Oregon community colleges (most recently Portland Community College). Contracted functions included Office of the Registrar, Student Financial Aid, and many Office of Instruction functions such as curriculum approvals and assessment of student learning outcomes. To apply for and receive approval from the Department of Education to operate as an independent Title IV institution, many complex new systems comprised of new staff positions, policies and practices, and applications software, have been developed and implemented. Since OCCC was recognized as an independent regionally accredited college by NWCCU in early 2020, we have established an Office of the Registrar and a Financial Aid Office which functions in partnership with a third-party processor (Global Financial) and have completed the buildout of the Office of Instruction to include curriculum management and assessment.
Enterprise Resource Planning (ERP) System Migration
In 2018, OCCC learned our Enterprise Resource Planning (ERP) system “SharkNet” would be discontinued. This placed OCCC in a reactive mode for large-scale change requiring the reallocation of major fiscal and human resources which began in 2019. The impact of migration to a new Student Information System (SIS) has been profound, compounded by pandemic conditions and a move to remote work for much of 2020-2021. Unfortunately, the implementation of certain applications (student information system, financial aid, registrar functions) continues to require staff time to diagnose, test, and work with the vendor to resolve or devise workarounds for ongoing and lingering issues. Beyond the SIS, two additional modules are now being implemented: Financial and “Reach,” the contact-management module within the ERP.
Pandemic: From Reaction to Adaptation & Growth
Technology’s importance in the delivery of education has been on an upward trajectory for several decades. For many institutions, including OCCC, technology took on even greater importance during the pandemic, at times providing the only access point for many students. Pandemic conditions further demonstrated the potential of technology as a vehicle to enrich teaching and learning. However, many individuals and communities were unable to adequately access critical online services during the pandemic, which OCCC strived to mitigate for students. Now that we are transitioning from pandemic to endemic conditions, the need for technology mediated instruction and student services continues. In fact, expectations have risen not only regarding access, but as a sea-change in instructional delivery which now must support instruction and student support services distributed across time and space.
Framework for Ongoing Accreditation Efforts
OCCC continues to use its two core themes to assess efficacy: Student Success and Educational Pathways. The core themes incorporate the mechanisms by which we assess mission fulfillment and institutional effectiveness within the standards established by NWCCU. As a rural community college, there are many vital functions we fulfill for our community, for example economic development and civic engagement, which are not fully captured within this NWCCU-driven framework, and are assessed via other methods. Our core theme framework was first developed in 2015, and has undergone review and revision since that time, most recently following upon the Board mission review process. The following definitions and “Over-Arching Objectives” reflect the latest refinements of the Core Themes.
Student Success Definition:
At Oregon Coast Community College, we equip students for success in college and in life by providing exemplary teaching, student development programs, and support services. Students receive customized and relevant advising and enriched supports to maximize completion and success. In response to the diverse needs and histories of our community, we are institutionalizing a philosophy of student success and strengthening the College’s policies, processes, and business practices to facilitate equitable outcomes.
Student Success Over-Arching Objective:
OCCC will improve post-secondary educational attainment across Lincoln County and close achievement gaps for underserved populations in our community.
Educational Pathways Definition:
At Oregon Coast Community College, we assess the needs of individuals and employers, and respond by designing pathways and partnerships that address community and regional priorities. We create bridges into our credit pathways from high school – and other feeders – and facilitate transitions to transfer or employment. By narrowing achievement gaps in post-secondary education and raising post-secondary educational attainment, we advance the economic and civic vitality of Lincoln County and beyond.
Educational Pathways Over-Arching objective:
OCCC will offer rigorous and engaging academic programs and educational options comprised of clear pathways to transfer and employment that enrich individual lives and promote the economic and civic vitality of Lincoln County and beyond.
Core Theme Assessment Tables provide indicators of achievement and corresponding achievement thresholds for each sub-objective, along with a brief rationale for the set of indicators, with methodology for the indicator if not self-evident. This framework places high value on the disaggregation of data within the various indicators, used during the evaluation process to identify potential inequities in access and to examine presence of achievement gaps within underserved populations. While disaggregated data is relied on during assessment and evaluation, the aggregated data is currently used for assessment of overall performance on an indicator.
.
Student Achievement
As part of its ongoing self-reflection and evaluation of mission fulfillment, OCCC utilizes data from student achievement measures, and comparative data and information from five Oregon community colleges identified as peer colleges, to benchmark student achievement. Gap analysis is conducted utilizing disaggregated student achievement data.
These whole-college student achievement measures are utilized in conjunction with assessment of student achievement of learning outcomes (at the course, program or discipline, and institutional level) and the College’s Service and Program Area Review (SPAR) reports to gain a comprehensive view of student achievement at OCCC (see link: Assessment Plan and Handbook). Data from whole-college student achievement measures, aggregated and disaggregated, are reviewed and discussed with faculty and staff at in-service sessions, with feedback solicited to identify potential strategies to foster continuous improvement, and are used to inform the development of funding priorities for coming budget years.
Funding priorities based on results from student achievement measures are linked to the planning process through the College’s SPAR process (see Addendum 1: System of Evaluation of Programs and Services), with funding requests from SPAR reports reviewed by the Executive Team and funded based on the request’s relevance to the mitigation of identified achievement and equity gaps, and improvements in student achievement. A further link, at the comprehensive instructional program review level (SPARC), exists with the analysis and evaluation of disaggregated student achievement measures at the program level on the basis of gender and race/ethnicity. These measures include student enrollment, student retention, and student successful completion rate, with programs asked to provide strategies for improved support for specific student populations to close identified achievement gaps.
Comprehensive Student Achievement Measures
Core themes, objectives, indicators, and measures of achievement are published and available on the College Institutional Assessment webpage. To identify achievement gaps between student populations, performance measures for student engagement, retention/persistence, pace and completion are disaggregated by race/ethnicity, age, gender, socioeconomic/Pell status, first generation college status, and geography. The College regularly uses benchmark comparisons in planning, assessment, and review of student achievement, with the benchmark comparisons publicly available on the college’s Institutional Effectiveness webpage.
Benchmark colleges have been selected using the Voluntary Framework of Accountability (VFA) Benchmarking tool for comparator single-campus, rural/town colleges of similar size and student demographics. The benchmarking tool returned eight colleges – five of which are in Oregon. These five Oregon colleges were selected as most useful for comparison to OCCC.
These five colleges also participate in the Oregon Guided Pathways network and track many of the key performance indicators (KPIs) that align with OCCC indicators from the OCCC Core Theme 1: Student Success. Oregon Community College Presidents are currently drafting an MOU that will promote data sharing among Oregon Guided Pathways Colleges for the purposes of benchmarking. While targets for many of the student achievement metrics are currently set from the average of the last three years of Guided Pathways KPIs, it is anticipated that the Colleges in the Oregon Guided Pathways network will begin sharing their data publicly, facilitating the revision of targets for evaluation of performance against comparator colleges.
Institutional performance on student achievement metrics has been greatly impacted by the pandemic, with declines across all metrics across all groups from 2019-20 to 2020-21 (see summary table, below). Though most groups have shown some recovery from 2020-21 to 2021-22, Women, People of Color, and traditionally aged college students are disproportionately vulnerable to loss during their first term of attendance. Traditionally aged male students show lower levels of first year credit accumulation and year-to-year retention. Institutional responses (in development) are tailored to positively impact these student populations. For example: to address poor credit accumulation and retention among traditionally aged male students, the College is implementing a mentoring program pairing male students with working professionals.
Full analysis of student achievement measures with peer comparisons (when available) and graphs depicting disaggregation of special populations can be found on the Institutional Assessment webpage. Next steps include convening internal and external stakeholders to develop support strategies and review institutional practices that may be adversely affecting vulnerable populations.
Due to reporting and testing cycles, analysis of some Core Theme indicators (OCCC Core Theme 2: Educational Pathways) related to student achievement remains underway. These indicators are:
- Professional Licensing
- Employment
- Transfer to 4-year Institution
As a newly independently accredited college, numerically substantive transfer data as reported through the National Student Clearinghouse will not be available for several years, until a more substantial number of graduates have completed programs and transferred to a four-year college. The College is investigating alternate methods of soliciting intent to transfer information from completing students.
In sum, the College has established a strong system of data collection, analysis, dissemination, and implementation of intervention strategies to ensure continuous improvement of student achievement. The student achievement indicators, combined with the other Core Theme indicators, are used to assess mission fulfillment, with their value being enhanced by data sharing among the five Oregon Guided Pathways colleges identified above.
Programmatic Assessment
OCCC has implemented a comprehensive system of program level assessment (SPAR – Service and Program Area Review) to guide institutional planning and evaluate student learning. A detailed analysis of the College’s SPAR process is provided in Addendum 1 (System of Evaluation of Programs and Services), supplemented by the OCCC Assessment Plan and Handbook.
Transfer disciplines aligned to general education requirements and Career and Technical Education programs engage in instructional program assessment that is distinct from the process utilized by non-instructional service areas. Instructional program assessment utilizes student learning outcomes and data trends to identify and close equity gaps, and to improve teaching and learning. Faculty use data collection forms to gather student performance on Comprehensive Institutional Learning Outcomes (CILOs), Course Learning Outcomes (CLOs) and Discipline/Program Learning Outcomes (PLOs), analyze data, and create plans for change. When planning for implementation, funding requests supported by the programmatic assessment – and related to continuous improvement of student learning outcomes and achievement – are reviewed by instructional administrators prior to review by the Executive Team.
Non-instructional service areas work with area supervisors to set yearly goals that support institutional Core Theme objectives and/or elements of the College Strategic Plan. Each service area has established service area outcomes and assessment methods for these outcomes, with assessment results reported in the annual reviews and the comprehensive, four-year report. Service areas also assess goal attainment from the prior year, establish or update new goals for the current year, and develop funding requests for the upcoming fiscal year to strategically support goal attainment.
With the recent achievement of independent accreditation, and with the College assuming responsibility for some operational functions that had previously been the responsibility of a contracting college, the assessment of service areas has been integral to guiding improvement plans and evaluating institutional performance, integrity, and mission fulfillment.
Comprehensive programmatic assessment (SPARC) occurs every four years, with more-focused annual assessments reporting progress toward goal attainment in the intervening years (SPAREs). Program assessments are reviewed by the College Executive Team (the president and vice presidents) and inform both the institutional budgeting process and strategic and operational planning. The updated schedules for Service Area and Instructional Program SPARE/SPARC submission are presented below:
A summary of institutional efforts toward continuous improvement as an outgrowth of instructional and service area program assessment and review is provided below, with two examples of SPARCs: Student Services: Advising Service Area and Aquarium Sciences Instructional Program.
Example 1 (Instructional SPARC): Aquarium Science Program Assessment
The Aquarium Science program is a nationally recognized, comprehensive two-year degree of both theory and practical experience designed to prepare students for a career in aquatic animal husbandry. Students that have already earned a four-year degree in one of the Natural Sciences may earn the Certificate in Aquarium Science through the program. The program employs both formative and summative assessment through its science core, program core, internship courses, and industry certifications. Additionally, the program engages in a yearly, voluntary assessment of its curriculum, outcomes, policies, and facilities by peer evaluators from aquariums across the country. While program completion rates for certificate students hover near 100% in most years, completion rates for AAS candidates are considerably lower.
The AQS program review shows faculty are implementing the use of course learning outcomes (CLOs) and disaggregated student achievement metrics to assess courses and implement continuous improvement practices at the program level. For example, while course and program completion rates for male and female students are similar, there is a trend for male AAS students to return at a higher rate after the first year. As studies have indicated that female students in the sciences are less likely to ask questions in departmental seminars, the program has added additional instructional support (aquarists) to work with students outside of class hours.
Course-level improvements are documented in CLO reports and inform continuous improvement at the course and program level. Examples include the recent year-long process of adjusting course prerequisites, course content, course credits, learning outcomes, and course sequencing to accommodate a new course targeting weaknesses identified in CLOs. This also led to the incorporation of a new industry certification (AALSO Water Quality Analyst Level 1) exam that will be used to further assess program improvement in the identified area of weakness (Water Chemistry). Finally, the comprehensive four-year program review illustrates specific actions the program has taken to expand racial/ethnic diversity both within the program (83% white) and within the industry, such as the expansion of its geographic recruiting efforts, the development of a Diversity in Aquarium Science subcommittee of its Local Advisory Committee, and membership in the Minorities in Aquarium and Zoo Science organization.
Example 2 (Service Area SPARC): Academic Advising Program Assessment
In 2021, the Division of Student Affairs completed a Comprehensive Service Area Program Review (SPARC) for the Academic Advising Office. This review aimed to identify program improvements, align goals and objectives to the College’s Core Themes, and plan program improvements. The Academic Advising Office provides support to all students by advising and guiding students as well as providing information to students in areas related to academic programs, course options, and academic policies. Academic Advising Specialists monitor early alerts and refer students to other support services on-campus. The assessment tools used during the review included surveys and feedback from advising sessions, new student orientations, and strategic-planning sessions. This service area review process identified how the advising office is the singular student support service that helps students navigate degree and course options. The singularity failed to meet students’ overall needs, a significant weakness.
As part of the program’s process improvement plan, the Division of Student Affairs implemented a comprehensive and integrated Student Success coaching model to assist students with a broad range of relevant academic and non-academic support services. The model created Student Success Coaches that support student recruitment, financial, and academic advising. The changes support the College’s Core Themes: Student Success and Educational Pathways. The model allows students to feel welcomed, included, engaged, and supported by an assigned coach throughout their academic experience, from recruitment to graduation. The model facilitates a better way for students to transition to OCCC from high school, with student success coaches assigned to local high schools to advise onsite. Coaches can now build strong relationships with students that lead to increased academic involvement that will boost persistence and degree completion.
The two comprehensive program reviews discussed above broadly represent institutional efforts toward continuous improvement in instructional programs and service areas, in response to assessment of learning outcomes and progress in achievement of goals and objectives.
Moving Forward
OCCC will be prepared for a successful Year Seven review by centering all our efforts on equitable success for students, supported by comprehensive planning and the systematic institutionalization and integration of recently developed systems.
The following are important areas of focus for OCCC over the next four years, and progress will be reflected in the Year Seven report and visit (2026):
- Launch and implement the College Strategic Plan (2023-2028) with annual implementation targets guiding tangible progress towards the five Strategic Priorities.
- Students at the Center
- Resourced and Ready
- Careers Built Here
- First Choice, Best Choice
- Sharks Make Their Marks
- Subject to Board approval, develop and launch a capital construction local bond measure to obtain funding to complete a new building focused on Career Technical programs. When successful, engage stakeholders to design and construct the building.
- Systematize the integration of planning and goal setting, assessment, and linkage to budget throughout all levels of planning, ensuring that all levels of planning support and advance Mission, Core Themes and the College’s Strategic Plan
- Increase learning initiatives focused on equity and inclusion and evaluate their effectiveness.
- Mentoring program for male students
- Customized outreach to under-engaged populations (no college, some college no degree/certificate, no HS diploma)
- Improve student achievement in retention, pace, and graduation. Planned strategies include:
- Form a Recruitment and Retention Committee with faculty representation from program/degree clusters
- Instruction, faculty and Student Affairs to create a comprehensive retention plan
- Reduce high DFW rates
- Expand career exploration and integrate into first year/first term courses and program/degree clusters
- Articulate clear paths to certificate/degree attainment that link to a reliable two-year schedule
- Explore Credit for Prior Learning and Competency Based Education to shorten time to degree
- Improve student engagement, satisfaction, and learning achievement
- Provide expanded opportunities for faculty professional development to promote student engagement and achievement
- Use technology to engage and connect with students learning both within and outside the classroom
- Further refine assessment process to improve the analysis of student learning, improve student attainment of educational goals, and close achievement gaps
- Continue to examine and enhance the student journey and experience to remove/reduce barriers, and increase engagement and sense of belonging
- Enhance the new-student onboarding experience based on individual student academic pathways.
- Develop a student-leadership program to increase student involvement and engagement.
- Expand programs and pathways of interest to students and meet the needs of the region
- Develop labor-market-informed pathways that meet regional and local economic needs and offer flexible learning models
- Develop non-credit programming (non-credit training certificates) that are pathways to degrees, career advancement, and/or workforce development
- Conduct Program Reviews with all programs (Instructional and Service Areas) as scheduled, ensuring outcomes associated with all degree programs are assessed on a regular basis, that the results of learning outcomes are reported, and that appropriate improvement plans are documented and implemented with support from the Office of Instruction and the Executive Team.
- Ensure systems of employee performance evaluation, professional development, and employee training and support are all operating effectively and contributing to a satisfied and engaged workforce.
- Implementation of the new ERP Finance module to replace Excel as the tool for:
- Budget development
- Multi-year forecasts and projections
- Cost analyses
- Grant accounting and reporting
- Management reporting
- Update Facility Master Plan to include a new CTE building at the Central Campus and a long-term plan for the South County Center. Develop a five-year deferred maintenance plan and resource request system.
- Business and Workforce Engagement
- Develop new SBDC-led, industry-specific training programs for County businesses
- Conduct needs-assessment outreach with local businesses (and agencies, including NOAA, ODFW, etc.) and partners (HMSC, Oregon Ocean Innovation Hub, etc.)
- Engage with Academic Affairs to develop noncredit certificate programs where OCCC capacity meets local industry needs
- Integrate SBDC planning with College planning
- Align SBDC strategic plan more closely with OCCC’s Strategic Priorities
- Engage All-Managers Group in SBDC strategic planning discussions
- Continue ongoing efforts to cultivate stronger relationships with, and services to, Lincoln County’s under-represented entrepreneur community.
- Develop new SBDC-led, industry-specific training programs for County businesses
In Closing
OCCC is pleased to share our progress and plans with NWCCU and our Mid-Cycle peer evaluators. We hope this report has been effective in conveying the profound organizational change of the past eight years. While OCCC was founded in 1987, there are many parallels in the past eight years to founding a new college entirely. We benefit, however, from the significant achievements that came before: the audacious idea that this rural edge of Oregon should have its own college, the beautiful learning spaces constructed in 2008-2009, and the “scrappy” (in the best sense of the word) culture all the people of OCCC have nurtured and sustained. As we look ahead to our Year Seven Evaluation of Institutional Effectiveness, we remain in a place of organizational evolution. For the past eight years, we devised and built the systems and culture of a truly independent college. Now we are engaged with assessing the efficacy of the systems we have built, and ensuring change is institutionalized. This is the next phase of organizational growth for OCCC. Our north star remains student success and equity, a focus which continues to inspire us to persevere through the considerable efforts behind us and ahead.
The next section of this report provides updates and status on the recommendations provided by the Northwest Commission on Colleges and Universities when we were awarded independent accreditation. Two of the four recommendations augment the focus areas of this Mid Cycle Evaluation Report: Student Achievement/Student Learning Outcomes, and Programmatic Assessment.
Addenda of Recommendations
Introduction
Communications from the Northwest Commission on Colleges and Universities (NWCCU), dated Feb. 3, 2020, documented the action taken by the Commission at its meeting in January 2020, granting accreditation to Oregon Coast Community College (OCCC) based upon the Fall 2019 Initial Accreditation Evaluation of OCCC. The letter also provided OCCC with five Recommendations Substantially in Compliance But in Need of Improvement. A subsequent NWCCU letter, dated July 23, 2021, documented receipt of OCCC’s Ad Hoc Report and noted that Recommendation 5: Fall 2019 Initial Accreditation had been fulfilled. The five recommendations are listed below, with an addendum provided for each of the four remaining recommendations, all scheduled for review as part of this mid-cycle evaluation.
Recommendations Substantially in Compliance But in Need of Improvement:
The Commission recommends that Oregon Coast Community College:
- Fully implement the system of evaluation of its programs and services to evaluate achievement of clearly identified program goals or intended outcomes and use the results of its assessments of services for improvement by planning, decision-making and allocation of resources and capacity (2020 Standard: 1.B.1, 1.B.3)
- Refine and fully implement the system for assessing student learning outcomes for courses, programs, and degrees, utilizing the results to inform academic learning, support planning, and practices leading to the enhancement of student learning achievement (2020 Standards: 1.C.5, 1.C.7).
- Develop a purposeful, systematic, integrated, and comprehensive plan articulating priorities and guiding decisions on resource allocation and application of institutional capacity (2020 Standards: 1.B.3)
- Assess the performance evaluation process, implementation, and feedback mechanisms, professional growth and development, and support resources for employees necessary for improvement of its operational functions (2020 Standards: 2.F.2, 2.F.4)
Develop, implement, and review a comprehensive technology infrastructure plan (2020 Standards: 2.I.1) NWCCU since noted this recommendation as fulfilled. The updated OCCC Information Technology Operations Plan (2022-2028) can be accessed here.
Addendum 1: System of Evaluation of Programs and Services
Recommendation: Fully implement the system of evaluation of its programs and services to evaluate achievement of clearly identified program goals or intended outcomes and use the results of its assessments of services for improvement by planning, decision-making and allocation of resources and capacity.
Relevant Standard (2020)
1.B.1 The institution demonstrates a continuous process to assess institutional effectiveness, including student learning and achievement and support services. The institution uses an ongoing and systematic evaluation and planning process to inform and refine its effectiveness, assign resources, and improve student learning and achievement.
1.B.3 The institution provides evidence that its planning process is inclusive and offers opportunities for comment by appropriate constituencies, allocates necessary resources, and leads to improvement of institutional effectiveness.
Oregon Coast Community College has completed the implementation of its SPAR (Service and Program Area Review) process with the linkage of the process to the budget planning process. With the implementation of this linkage, the College has utilized SPAR results to support the achievement of program goals, intended outcomes, and service area improvements, and integrated these results into planning, decision-making, and allocation of resources and capacity.
OCCC’s Service and Program Area review process is presented in the OCCC Assessment Plan and Handbook, which was revised in 2022 to reflect updates to the process.
The SPAR process encompasses the ongoing and systematic evaluation of all College instructional programs and service areas, combining long-term planning with interim reports. A Comprehensive Service and Program Area Review (SPARC) is conducted by program or area personnel every four years. The SPARC is supplemented by the Service and Program Annual Review and Evaluation (SPARE), which provides an annual progress report toward achievement of goals and outcomes.
SPARC and SPARE report templates are specific to instructional programs (SPARE template; SPARC template) or service areas (SPARE template; SPARC template) and are designed with flexibility to allow for the unique functions and needs of each unit. Both reports integrate the assessment and evaluation of program/service area goals and intended outcomes, with progress toward goals and outcomes achievement reported on an annual basis. These templates were updated in 2022 for programs and service areas to establish long- and short-term objectives in support of equity and inclusion and to reflect the goal of equitable outcomes for all students that was added to the college mission statement.
Instructional program review and evaluation is conducted by program or discipline faculty, who collaborate in setting four-year goals and short-term objectives for the program. Program faculty review progress toward achievement of program goals and objectives annually (SPARE), with an update provided to the Vice President of Academic Affairs. The SPARE report includes an evaluation of student achievement of intended Program Learning Outcomes, and a report on changes made to program instruction during the year to foster continued improvement in student success.
Similarly, service area review and evaluations are compiled by area staff and supervisors and submitted to the member of the College Executive Team with oversight of the area. Service areas develop intended outcomes for their areas and methods of assessing these outcomes and submit a report on progress toward outcomes achievement in their annual SPARE report. Service areas also develop goals and objectives for their area and assess progress toward meeting these goals.
Comprehensive reviews and annual updates include a section for resource allocation requests to support program/service area and College goals in support of Core Theme objectives and Strategic Priorities or for identification of unmet needs. Each vice president or area supervisor reviews the evaluation with program or service area personnel and prioritizes funding requests based on their relation to College strategic priorities and core themes, and with a focus on continuous improvement of student learning and student support services. Resource allocation requests are then taken to Executive Team meetings for joint review and prioritization by members and forwarded to the Budget Planning Committee. Funding decisions are based on relation to strategic priorities and improvement in student learning, balanced by revenue projections at the state and local level.
Linking Program/Service Area Evaluation with Resource Allocation
At the time of the initial accreditation visit the College had completed the development of this process but had not fully closed the loop on its implementation. Implementation efforts were hampered by the realization that the timelines of the SPAR process and that of the Budget Planning Process were misaligned, with the result that SPAR resource allocation requests for the coming budget year were not available with sufficient advance timing for Executive Team and Budget Planning Committee members to utilize in the budget recommendation and formulation process.
College budgets are typically planned and formulated one year in advance for a subsequent year, meaning that evaluative information for resource allocation must be available one year in advance for its utilization in decision-making. The submission timeline for program and service area reviews therefore needed adjustment. Previously at the end of the academic year, the deadline was shifted forward to the months of August and September (start of the academic year), for funding consideration for the formulation of the budget for the coming (subsequent) academic year.
Shift in Submission Schedule for Individual Programs/Service Areas
Further adjustments were necessary to the submission timeline for programs and service areas. The College had originally envisioned a phased implementation for submission of SPARE/SPARC reports, with selected programs and service areas submitting either a SPARE or SPARC in the 2021-22 academic year. This would have enabled an evaluation of the process as it was implemented in the 2019-20 and 2020-21 years, with any identified desirable improvements being implemented during that time. However, before the first full cycle was implemented with the linkage to the budget planning process, it was recognized that this phased implementation would not yield the whole-college view necessary for budget decision-makers and would also not provide all programs and service areas input into the budget planning process for the first cycle.
To ensure a baseline planning platform and provide for College-wide input into the planning and resource allocation process, all programs and service areas were asked to submit, if they were not scheduled for a comprehensive review, a two-year SPARE report, due in August 2021. This initial two-year SPARE provided, at once, a retrospective review of the previous year (2020-21) and a forward-looking perspective of the 2021-22 year for each area submitting the report, with resource allocation requests for the 2022-23 year. These submissions would then be available to meet the adjusted timeline necessary for decision-makers and the budget planning process, with funds disbursement occurring in July of the coming year.
Development of Service Area Outcomes, Goals & Objectives
This new SPARE/SPARC submission schedule (see above, Programmatic Assessment section) also necessitated the accelerated development of goals and objectives for all program and service areas, and, for the service areas, the development of outcomes and assessment methods. While service areas have completed this development and reported these outcomes, goals and objectives in their submitted reports, some areas may choose to further refine these outcomes in subsequent submissions. In addition, the Registrar Office, new since independent accreditation, is still in the process of implementing an outcomes assessment survey. This survey will be implemented in Fall 2022.
2021-22 First Cycle Completion
In accordance with the new schedule, SPARE/SPARC reports were submitted to the area supervisor in August 2021. The reports were reviewed by the area supervisor or Vice President, informed by meetings with area personnel, with a discussion of resource requests and their relation to College Strategic Priorities and Core Theme sub-objectives. During the Fall 2021 term these SPARE/SPARCs were available for the Executive Team for review and discussion for planning purposes, prioritization of funding requests based on planning priorities, and identification of funding possibilities for those approved requests. These decisions and recommendations were then available during Winter and Spring terms to the Budget Planning Committee for budget planning and formulation, and submission to the budget approval process. The 2022-23 budget was approved prior to the end of June 2022, with area supervisors notifying their personnel of the status of their funding requests.
Prioritized and funded resource requests were funded from either the 2021-22 or 2022-23 General Fund or identified as meeting a grant-funded opportunity. Appendix 1 presents the requests funded through the General Fund and those projected for grant funding, along with the Core Theme Sub-objective or Strategic Priority these funded requests support.
In Summary
With the completion of the first full cycle of the SPAR process, culminating in the linkage to resource allocation and planning, the design and implementation of the process is substantially complete, and the next cycle of SPARE/SPARC submission is in progress. As with all processes, there will be opportunities for continuous improvement and further refinement. For example, some service areas are interested in revising their service area outcomes and improving their assessment tools. The College will continue to work on improving the process as the cycle progresses, to inform planning, decision-making and allocation of resources and capacity.
Addendum 2: Refine and Fully Implement Assessment
of Student Learning Outcomes
Recommendation: Refine and fully implement the system for assessing student learning outcomes for courses, programs, and degrees, utilizing the results to inform academic learning, support planning, and practices leading to the enhancement of student learning achievement.
Relevant Standards (2020):
1.C.5 The institution engages in an effective system of assessment to evaluate the quality of learning in its programs. The institution recognizes the central role of faculty to establish curricula, assessing student learning, and improving instructional programs.
1.C.7 The institution uses the results of its assessment efforts to inform academic and learning-support planning and practices to continuously improve student learning.
Oregon Coast Community College has substantially completed the implementation of its assessment plan and process, and has identified, through analysis of the process to date, further refinements to improve the quality of information provided through these assessments and to enhance effective collaboration among faculty groups on improvement strategies.
The College’s Assessment Plan and Handbook, updated in 2022, details the process of learning outcomes assessment and program review, and is formulated as a resource for faculty and staff.
All student learning outcomes, including course learning outcomes (CLOs), program/discipline learning outcomes (PLOs, DLOs), and comprehensive institutional learning outcomes (CILOs), are assessed on a quarterly basis by faculty utilizing the outcomes assessment forms posted on the College website (see “Assessment” section, here). Using these forms, faculty document the number of students who successfully completed the class (grade “C” or better) and who are assessed as having achieved the learning outcomes at four levels: Emerging, Developing, Competent, and Proficient. Based on these assessments results, faculty also respond to the following narrative questions:
- Are you comfortable with the wording of your course learning outcomes? Do they capture what you want to capture? Are they measurable? Or would you like to make some changes to them?
- What did you discover about student learning based on the assessment you conducted? Where did students exceed your expectations and where did they fall short? What surprised you in this term?
- When you think about teaching this course the next time, what might you change in the way you assess your outcomes? In the way you might present your material?
- Have you made changes this term based on prior assessments? If so, what impact have these changes made on student achievement?
Faculty responses to these questions provide documentation of analysis of assessment results and adjustments to instruction, content or instructional methods the faculty member plans for the next term to improve student achievement of outcomes. They also provide the Assessment Task Force feedback on assistance that can be provided for faculty in formulating improved outcomes or in facilitating collaborative faculty forums for discussion and review. See Appendix 2: Sample Spring 2022 Selected Faculty Narrative Responses to Assessment Questions.
This same process, utilizing the same assessment forms, is utilized by faculty for evaluating program/discipline learning outcomes. All course learning outcomes have been analyzed for their relevance and contribution to program/discipline learning outcomes and then “mapped” to these broader learning outcomes. Student achievement of program/discipline learning outcomes is evaluated using the same four categories of Emerging, Developing, Competent, or Proficient. Program or discipline faculty analyze these results together and report improvement strategies in the annual and comprehensive program reviews (SPARE/SPARC) for their area, accompanied by resource requests, if deemed necessary, to support these strategies. (See sample SPARE, Medical Assisting.)
Comprehensive Institutional Learning Outcomes (CILOs) – Two-year Degree Programs
CILOs are assessed using a different outcomes assessment form, utilizing the AACU VALUE rubrics which have been edited and adapted by OCCC faculty for this purpose. These rubrics identify specific skills and abilities which contribute to the achievement of the much broader, more general CILOs of Written and Oral Communication, Cultural Awareness, Problem-solving Skills, and Personal Accountability. At the time of the initial accreditation visit, faculty had completed the revision and adaptation of the VALUE rubric for the Communication CILO. The rubrics for all institutional learning outcomes have now been completed and are in use by faculty and posted on the College website for their reference. (See Learning Outcomes Assessment.)
The breadth of a Comprehensive Institutional Learning Outcome means that the knowledge or skill set it represents is contributed to by a cross-disciplinary group of courses and across the student’s entire College experience. To assess CILOs, faculty and the Assessment Task Force identified a set of General Education courses common to two-year degree program requirements, and technical program courses required for two-year technical degrees, which most directly contribute to the achievement of the broader CILOs. Student CILO achievement is assessed in these contributing courses on a quarterly basis or each time the relevant course is taught, using the CILO assessment form. Students are evaluated on all sub-categories within each specific VALUE rubric, using the same four achievement levels utilized for course and program/discipline learning outcomes.
Student achievement of comprehensive institutional outcomes is recognized as being essential for continuing student success and lifelong learning. Graduate attainment of CILOs is a designated sub-objective for the Educational Pathways Core Theme, with the expectation that 70% of students with 65 or more credits achieve at least a level of Competent or Proficient on the CILO rubrics. While this threshold is utilized for evaluating achievement of the Core Theme sub-objective, faculty assess all students enrolled in the associated courses to have the most complete information available, and to provide for a better analysis and formulation of improvement strategies.
With the completion of the VALUE rubrics and identification of courses for CILO assessment, all CILOs were assessed in the Spring 2022 term. Analysis of these assessment results indicates that the achievement threshold was met, with 70% of all students enrolled in the assessed courses meeting a level of “Competent” or “Proficient.” (See Appendix 3: Spring 2022 CILO Assessment Results).
The evaluation of student achievement of these comprehensive outcomes invites collaboration among faculty across disciplines and programs, providing the strongest opportunity for meaningful discussion, understanding of student learning, and program improvement. Since these assessments were completed at end of the academic year in Spring term 2022 and submitted after grades with faculty leaving for the summer, discussion and evaluation of the CILO assessments is scheduled for Fall In-service in September 2022.
Use and Sharing of Assessment Results
As the development of the assessment process progressed toward completion in the 2020-21 and 2021-22 academic years, forums for the use and sharing of these results were also developed or improved. Originally scheduled during Fall In-service sessions, outcomes assessment discussion and review is now a part of both Spring and Fall in-service scheduled sessions.
In-service sessions cover a variety of topics. The ATF Chair conducts general sessions, designed for all faculty but highly recommended for new faculty, that focus on how to assess student achievement, how to align assignments with student learning outcomes for valid and accurate assessment, and how to create rubrics for evaluation of student achievement of course learning outcomes. These sessions are recorded and distributed to faculty who were unable to attend.
In addition to these more general In-service sessions, breakout sessions are also conducted with faculty grouped by discipline or program to discuss assessment results, improve alignment of assessment methods with specific outcomes, and to collaborate on the development of improvement strategies for student achievement. While faculty are required to attend In-service sessions, informal meetings among smaller discipline and program faculty groups also often occur during the three-day Fall In-service and throughout the academic year to ensure all program/discipline faculty have the opportunity to collaborate on review and evaluation. These sessions also provide input into the evaluation of student achievement of program learning outcomes, which is reported and analyzed in the program’s annual SPARE report.
The ATF Chair and members meet with individual and new faculty members in Zoom sessions throughout the academic year to familiarize new faculty with the process, and to give assistance to faculty members in refining student learning outcomes and the development of appropriate assessment tools that align with and accurately assess student achievement of these outcomes. The ATF Chair has recorded training videos on “Why and How to Do Course Assessment” and “How to Submit Course Assessments”, with these videos posted on the College website in the Instructional Resources/Assessment section.
Summary of Progress Since Accreditation Visit
In the academic year 2020-21, the College made the decision to change the leadership of the Assessment Task Force, previously led by a member of the administration, to a faculty member who had demonstrated proficiency in assessment and established a strong assessment record for their department. This was anticipated to be a temporary change but one that would heighten faculty engagement in the assessment process. The Chair of the Mathematics department assumed this responsibility.
The onset of the pandemic transformed the 2020-21 year, and the necessary and immediate redesign of instruction and services to a remote delivery mode placed an unprecedented demand on the resources of the College. The new ATF Chair and committee members, faculty members who themselves were adapting the delivery of their own courses and providing additional hours to students to assist them in the transition, focused committee efforts on the continuing assessment of course and program learning outcomes, the onboarding of new faculty to the process, and providing remote professional development for faculty in assessment practices. While slowed somewhat, progress continued on the review and finalizing of the AACU VALUE rubrics for Cultural Awareness and the remaining Comprehensive Institutional Outcomes.
Faculty leadership of the Assessment Task Force was converted to a permanent position, carrying faculty load release time, for the 2021-22 academic year. This change provided continuity and stability to the leadership of the Task Force, with remaining implementation as well as planning and the formulation of goals and improvement strategies able to proceed with more focus. The review and editing of AACU VALUE rubrics for all CILOs was completed, with the rubrics presented to and approved by the College Council. Courses were identified as having a significant contribution toward the development of these skills and targeted for CILO assessment. Faculty training was provided and CILO assessment forms were developed, with all CILOs being assessed at end of Spring term 2022. Results of these assessments will be evaluated and discussed, with faculty discussing improvement strategies, in Fall 2022 In-Service sessions.
At the time of initial accreditation, implementation of CILO assessment was the remaining component of OCCC’s assessment process to be completed. With the assessment of all CILOs in Spring 2022, and analysis and collaborative discussion of results scheduled for Fall 2022 In-service, implementation of the assessment process is now substantially complete. Assessment results for CLOs and DLO/PLOs are being utilized to inform academic learning and support services planning, and practices targeted to enhance student learning achievement are being identified on an ongoing basis. Changes to the program review timeline (see Addendum 1) have integrated the evaluation of program learning outcomes into the resource allocation process, with resource requests identified as contributing to learning outcomes achievement being submitted with annual program reviews.
Assessment Challenges in a Small College
Oregon Coast Community College is one of the smallest among Oregon community colleges and is characterized by small section sizes and focused scheduling for enrollment management purposes. Its small size contributes to a personalized learning and student support experience for our students. The small section size, however, yields, each assessment cycle, a low “n” to be reported at each level of learning outcomes for many classes, yielding results that are often not statistically significant and that make it difficult to extrapolate/analyze and make meaningful, generalized observations on student achievement.
Economy of scheduling means some courses, especially in the electives area, may only be offered once a year, sometimes with a student enrollment count of less than 15. For these courses, the lack of frequent and consistent assessment weakens the value of the assessment information and the ability to draw conclusions to assess the impact of improvement strategies. In addition, the College’s small size means that some instructional programs and/or “departments” are represented by only one faculty member. This presents challenges in promoting collaboration on course assessment and results analysis.
Finally, small student enrollment means that some programs may have only a few completers each year, making it difficult to accurately evaluate student learning achievement upon program/degree completion.
Goals for Continuous Improvement in Assessment Process
While implementation of the assessment process is substantially complete, the College strives to steadily improve the quality of assessment information and, by extension, its value for identifying strategies for improvement in student learning and support. The Assessment Task Force has completed an informal and preliminary review and evaluation of the assessment process to assist in identification of 2022-23 goals and planning priorities. This preliminary report will be presented at the Fall 2022 In-Service, with opportunities for input and feedback, and then subsequently submitted to OCCC’s College Council. This is planned to become an annual Assessment Review and Report to be presented for discussion at Fall In-service.
Preliminarily, the Assessment Task Force has identified, for 2022-23, the following goals or improvement strategies for improved assessment:
- With College leadership, continue to foster a culture of assessment and increase faculty participation. Faculty participation during the pandemic did not meet target participation rates, with CLO participation rates just over 50% and CILO assessment participation at 67%. There is little doubt that the extraordinary demands being placed on faculty during the pandemic had a negative impact on participation rates. The College is striving to achieve the goal of 100% faculty participation to help ameliorate the challenges of small enrollment rates and low frequency of some class offerings.
- To counter the impact of one-person departments and discipline faculty, consider the re-alignment of collaborative faculty groups according to disciplines rather than programs, for analysis and evaluation of assessment results.
- Continue the focus on professional development sessions and one-on-one coaching sessions for faculty; consider identifying faculty leaders for discipline groups to expand assessment leadership and advocacy.
- Increase number of courses identified for Cultural Awareness and Personal Responsibility CILO assessment.
- Explore methods for aggregation of assessment results within individual program/discipline learning outcomes to enhance utility of small number of data points.
- Continue to evaluate the feasibility of a transition to tracking assessments of individual students for individualized assessment of progress in learning outcome achievement. This has been identified as a possible means of improving assessment value with the challenges of small observation numbers. To date the apparent complexity of implementation of this approach, involving technical challenges and protection of student confidentiality, makes this an ongoing subject of research and evaluation.
In Summary
With the implementation of the assessment process complete, the College has moved into the continuous improvement phase on assessment practices and process. Like most small colleges, OCCC experiences challenges associated with the small enrollment numbers and department size that, while an asset in providing a personalized and supportive learning and teaching environment for students, may complicate effective analysis and evaluation of assessment results. The Assessment Task Force has formulated goals and improvement strategies to strengthen collaboration and the evaluation process and continue to increase faculty participation. The College continues to strive toward enhancement of the process, the refinement of assessment results, and their evaluation to improve student learning, support, and achievement.
Addendum 3: Integrated Planning
Recommendation: Develop a purposeful, systematic, integrated, and comprehensive plan articulating priorities and guiding decisions on resource allocation and application of institutional capacity.
Relevant Standards (2020)
1.B.3 The institution provides evidence that its planning process is inclusive and offers opportunities for comment by appropriate constituencies, allocates necessary resources, and leads to improvement of institutional effectiveness.
Summary of Planning System Development Since Last Visit
OCCC continues to refine and enhance its comprehensive planning system. Comprised of three interconnected levels of planning and assessment, the system is designed to allocate necessary resources to improve institutional effectiveness. This system evolved from 2014 to 2020, the timeframe in which OCCC was working toward and achieved independent accreditation. It was last documented in OCCC’s 2019 Self Evaluation Report. Since 2020, the planning system is increasingly being integrated throughout the institution and individual plan elements have been updated as current plans reach the end of their planning horizon. College plans are available to constituents and stakeholders via the Institutional Effectiveness webpage.
The three levels of interconnected planning and assessment are as follows:
- Comprehensive Level
- Mission/Vision/Values
- Institutional Effectiveness framework expressed via Core Themes
- College Strategic Plan
- Institutional Level
- Budget
- Enrollment
- Facilities
- Operational Level
- Accreditation
- Assessment
- Business Continuity and Emergency Response
- Information Technology
- Special project plans – which vary over time
Institutional plans (Budget, Facilities, Enrollment) function to support the Core Themes mission fulfillment framework as well as supporting and advancing the College Strategic Plan. Operational plans support and advance the College Strategic Plan. For example, the Information Technology Operational Plan situates IT plans, goals, and strategies within the College Strategic Goals.
In 2021 the OCCC Board of Education completed a review of the College Mission (adopted in 2015). The 2021 review resulted in the inclusion of the phrase “a commitment to equitable outcomes.” In 2022, the President led the College community in a strategic planning process to develop a College Strategic Priorities 2023-2028 (CSP), replacing the prior Five Big Ideas strategic framework. The CSP has five over-arching Strategic Priorities. Operational plans may address any of the five college priorities; there is no expectation that operational plans (such as Information Technology) set goals in all five priority areas.
Process and Input
The process to develop the College Strategic Plan 2023-2028 has been inclusive and to date has presented numerous opportunities for comment by appropriate constituencies including all college staff (at In-Service and numerous Town Halls), the Equity and Inclusion Committee, the Economic Development Alliance of Lincoln County, and the Board of Education and any audience via multiple Board presentations as the plan has developed. Operational plans also seek input and provide opportunities for comment from stakeholders which are typically more internal. As an example, the Information Technology Operations Plan provides narrative as to how input was solicited and what groups were consulted.
At the time of this mid-cycle report, the Strategic Priorities of the new College Strategic Plan have been accepted by the Board of Education and are being integrated into other supporting plans. Additional elements of the CSP (executive summary, year one supporting activities, and metrics) are planned to be finished in time for a Fall 2022 launch.
Strategic Priorities of the College Strategic Plan 2023-2028 (CSP):
- Students at the Center
OCCC is a Student-Centered College.
OCCC strives to provide equitable outcomes for all students; Our culture is welcoming of all and values diversity of students and employees. College systems are designed and organized for student success; Educational Pathways are designed to ease access and streamline successful attainment of student goals such as degrees, employment, or transfer.
- Resourced and Ready
Our People, Technology and Buildings are Student Ready,
The College is Fiscally Strong and Sustainable.
This strategic goal enumerates specific elements of being a student-ready college and draws attention to the centrality of sustainable financial resources. Those elements are College culture (for students and employees); professional development and training to fully optimize the many system enhancements of the past six years; and technological and physical learning environments are well-maintained and forward focused. Key elements of fiscal soundness will be the alignment of college enrollment (in times of expansion or contraction) with fiscal resources, and the cultivation of alternative sources of revenue.
- Careers Built Here
The Next CTE Building is Funded, Built and Occupied.
The primary focus of this goal is to develop and pass a Bond measure providing the match investment to fund a new building for specialized Career Technical Educational (CTE); to construct and populate the new facility; and to ensure that new and existing CTE programs (regardless of location) are equipped to provide industry-relevant learning.
- First Choice, Best Choice
Lincoln County Thinks OCCC First for Education and Employment.
We will reach out to all who may have interest in post high-school education with particular attention to those who have stepped away from education. We intend to be the first choice in Lincoln County for those seeking a college degree or certificate, workforce education, small business skills, GED, enrichment, and life-long learning. Additionally, for those seeking employment in Lincoln County, OCCC will be perceived as an employer of choice due to our culture and working environment.
- Sharks Make Their Marks
OCCC Positively Impacts Lincoln County.
Our efforts will grow the living wage workforce, and support, engage with, and grow the business community. OCCC will also serve as a convener of civil community discourse and learning. Our students and our work will have meaningful and positive impact on our home, Lincoln County, Oregon.
Integration of Planning and Resource Allocation for Improvement
of Institutional Effectiveness
Since at least 2015, budget development at OCCC has been grounded in the integration of planning and budgeting. The budget remains designed to fulfill the mission of the College, and to advance strategic priorities and mission fulfillment at OCCC framed in the two core themes of the College: Student Success and Educational Pathways.
The College utilizes a structured budget-planning process that incorporates systematic engagement at the department and executive levels and provides for additional college input via staff and student forums and community input via public hearings. In 2018-19, the College formed the Budget Advisory Committee to broaden college-wide input into – and understanding of – budget formulation. The annual budget development cycle begins in December and ends with budget adoption in June.
Service and Program Area Review and Evaluation (SPARE Process)
The SPAR (Service and Program Area Review) process links planning with resource requests and allocation at the unit level. SPARE was developed in 2018-2019 as an annual process used at OCCC for programs and specialized operational divisions to document objectives, assess progress, and identify needed resources to address gaps between objectives and outcomes. Though the onset of pandemic conditions slowed down implementation of the SPARE process, there has been meaningful progress since 2020. For the first time in 2021-22, most unit areas (instructional programs and service areas) were asked to complete a SPARE to submit resource requests as an input to the 2022-23 budget development process. All SPAREs were reviewed (and for some, refined) by the Executive Team (ET) to develop a resource request master list, adding institutional priorities which did not emerge via the SPARE process. ET then worked through this list to identify requests to be funded using available funding sources: grants if applicable, unspent 2021-22 funds, inclusion in the 2022-23 budget, or those not funded at this time. The SPARE process is now an integral part of budget development, and we intend to refine and improve the process over time.
The SPARE process completed by Human Resources serves as a strong example of the impact of planning and assessment on resource allocation. SPARE identified several gaps in desired outcomes in OCCC’s performance evaluation processes and some underlying root causes (see addendum four for further discussion). Via the SPARE the Director of Human Resources identified some resources needed to address the root causes. When the HR SPARE was reviewed by the Executive Team, they not only supported, but augmented the resource request, understanding the importance of accelerating progress in enhancing the performance evaluation outcomes. These changes were funded in the 2022-2023 Budget, and work is now underway.
In Summary
OCCC has made significant strides in maturing, enhancing and institutionalizing the comprehensive planning system first envisioned in 2015. Currently there are no major revisions planned for the planning system and processes, but rather attention and resources are focused on building out and renewing the individual components. Since the last visit by NWCCU in fall 2019, OCCC has reviewed and made a minor (but meaningful) revision to its mission statement, largely completed the next strategic planning process, developed a new six-year technology plan, and made significant strides in integrating the service/area program review process (SPARE) into budgeting. This system is working effectively. Our planning process is inclusive and offers opportunities for comment by appropriate constituencies, allocates necessary resources, and leads to improvement of institutional effectiveness.
Addendum 4: Employee Performance Evaluation and Professional Development
Recommendation: Employee Performance Evaluation and Professional Development systems. Assess the performance evaluation process, implementation, and feedback mechanisms, professional growth and development, and support resources for employees necessary for improvement of its operational functions.
Relevant Standards (2020)
2.F.2 The institution provides faculty, staff, and administrators with appropriate opportunities and support for professional growth and development.
2.F.4 Faculty, staff, and administrators are evaluated regularly and systematically in alignment with institutional mission and goals, educational objectives, and policies and procedures. Evaluations are based on written criteria that are published, easily accessible, and clearly communicated. Evaluations are applied equitably, fairly, and consistently in relation to responsibilities and duties. Personnel are assessed for effectiveness and are provided feedback and encouragement for improvement.
Performance Evaluation Processes, Systems, and Implementation at OCCC
OCCC’s performance evaluation systems are typical among Oregon community colleges. Evaluation of represented employee groups (faculty and classified) is collectively bargained, and documented in the respective contract, and any changes in the process must be bargained. There are a very limited number of positions with technical exempt or classified duties (such as confidential) whose evaluation processes are aligned with the CBA (Collective Bargaining Agreement), even though they are not part of the bargaining units. Administrative leadership employees are not part of a bargaining unit, and their current performance evaluation system is based on AACC (American Association of Community Colleges) leadership competencies.
The timely completion of performance evaluation processes has been inconsistent as documented by annual inventories completed by the HR office. The Director of Human Resources included an assessment of current evaluation systems as part of the Service/Program Area Review (SPARE) process for 2021. The HR SPARE noted: It has been difficult for OCCC managers to consistently complete employee performance evaluations, and this has held true over time. Root causes are multiple and occur within both design and implementation:
- Process design
- Ratio of evaluators/evaluatees (especially for PT faculty)
- Significant other responsibilities for managers, especially recent
accreditation, and new federal requirements - Historically high number of new hires
- A lack of automation
- Pandemic impacts
Multiple changes have occurred or are underway to address root causes. The HR SPARE (Program Review) served as an input to the budget development process which then allocated resources to allow HR to move forward with the automation of performance evaluation documentation beginning in Summer 2022, incorporating position descriptions, evaluations, tracking and reminders into the existing Paylocity payroll system. The Director of Human Resources and the VP of Academic Affairs have or will convene representative committees to collaborate on additional recommendations for changes in evaluation templates and processes, to better address the elements of NWCCU Standard 2.F.4. The faculty evaluation committee has designed a new student evaluation process and in 2022-23 will consider other components of the faculty evaluation process. The faculty group has also given considerable attention to how best to assess and provide constructive feedback to ensure that technology-mediated instruction is supported and high-quality. The VP of Academic Affairs and the Director of HR have worked together to expand the number of academic administrators who can complete faculty evaluations, particularly for part-time faculty, a group which sees higher and more frequent turnover. HR and the Classified representative committee have met and proposed refinements to the classified evaluation system. The success of these changes will be assessed over time, based on whether there is progress in the rate at which evaluations are completed on schedule and their consistency with the elements of Standard 2.F.4.
There are some indications that performance evaluation feedback mechanisms could be improved. All employees may submit a self-evaluation as part of the employee evaluation process, and some take advantage of this process. Town halls and employee meetings that ask for suggestions often bring forth comments that communication at OCCC overall could be improved. HR conducts exit interviews, and occasionally departing employees indicate they would prefer more frequent or more constructive feedback from supervisors. It is likely that some supervisors may be better versed or inclined to provide feedback more often and in a way that encourages growth. College administration plans to provide some training in 2022-2023 on how to effectively provide timely and more frequent constructive and helpful feedback to employees.
Professional Growth & Development Opportunities
Most OCCC employees have access to numerous opportunities for professional development and growth. These include participation at conferences, modest tuition reimbursement at other institutions and full waivers of OCCC tuition, and four days of annual In-Service events (Spring and Fall) during which the college is closed to students and the public so that employees can engage in learning and cross-functional collaborations. Since 2014, professional development regarding equity and inclusion has been part of each In-Service. Employees also can engage with statewide affiliation groups, which is particularly helpful for employees of a small college where there may not be a local peer group. Recent years have brought refinements to faculty professional development. The Director of Library Services and Media routinely and regularly distributes links to faculty for abundant optional (unpaid) PD opportunities. Since 2018-2019 funds for faculty professional development are formally budgeted, and self-directed applications for faculty professional development are now peer-reviewed. Faculty are routinely surveyed for desired PD topics.
Given the relatively positive state of professional development opportunities for employees at OCCC, there are no plans to convene representative committees to review PD systems at this time, allowing bandwidth to focus on the needed improvements in employee performance evaluation described above. There are a couple of refinements in professional development currently under discussion by the Executive Team. Both center on how to broaden engagement in professional development, which is currently strongest among full-time faculty, student affairs staff, and college administrators. Some support positions, particularly those that work alternate schedules, miss access to professional development, especially In-Service. Part-time faculty, while they have access to funding for conferences, may not be able to afford the unpaid time to attend. The Executive Team and the Director of HR are considering remedies.
Support Resources for Improvement of OCCC Operational Functions
The President led all employees in a College-wide SWOT (Strengths, Weaknesses, Opportunities, Threats) activity during the Fall 2021 In-Service. One theme which emerged was widespread desire for additional training on the new tools introduced in the past few years. This SWOT theme regarding operational training needs was expected. Despite many new operational training sessions implemented in recent years, the growth in new systems has been intimidating, and the two-year pandemic conditions have made training difficult.
This theme was woven with others into a strategic objective in the new College Strategic Plan, for OCCC to be “Resourced & Ready.” The college-wide strategic priorities are integrated into supporting plans as they are updated. The newly completed Information Technology Operation Plan (ITOP) is the most recent example and references IT training sessions in multiple areas.
Currently, the college provides numerous support resources to enable employees to operate effectively in their positions. These include peer-to-peer on-the-job training, SafeColleges and Vector Solutions compliance training sessions, Anthology (new ERP) training, on-boarding for new employees, IT/cybersecurity training sessions, and an HR employee resource page on the College website.
Ensuring that faculty are prepared to teach in ever more complex teaching environments is critical, and OCCC strives to stay ahead of those needs. Prior to 2020, technology-mediated instruction was mostly the province of self-selecting faculty. OCCC used Quality Matters to enable these faculty to advance their skills in on-line teaching and provided faculty training sessions in Canvas (Learning Management System/Platform) and Zoom. Regular student surveys provided feedback for faculty who had time and experience to develop their skills in a technology-mediated environment.
Then came the “Pandemic Pivot,” and all remaining faculty had to move quickly on-line. OCCC built in an extra week of Spring Break in 2020 to make that transition, but it remained a very steep learning curve. Leadership in Student and Academic Affairs conducted on-the spot assessments and observations which showed that faculty needed additional support and training to be effective in new modes of tech-enabled instruction. This led to the rapid development of a “training wheels” video (2020), and more support for live streaming. Soon the new Distance Education and Planning Committee (DEP) was formed, yielding greater standardization of the student experience.
In Summary
This recommendation deals with three related but distinct aspects of the employee experience at OCCC: performance evaluation, professional growth and development, and support resources enabling employees to do their best work for students and the College. Of these three areas, performance evaluation has been the most tenacious challenge for the College and is receiving the most attention, and assessment of outcomes and gaps. Specific changes have resulted, and more are pending. College leadership is also aware employees want and need more tools to do their work effectively, and this need is centered in the new College-wide strategic priority of “Resourced and Ready.” The comprehensive planning process described in Addendum Three will guide consideration of operational training needs across all functions of the college. And not least in this recommendation, OCCC values professional growth and development and is focusing on ensuring these opportunities are available in an equitable manner across the college.
Program or Service Area | Resource Request | Supports Core Theme and/or Strategic Priority: | Budget Year | |
Advising and Financial Aid | Success Coach Professional Development (In SPARE from both Advising and F/A) | Core Theme 1: Student Success, Sub-objectives 1,2,4; Strategic Priority 2 | 2021-22 | |
Policy Review/Creation | Core Theme 1 Student Success Sub-objectives 4,5 | 2021-22 | ||
Data Tools | 2021-22 | |||
Advising | ADA Training for 3 Staff members | Core Theme 1 Student Success: Sub-objectives 1,2,4,5 | 2021-22 | |
Registrar | Registrar: Diploma Printing and Mailing | Strategic Priority 1, Core Theme 1 Student Success Sub-objective 5 | 2021-22 | |
Registrar | Registrar: Miscellaneous Office Supplies | Core Theme 1 Student Success Sub-objective 5 | 2021-22 | |
Registrar | Registrar Supplies: Books and Manuals for Registrar reporting and operations | Strategic Priority 1; Core Theme 1 Student Success Sub-objective 5 | 2021-22 | |
Aquarium Science | Aquarium Science Investment Package (12 Dissection Trays) | Core Theme 2 Educational Pathways Sub-objectives 4,6 | 2021-22 | |
Biology (supports AQS and Allied Health) | Biological Science; Investment Package (Analytical balances, refrigerator, incubator) | Core Theme 1 Student Success Sub-objective 1; Core Theme 2 Educational Pathways Sub-objectives 3,5 | 2021-22 | |
Science | Physical Science Investment Package (Stream Table w/sand, Graduated cylinders, online simulations) | Core Theme 1 Student Success Sub-objective 1; Core Theme 2 Educational Pathways Sub-objectives 3,5 | 2021-22 | |
Welding | Professional Development for Welding Faculty – AWS Certified Welding Inspector | Core Theme 2 Educational Pathways Sub-objectives 4,6 | 2021-22 | |
Early Childhood Education | Early Childhood Education Investment Package (Pre-K Curriculum) | Core Theme 2 Educational Pathways Sub-objectives 4,6 | 2021-22 | |
Human Resources | Evaluation module in Paylocity – Annual cost | Strategic Priority 1,3 | 2021-22 | |
Technology | UPS battery replacement | Strategic Priority 4,5; Core Theme 1, Sub-objectives 4,5 | 2021-22 | |
Facilities | Facilities Refresh (Waldport) | Strategic Priorities 2,4; Core Theme 1 Student Success Sub-objectives 1,4,5 | 2021-22 | |
Aquarium Science | Full-time AQS Faculty | Core Theme 2 Educational Pathways Sub-objectives 4,5,6 | 2022-23 | |
Aquarium Science | AQS Recruitment efforts | Core Theme 2 Educational Pathways Sub-objective 1 | 2022-23 | |
Clerical support | Strategic Priority 1,2,4,5 Core Theme 1 Student Success Sub-objectives 1,2,4 | 2022-23 | ||
Facilities | Boom lift rental | Strategic Priority 4 | 2022-23 | |
Welding | Welding Faculty | Core Theme 2 Educational Pathways Sub-objectives 1,4,6 | 2022-23 | |
Distance Education | Director of Distance Education/Library Services | Strategic Priority 2,4; Core Theme 1 Student Success Sub-objectives 1,2,4,5 | 2022-23 | |
Early Childhood Education (ECE) | 0.25 FTE Education/Early Childhood Education Coordinator | Core Theme 1 Student Success Sub-objectives 2,4 | 2022-23 | |
Education/ECE | Move 0.75 FTE of Education Faculty to PT Faculty | Core Theme 1 Student Success Sub-objectives 2,4; Core Theme 2 Educational Pathways Sub-objective 1 | 2022-23 | |
Program or Service Area | Resource Request | Supports Core Theme and/or Strategic Priority: | ||
Welding | Welding Investment Package (Equipment, Trailer) | Core Theme 2 Educational Pathways Sub-objective 1 | ||
Welding | Welding Instructional Specialist (0.77 FTE) | Core Theme 2 Educational Pathways Sub-objectives 4,6 | ||
Office of Instruction | Scheduling Software | Core Theme 2 Educational Pathways Sub-objective 1 | ||
Library | Library Print Materials on Native Americans in the PNW | Strategic Priority 2; Core Theme 1 Student Success Sub-objective 1 | ||
Advising, Registrar | Degree Progress Audit Software (Conclusive) | Core Theme 1 Student Success Sub-objective 5 | ||
SBDC | .5FTE Spanish-language (SBDC Business Advisor Position) | Strategic Priorities 2,4; Core Theme 1 Student Success Sub-objectives 1,4,5 | ||
Nursing and Allied Health | Funding for nursing and allied health faculty professional development in the area of simulation ($500 per faculty) | Core Theme 1 Student Success Sub-objectives 4,5,6 | ||
Nursing and Allied Health | Nursing: Lab assistant & student worker | Core Theme 2 Educational Pathways Sub-objective 4 | ||
Nursing and Allied Health | Medical Assisting: Lab assistant & student worker | Core Theme 2 Educational Pathways Sub-objective 4 | ||
Nursing and Allied Health | Nursing: Additional hours for faculty during the summer to create assessment plans, design a more robust mentorship model and begin curriculum review | Core Theme 2 Educational Pathways Sub-objectives 4,5,6 | ||
Nursing and Allied Health | Nursing: Clinical coordinator summer hours. | Core Theme 2 Educational Pathways Sub-objectives 4,6 | ||
Instructional Programs | Additional hours for faculty during the summer to create assessment plans, design a more robust mentorship model and begin curriculum review | Core Theme 2 Educational Pathways Sub-objectives 4,5,6 | ||
Nursing and Allied Health | Medical Assisting: Clinical coordinator summer hours. | Core Theme 2 Educational Pathways Sub-objectives 4,6 | ||
Facilities | Additional Supplies Emergency preparedness supplies | Strategic Priorities 1,4 |
Big 5 Strategic Priorities | Core Theme 1: Student Success | Core Theme 2: Educational Pathways | ||
1. Independent Accreditation with NWCCU | 1. Students feel welcomed, included, engaged | 1. OCCC educational pathways are accessible | ||
2. Student Success and Intentional and Aggressive Growth in Enrollment | 2. Students transition successfully to OCCC | 2. Students graduating Lincoln County high schools enroll at OCCC | ||
3. Becoming a Great Place to Work | 3. Students complete their College level writing and math requirement in their first year | 3. Graduates attain General Education/Comprehensive Institutional Learning Outcomes | ||
4. Systematic and Comprehensive Planning | 4. Students progress/persist toward their educational goals | 4. Graduates meet industry standards by demonstrated mastery of technical skills | ||
5. Becoming an Agile and Flexible Organization | 5. Students experience academic success | 5. Programs use the results of assessment to improve teaching and learning | ||
6. Graduates of CTE programs will be employed in their field of study | ||||
7. Graduates of transfer programs will be enrolled in 4-year institutions | ||||
Appendix 2: Spring 2022 Selected Faculty Responses to Assessment Questions
- Are you comfortable with the wording of your course learning outcomes? Do they capture what you want to capture? Are they measurable, or would you like to make some changes to them?
- Yes, I am. When paired with the learning objectives and overall tasks they lend themselves to ensure alignment and to be measured.
- I have applied for course outcomes changes to be presented at ILT in October for winter changes.
- I am very comfortable with these outcomes. They work well for my curriculum and I’m able to map activities and assessments easily to them.
- Changes were recently made to the 4th Learning Outcome for MTH 111. (Not used for this Course Assessment) Now I agree with the wording for all.
- Outcomes need to be more specific and be restricted to like skills. Outcomes as stated include a mixed skill set and are also repetitive. Assessment of outcomes would be more accurate if the outcomes were stated more specifically.
- Most of the learning objectives include aspirational statements that extend beyond the scope of the course subject matter, and can’t be practically measured
- The more I work with them, the more I think they’re too broad, but yes. I am comfortable with them.
- What did you discover about student learning based on the assessment you conducted? Where did students exceed your expectations and where did they fall short? What surprised you in this term?
- My students are still working their way out of a pandemic educational experience. More than normal, they seemed to struggle with following rubrics and instructions. They exceeded my expectations on their understanding of messaging within the text, and how the message of the author created opportunities for their own self reflection, but they struggled with being able to fully articulate how choices made by the author or text creator developed that message.
- Their critical thinking skill development coupled with their speed of understanding exceeded my expectations.
- The stronger students mastered the application problems but the weaker students did not. I clearly needed to have more learning activities for that learning objective.
- Student achieved much of what I thought they would achieve. The written feedback given to peers did not grow as much for some students, but those students did a nice job in class discussions.
- Those students that exceeded were here and put forth an effort. Some students fell short with their attendance. I am surprised that Covid is impacting students.
- Students really struggled with Math 112 due to the lack of their basic geometry skills. These students took geometry over COVID and had never been exposed to any trigonometric functions. This made trigonometry extra difficult for them. Students that had taken physics prior were more likely to be successful as they had already had prior exposure to most concepts.
- When you think about teaching this course the next time, what might you change in the way you assess your outcomes? In the way you might present your material?
- I’m going to attach the objectives to individual assignments and give students a “skills and knowledge checklist” that they can use for self-assessment as we move through the term and its many many many ideas.
- Assessment is going to be more transparent, with course objectives listed on every assignment to show connection. I’m also going to start sharing the (new) AAOT-ElemEd degree objectives alongside the state/national TSPC and In TASC standards on my syllabi for ED courses.
- I would like to change my course in that I would like to have more diversity in the way that materials are presented. For example, provide more video content and multi-media presentations.
- I could do a better job in my written comments to the student in laying out my example of what a better historical argument might be than the one they provided.
- I need to more regularly focus the conversation how an author develops meaning as opposed to just what the meaning/message is in a particular piece of literature. As part of that, the way I assess the outcomes will be on a more regular/smaller scale model, rather than just in the final essay.
- When I teach this course from the beginning next term, I will clean slate what has been done before. My assessments will mostly be formative helping the students develop their skills with summative assessments through writing projects and presentations.
- I discovered there is a real connection between doing well on objective tests over material and performance on the essays. Students who did well on the objective tests seemed to have more evidence when they wrote their essays.
- Students did well on the objective tests but seemed to sometimes underperform on the essays. I felt the majority had the ability, but seemed rushed, unable in that moment, or unwilling to consistently write to the best of their abilities on the essays. Sometimes they’d hit a “home run”, which was an indicator they possessed the ability.
- Have you made changes this term based on prior assessments? If so, what impact have these changes made on student achievement?
- The biggest changes this term from last year was shifting to more collaborative work and having more discussion and peer review of assignments and not just topical discussions. Some liked this, some did not and although it didn’t have big impact on grades I was able to “see” growth in objective areas through the discussion boards throughout the term.
- The changes seem to improve achievement in that there is a combination of historical and contemporary materials and content that are presented to the students.
- Students are more successful when there is more one on one communication as well as more multi media approaches to teaching and presenting material.
- I’ve taught this course now for 14 years at OCCC, and I make revisions based on outcomes every year! The result is that students do seem to struggle less now than in earlier versions of the course so the process seems to be working.
- I slowed down the amount of curriculum this year to try and help facilitate a return to in person education and I actually feel that this was detrimental to their learning process — they never quite came up to speed on pacing of learning. I will be pushing next year a little harder.
- I made changes in how the videos were presented – using the ones in MOM instead of the ones I created. I believe they were not as effective as the ones that are tailored to this course. I plan to create new videos for fall term.
- This is the first time teaching this class face to face since covid started. The changes made allowed for more hands on work in class projects and activities. I think they helped team build and also helped with understanding and identifying literary elements and devices.
- I put much more time, for better or worse, into the literary criticism aspect of the course. I think students enjoyed this. Now that I think about it, maybe I’ll cut it back a little next year.
Written and Oral Communication CILO | ||
Sub-Category | Percent Scoring 3-4 | Average Score |
Context of and Purpose for Communication | 84% | 3.41 |
Content Development | 84% | 3.38 |
Sources and Evidence | 79% | 3.33 |
Control of Syntax and Mechanics | 88% | 3.39 |
Delivery (for Oral Communication) | 83% | 3.29 |
Aggregated Total | 83% | 3.36 |
n=79 | ||
Cultural Awareness CILO | ||
Sub-Category | Percent Scoring 3-4 | Average Score |
Cultural self- awareness | 84% | 3.16 |
Knowledge of cultural worldview frameworks | 78% | 2.97 |
Verbal and nonverbal communication | 79% | 3.21 |
Curiosity and Empathy | 76% | 3.24 |
Openness | 79% | 3.15 |
Aggregated Total | 79% | 3.15 |
n=38 | ||
Personal Responsibility CILO | ||
Sub-Category | Percent Scoring 3-4 | Average Score |
Ethical Self-Awareness | 74% | 2.79 |
Understanding Different Ethical Perspectives/Concepts | 74% | 3.15 |
Ethical Issue Recognition | 79% | 3.00 |
Application of Ethical Perspectives/Concepts | 76% | 2.85 |
Evaluation of Different Ethical Perspectives/Concepts | 74% | 2.88 |
Aggregated Total | 75% | 2.94 |
n=34 | ||
Problem-Solving CILO | ||
Sub-Category | Percent Scoring 3-4 | Average Score |
Define Problem | 80% | 3.18 |
Identify Strategies | 78% | 3.19 |
Propose Solutions/ Hypotheses | 76% | 3.16 |
Evaluate Potential Solutions | 73% | 3.10 |
Implement Solution | 72% | 3.10 |
Evaluate Outcomes | 69% | 3.06 |
Aggregated Total | 76% | 3.13 |
n=141 |