Quality Assurance Systems: a tool for the continuous improvement of training programs, centers and universities
Sistemas de Aseguramiento de la Calidad: herramienta para la mejora continua de los programas formativos, centros y universidad
Gloria Zaballa-Pérez
Universidad de Deusto, Bilbao, España
gloria.zaballa@deusto.es
https://orcid.org/0000-0002-6609-5968
(Received on: 18/11/2021; Accepted on: 22/11/2021; Final version received on: 15/12/2021)
Suggested citation: Zaballa-Pérez, G. (2022). Quality Assurance Systems: a tool for the continuous improvement of training programs, centers and universities. Revista Cátedra, 5(1), 55-67.
Abstract
The University of Deusto has defined and implemented in its faculties a Quality Assurance System that covers all undergraduate, master's and doctoral degrees, as well as the management of the activities carried out in all faculties as a whole. The Quality Assurance System is implemented through the systematization of the continuous improvement methodology, by measuring and analyzing the satisfaction of stakeholders and the results of its processes. To broaden the scope of the system, a Global Quality Management System has been defined to achieve student satisfaction, from their first contact with the university world, to their insertion as graduates in the labor market. This system is based on the philosophy of process management. The Global Quality Management System is deployed in the Centers, Functional Areas and Services involved in the life of the university community. The system includes the Institutional Accreditation of the Faculties and the Quality Certifications according to the UNE-EN ISO 9001:2015 standard, in the following Functional Areas and Services: New Students, Library, Hall of Residence and Employment Service.
Keywords
Quality, tool, systematic, improvement, sustainability.
Resumen
La Universidad de Deusto tiene definido e implantado en sus facultades un Sistema de Aseguramiento de la Calidad que abarca todas las titulaciones de grado, máster y doctorado, así como la gestión de las actividades que se desarrollan en todas las facultades en su conjunto. El Sistema de Aseguramiento de la Calidad se implanta a través de la sistematización de la metodología de mejora continua, mediante la medición y análisis de la satisfacción de los grupos de interés y de los resultados de sus procesos. Para ampliar el alcance del sistema, se ha definido un Sistema de Gestión de Calidad Global para lograr la satisfacción de los estudiantes, desde su toma de contacto con el mundo universitario, hasta su inserción como egresados en el mercado laboral. Dicho sistema está basado en la filosofía de la gestión por procesos. El Sistema de Gestión de Calidad Global se despliega en los Centros, las Áreas Funcionales y los Servicios implicados en la vida de la comunidad universitaria. El sistema incluye la Acreditación Institucional de las Facultades y las Certificaciones de Calidad de acuerdo a la norma UNE-EN ISO 9001:2015, en las siguientes Áreas Funcionales y Servicios: Nuevos Estudiantes, Biblioteca, Colegio Mayor y Servicio de empleo.
Palabras clave
Calidad, herramienta, sistemática, mejora, sostenibilidad.
1. Introducción
Since 2007, the University of Deusto has defined and implemented a Quality Assurance System (QAS) for undergraduate, master's and doctoral degrees, in accordance with the AUDIT program guidelines defined by ANECA. This system is subject to a process of continuous improvement through the measurement and analysis of the satisfaction of stakeholders, the analysis of the results of its processes, audits and their review. The aim is to ensure that all our degrees respond as effectively and efficiently as possible to the current and future expectations and needs of our stakeholders, in the context in which our university is located and adapt quickly to changes in the context.
To broaden the scope of the Quality System, the University of Deusto has defined a Global Quality Management System. This System aims to emphasize and achieve the satisfaction of the essential piece of our university, which is the student. The objective is to improve student satisfaction from their first contact with the university world in secondary schools, until their insertion as graduates in the labor market. The Global Quality Management System follows the guidelines of the AUDIT program of ANECA in the Faculties for undergraduate, master's and doctoral degrees and the ISO 9001:2015 standard for the Functional Areas and Services of the University more directly related to our main interest group which are the students.
The criteria of ANECA's AUDIT program are summarized below:
§ How the Center defines its quality policy and objectives. The Center must consolidate a quality culture supported by a quality policy and objectives known and publicly accessible, as a commitment to quality assurance (ANECA, 2018, p. 4).
§ How the Center guarantees the quality of its training programs. The Center must have mechanisms that allow it to maintain and renew its training offer, developing methodologies for the design, approval, control and periodic review (internal and external), of the programs (ANECA, 2018, p. 5)..
§ How the Center orients its teaching to students. The Center must be equipped with procedures that allow it to verify that the actions it undertakes are fundamentally aimed at favoring student learning and their access to the rules that regulate all phases of the formative process, from their admission, acquisition of competencies and skills, to the certification of the acquired learning outcomes (ANECA, 2018, p. 7).
§ How the University and/or Center guarantees and improves the quality of its academic personnel. The University and/or Center, must have mechanisms that ensure that the access, management, development and training of its academic and teaching support staff, is carried out with the due guarantees so that it fulfills its own functions (ANECA, 2018, p. 9).
§ How the University and/or Center manages and improves its resources and services. The University and/or Center must equip itself with mechanisms that allow it to design, manage and improve student support services, as well as the human and material resources necessary to facilitate an adequate development of their learning (ANECA, 2018, p. 10).
§ How the Center analyzes and takes into account the information of the results obtained by the Internal Quality Assurance System processes. The Center must be equipped with procedures that allow it to measure, analyze and use the results generated by the Internal Quality Assurance System, among others, of learning, the satisfaction of the different stakeholders and job placement/employability, for decision-making that leads to an improvement in the quality of the teaching it provides and the rest of the activities it carries out (ANECA, 2018, p. 11).
§ How the University publishes information on degrees and other activities carried out. The University and/or the Center must have mechanisms that allow it to guarantee the publication and periodic dissemination of reliable, updated and accessible information related to the degrees offered and their results, as well as other activities carried out therein (ANECA, 2018, p. 12).
§ How the Center guarantees the maintenance and updating of the Internal Quality Assurance System. The Center must have a working methodology and the necessary resources to design and implement an Internal Quality Assurance System that effectively helps the achievement and improvement of its results, and enables its external evaluation on a regular basis (ANECA, 2018, p. 14).
2.2 Continuous improvement system. Basic pillar
One of the main pillars of the Quality Assurance System consists of the implementation of the continuous improvement system following the PDCA Cycle (Zaballa, 2000), as shown in Figure 1.
Figure 1. PDCA Cycle
2.1 Fase Plan
The planning phase consists of defining the processes necessary to comply with the guidelines of ANECA's AUDIT program and to achieve the planned results through their implementation. These processes are represented in a process map, whose deployment allows us to ensure the continuous and sustainable improvement of our University, our Centers and our degrees. It also helps us to comply with European and ministerial guidelines regarding the official nature of the degrees.
Below is an image of the map of processes of the Faculties of the University of Deusto (Figure 2.).
Figure 2. Process map
Likewise, in this planning phase it is very important to define the process monitoring indicators and the goals to be achieved for each indicator, in order to achieve the planned results.
2.2 Fase Do
The execution phase consists of implementing the processes planned in the previous phase.
2.3 Fase Check
The verification phase consists of monitoring the processes through the analysis of the planned indicators to check whether the goals defined in the planning phase have been achieved with respect to the process indicators.
2.4 Fase Act
The action phase consists of defining the actions necessary to continuously improve the performance of the processes. Based on the results of the previous phase, a series of improvement actions are identified and must be planned for implementation.
3. Cycle of continuous improvement of the academic program and the faculties
The academic programs of our university, throughout their academic life, from their design to their implementation, accreditation, reaccreditation, walk in a single direction, which forces them to a permanent review and as a result of this to the continuous improvement of all academic programs, as described in the Royal Decree 822/2021.
This system of continuous improvement extends to the faculties and the university, through the processes that are aligned with the AUDIT guidelines. The following figure represents the continuous improvement cycle followed by the academic programs and faculties in our University.
Figure 3. Continuous improvement cycle
3.1. Academic program design
In this process, proposals are prepared for official university academic programs designed in accordance with the European Higher Education Area. The following sections are defined:
§ Description of the title.
§ Justification.
§ Objectives and competencies.
§ Admission of students.
§ Teaching planning.
§ Academic staff.
§ Planned resources.
§ Quality assurance system.
§ Implementation calendar.
3.2 Follow-up, analysis and improvement of the academic program (annual)
According to what is described in the guides "Guía de Apoyo para el proceso de seguimiento de títulos oficiales de Grado y Máster" and "Guía de Apoyo para el proceso de seguimiento de títulos oficiales de Doctorado" (ANECA, 2016, 2020), each academic program is monitored annually, i.e. how the program is developing annually according to certain criteria.
As explained in the publication of Zaballa (2018), at the University of Deusto we have implemented the strategic process called "Analysis and improvement of the Degree", which is carried out by the Quality Committee of each Degree. Said Commission is made up of the following members.:
§ The person in charge of the degree.
§ Representatives of the PDI.
§ PAS representatives.
§ Student representatives.
§ Representatives of graduate students.
§ Employers' representatives.
§ Faculty Quality Manager.
§ Representatives of the Quality Unit of the University of Deusto.
The mission of the Quality Committee of the Degree consists, in a first phase, in carrying out an exhaustive annual analysis of the results of the strategic process called "Stakeholder Satisfaction" (Zaballa and Liñero, 2017). With regard to this process, it should be noted that it is used to gather information on the needs and expectations of our stakeholders.
The University of Deusto understands a stakeholder as any person, group or institution that has an interest in the University, in the teachings or in the results obtained. These include at least students, professors, administrative and service staff, parents, public administrations, employers and society in general.
The "Stakeholder Satisfaction" process establishes the mechanisms to be applied to know, analyze and evaluate the degree of satisfaction of our students, their families, graduates, employers, PDI and PAS, with respect to the fulfillment of their needs and expectations. In this way, the necessary information will be obtained to apply continuous improvement to all activities and processes included in the Quality Assurance System. This process allows the collection and analysis of information related, among other aspects, to:
§ Quality of teaching and faculty.
§ External internships and mobility programs.
§ Labor market insertion of graduates.
§ Satisfaction with the training received.
§ Satisfaction of the different interest groups.
This collection of information is done in several ways: satisfaction measurement surveys, focus groups and meetings with student representatives.
The stakeholder satisfaction process begins with the planning of the measurements to be carried out during the academic year. For this purpose, the objectives of the measurement are established, which will allow defining the characteristics and scope of the measurement. In addition, the population, the invited sample and the mode of measurement are also determined: survey (personal, telephone, mail, mail or fax), complaints or suggestions, contacts with students, etc. It is also necessary to establish the criteria for analyzing the results, the person responsible for carrying out the measurement and the deadline for its completion.
Based on the established planning, the University Quality Unit designs the support material that will be necessary to carry out the measurements. It is important to take into account that such material must be adapted to the defined measurement objective, the segment of students to be measured, the response time to be established and the subsequent processing of the data.
Once the measurement has been carried out and its results have been collected, the Quality Unit analyzes them, according to the criteria established in the initial planning. If the planned sample size is not obtained, it will assess in each case whether it is appropriate to extend the measurement period or whether the results obtained up to that moment will be analyzed. With the results obtained, it prepares a report measuring student satisfaction, which generally includes aspects such as the evaluation of results with quantifiable data, a retrospective comparison of data and the strong points and areas for improvement. The Quality Unit sends this report to those responsible for the degrees and the Center.
Likewise, the Quality Commission of the Degree analyzes the results of the support process called "Attention to suggestions and complaints", of the available public information, of the teaching activity and of the indicators established for all our processes. This committee also analyzes the improvement actions implemented, the academic rates obtained, the weak points detected and the monitoring report of the previous academic year. Once the analysis of all the information related to the Degree has been made, the Quality Commission of the Degree will obtain as a result the revised and updated public information available, the recommendations pointed out by the agencies and the analysis of the academic rates. In addition, the strengths and weaknesses of the degree in question will be extracted.
Based on the strengths and weaknesses found in relation to the Degree, the Quality Committee of the Degree will follow the guidelines described in the strategic process called "Improvement Actions" of our Quality Management System. In this process, the Quality Committee of the Degree must carry out a rigorous analysis to detect the main causes that have led to the origin of the weaknesses that have arisen in the Degree. Once the causes have been detected, the Quality Committee of the Degree must establish the most appropriate improvement actions to remedy the weaknesses and submit our Degrees to a process of continuous improvement.
The steps to be carried out by those responsible for the Degrees consist of making a description of the action or actions proposed to solve the weakness in question and appointing the person or persons responsible for carrying out the proposed action or actions. Those responsible for the degrees will also have to define the deadline or deadlines to be set for the execution of these actions and the parameter by which the effectiveness of the improvement action or actions implemented will be assessed.
Likewise, there is a Degree Monitoring Committee at the University level, whose main objective is to review the results of the analysis and improvement of each of our training programs. This review ensures the rigorousness of the results and compliance with the requirements set internally by the University, as well as with the external requirements demanded by quality agencies, other organizations and interest groups.
The Monitoring Committee acts once the process of analysis and improvement of all the degrees has been completed and before the results are taken to the Center's Quality Committee for analysis. In the event that the Monitoring Committee observes any type of incident, it will communicate it to the person in charge of the Degree so that the Committee can deal with it and act accordingly.
3.3. Degree accreditation (6 years)
The accreditation of degrees is developed according to what is described in the "Guía de Autoevaluación. ACREDITA DOCTORADO Program" and in the "Framework Document: Evaluation for the renewal of the accreditation of official Bachelor's and Master's degrees" (ANECA, 2018, 2019). The purpose of accreditation is to check whether the degree outcomes, among which are the learning outcomes, are adequate and allow guaranteeing the adequate continuity of the delivery of the program until the next renewal of accreditation (Figure 4. Evaluation criteria of the Acredita Program). It is carried out every six years for bachelor's and master's degrees on a mandatory basis.
Figure 4: Acredita Program Evaluation Criteria
For this purpose, the Quality Committee of the Degree evaluates the achievement of the learning outcomes of the degree, according to what was planned in the verified memory and thus ensure continuous improvement. For this purpose, the Quality Committee of the Degree analyzes the following aspects:
§ The implementation of the curriculum and the organization of the program are coherent with the profile of competencies, objectives and planned learning outcomes.
§ The degree has teaching coordination mechanisms that allow both an adequate allocation of student workload and an adequate time planning, ensuring the acquisition of learning outcomes.
§ Students enrolled in the degree have timely access to relevant information on the curriculum and the planned learning outcomes.
§ The Quality Assurance System implemented and periodically reviewed ensures the continuous collection and analysis of information and results relevant to the effective management of the degree, especially learning outcomes and stakeholder satisfaction.
§ The support staff, material resources and services made available for the development of the degree are adequate according to the nature and modality of the degree, the number of students enrolled and the competences to be acquired by them.
§ In the case that the degree contemplates the realization of external internships, these have been planned as planned and are adequate for the acquisition of the learning outcomes of the degree.
§ The training activities, their teaching methodologies and the evaluation systems used are adequate and reasonably adjusted to the objective of acquiring the expected learning outcomes.
§ The learning outcomes achieved satisfy the objectives of the training program and are appropriate to their level in the MECES.
§ The defined graduate profile (and its deployment in competences and learning outcomes in the curriculum) maintains its relevance and is updated according to the requirements of its academic, scientific and professional field.
3.4. Faculty analysis and improvement process
The process of analysis and improvement of the Faculty follows what is described in the "Guide for institutional accreditation" (ANECA, 2018) and aims to incorporate continuous improvement to the center and its degrees, for which a transversal analysis of the center and its degrees is performed. To this end, there is a Center Quality Committee in which all the stakeholders of the Faculty are represented (Figure 5. Members of the Center Quality Committee.).
Figure 5: Members of the Center's Quality Assurance Committe
This Commission carries out the analysis of:
§ Adequacy and effectiveness of the faculty's quality policy and objectives set based on it.
§ Stakeholders (suggestions/complaints and satisfaction surveys to stakeholders).
§ Results of the processes of the Faculty's Quality Assurance System.
§ Status of the improvement actions of the Faculty and its Degrees.
§ Recommendations of the Agencies (feedback from Unibasq to the reports of Analysis and improvement of the previous year).).
Once the aforementioned information has been analyzed, the Quality Committee of the Center will obtain as results the strengths and weaknesses of the Faculty and the results of the process of analysis and improvement of the degrees reviewed and approved. The Quality Commission of the Center is also responsible for adding or eliminating that information from the analysis of the degrees that it considers necessary.
3.5. Improvement actions process
The weaknesses detected in the previous processes constitute the Improvement Plan for each academic program and the faculty. For each area of improvement identified, concrete actions must be defined to achieve the desired objectives. The management of the Improvement Plan is carried out following what is described in the strategic process "Improvement Actions". This involves detailed planning of the specific actions to be carried out, the persons responsible for carrying out each planned activity, the deadlines defined for carrying out the actions and the effectiveness parameter of the improvement action or actions implemented. The actions are carried out within the established deadlines and their implementation is monitored up to the evaluation of the effectiveness of the actions carried out. Thus, we ensure that the improvement actions are implemented, are effective for the established purpose and do not remain good intentions.
4. Benefits of the Quality Assurance System
The Quality Assurance System has helped us to sustainably introduce the culture of quality and continuous improvement in the University and has become an effective tool for the systematic implementation of continuous improvement in the training programs, the Centers and the University.
With respect to the academic programs, it has allowed us to strengthen the system of:
§ Collection of process information about the academic program.
§ Analysis of the results obtained for decision making.
§ Management of the balanced scorecard of the academic program.
§ Definition of actions to improve the academic programs to adapt them to the needs of the labor market.
Likewise, we have been able to systematically improve our academic and service processes through evaluation and review.:
§ The quality of the teaching provided and of the teaching staff.
§ External internships and mobility.
§ Satisfaction with the training received and job placement.
§ Learning results.
§ Attention to suggestions and complaints and the satisfaction of all groups involved and the communication of the results.
§ Finally, it has facilitated the accreditation of academic programs, since the processes are developed throughout the life cycle of an academic program, which provides the collection and analysis of the results of the program for decision making and definition of improvement plans, all in a systematic way. This information is aligned with the needs to accredit the academic programs, so that when it has to be accredited, the Quality Assurance System has been providing us annually with significant information and data for the definition of improvement plans throughout the life of the program. The necessary data for the accreditation process of a program are available to us from the first year of implementation of the program through the processes of the Quality Assurance System. In other words, we do not have to start collecting data on the program when we decide to accredit it..
5. Conclusions
The Quality Assurance System is a useful tool to implement continuous improvement in our degrees, our faculties, services and the university through an effective and systematic management of its processes. It is incorporated into the day-to-day dynamics of the university and is increasingly internalized in the stakeholders.
It must be flexible, adapt to changes and facilitate the constitution of dynamic, flexible, innovative and quality universities. It facilitates the accreditation of degrees, the evaluation and improvement of teaching, internships, mobility, job placement, the degree of satisfaction of our stakeholders, the implementation of competency-based teaching, etc.
The Quality Assurance System allows us to achieve student satisfaction from their first contact with the university world to their insertion as graduates in the labor market, and to sustainably improve the degree of satisfaction of our stakeholders with respect to their needs and expectations. It is important and vital that the Quality Assurance System is adapted to the needs of each university and each context, respecting the autonomy of the university and its institutional guidelines.
Bibliography
Agencia Nacional de Evaluación de la Calidad y Acreditación. (2016). Guía de Apoyo para el proceso de seguimiento de títulos oficiales de Grado y Máster. PROGRAMA MONITOR [Archivo pdf]. http://www.aneca.es/Programas-de-evaluacion/Evaluacion-de-titulos/MONITOR/MONITOR-grado-master/Documentacion-y-herramientas
Agencia Nacional de Evaluación de la Calidad y Acreditación. (2018). Guía de acreditación institucional [Archivo pdf]. http://www.aneca.es/Programas-de-evaluacion/Evaluacion-institucional/Acreditacion-institucional/Documentos-y-herramientas
Agencia Nacional de Evaluación de la Calidad y Acreditación. (2018). Nuevo modelo del Programa AUDIT. ANECA [Archivo pdf]. http://www.aneca.es/Programas-de-evaluacion/Evaluacion-institucional/AUDIT
Agencia Nacional de Evaluación de la Calidad y Acreditación. (2019). Documento Marco: Evaluación para la renovación de la acreditación de títulos oficiales de Grado y Máster Universitario. Programa ACREDITA [Archivo pdf]. http://www.aneca.es/content/download/12737/157930/file/acredita_docmarco_v6_190124.pdf
Agencia Nacional de Evaluación de la Calidad y Acreditación. (2020). Guía de Apoyo para el proceso de seguimiento de títulos oficiales de Doctorado. PROGRAMA MONITOR [Archivo pdf]. http://www.aneca.es/Programas-de-evaluacion/Evaluacion-de-titulos/MONITOR/MONITOR-Doctorado/MONITOR-Doctorado-documentacion-y-herramientas
Agencia Nacional de Evaluación de la Calidad y Acreditación. (2020). Guía de Autoevaluación. Programa ACREDITA DOCTORADO [Archivo pdf]. http://www.aneca.es/content/download/14069/174333/file/acredita_doctorado_Gu%C3%ADa%20de%20Autoevaluaci%C3%B3n_julio2020.pdf
Real Decreto 822/2021, de 28 de septiembre, por el que se establece la organización de las enseñanzas universitarias y del procedimiento de aseguramiento de su calidad. 29 de septiembre de 2021, 119537 a 119578. https://www.boe.es/eli/es/rd/2021/09/28/822
Zaballa, G. (2018). Buenas prácticas en la gestión de calidad de las universidades, hacia la excelencia. Editorial Académica Española.
Zaballa, G. (2000). Modelo de calidad en Educación Goien. Ediciones Mensajero.
Zaballa, G. y Liñero, I. (2017). La satisfacción de nuestros grupos de interés, aspecto fundamental para la mejora de nuestras titulaciones. V Congreso Internacional de Docencia Universitaria.
Authors
GLORIA ZABALLA-PÉREZ holds a PhD in Computer Engineering from the University of Deusto, a Master's Degree in Computer Science and Technology and a Master's Degree in Quality Management. She has received the Educational Excellence Award 2020 Edition and the Honorary Doctor Honoris Causa Recognition (International Organization for Inclusion and Quality Education - OIICE). Her main lines of research are related to the definition and implementation of sustainable quality assurance systems in Higher Education Institutions, the management of academic processes to ensure continuous improvement, the evaluation of learning outcomes through quality assurance systems of universities.
She is currently director of the Quality Unit of the University of Deusto, and full professor at the Faculty of Engineering of the University of Deusto. She is also a collaborating professor at the Universidad Oberta de Cataluña in the Master's Degree in Evaluation and Quality Management in Higher Education. She has extensive experience in evaluation/accreditation of careers, faculties and centers, as well as in the certification of the implementation of quality systems in faculties and universities at national and international level in the different university Quality Agencies and Accreditation bodies. He has numerous publications and participations in congresses and conferences at national and international level.