An Actor- Perspective Ontological framework for Student Evaluation Processing Lifecycle in Higher Education

Introduction

The Student Evaluation Processing Lifecycle (SEPL) is an essential aspect of the higher education domain, with the primary objective of measuring learning outcomes, ensuring quality in academics, and maintaining the accountability of higher education institutions. In the university domain, the Student Evaluation Processing Lifecycle includes various processes and sub-processes related to assessment, examinations, result processing, etc., which involve the participation of various actors in the university domain, such as academics, administrative personnel, non-academic support staff, and students, during the process of evaluation.

In the domain of student evaluation in higher education institutions, including in Sri Lanka, the process is often conducted through traditional methods, which have not incorporated the use of any formal representation of the actors’ roles and interactions in the process of student evaluation. The integration of SMART-IT technologies in the university domain is inevitable in the present era of technology, and the lack of a formal conceptual modeling framework of the actors’ roles and interactions in the process of student evaluation is posing a major challenge in the integration of the student evaluation process in the university domain with SMART-IT technologies.

The ontology-based modeling technique offers an effective means to model complex domains by formally describing concepts, relationships, and constraints in an interpretable manner by computers. When actor-oriented thinking is incorporated, an ontology-based modeling technique can effectively model concepts, relationships, and constraints associated with stakeholder responsibilities, authority levels, and process dependencies in organizational processes. In the SEPL, an ontology-based modeling technique can effectively address coordination, policy, and interoperability challenges.

In light of these challenges, this research study aims to develop an Actor Perspective Ontological Framework (APOF) for the Student Evaluation Processing Lifecycle in Higher Education Institutions (HEIs) in Sri Lanka. The main aim of this study is to conceptualize an actor-process ontology to effectively model the SEPL using an actor perspective ontology-based modeling technique. Actors in the SEPL are individuals or organizational entities involved in or influencing various stages of the evaluation lifecycle.

There is no literature on an actor perspective ontology-based modeling technique to model the student evaluation processing lifecycle in an effective manner. This study is motivated by the absence of APOF mechanisms in higher education evaluation systems in Sri Lanka.

Literature Review

In this regard, the traditional frameworks of Student Evaluation Processing Lifecycle (SEPL) in higher education institutions are generally based on the outcomes of evaluations and grading systems, whereas limited importance is given to different actors and their relationships during the evaluation lifecycle. However, it has been suggested in recent research that it is critical to understand and analyze the processes of evaluation from an actor’s perspective in order to enhance transparency and effectiveness in higher education evaluation systems .

Ontology-based methods have increasingly been used in various educational domains for the formal representation of knowledge, processes, and relationships in a structured and machine-understandable way. Ontologies facilitate semantic consistency, interoperability, and reasoning with intricate educational data . In the framework of the Student Evaluation Processing Lifecycle (SEPL), the use of ontological modeling allows the clear representation of the constituents of the evaluation process, such as learning outcomes, evaluation criteria, methods of assessment, roles, and rules .

Actor-oriented modeling approaches emphasize the intentionality, responsibility, and interaction of various actors within the system. The i* framework and goal-oriented requirements engineering emphasize the intentionality and dependency of actors within the system and how they interact and negotiate constraints . Thus, adopting the actor-oriented approach for the Student Evaluation Processing Lifecycle (SEPL) would ensure that the entire process is analyzed in a holistic manner, taking into consideration various aspects such as institutional factors, instructor judgment, and student engagement, among others.

Some research has focused on the importance of the participation of various stakeholders in the design of the assessment and the governance of the evaluation process. For example, the student-centered and assessment for learning approaches have focused on the participation of the students through the provision of feedback, which is perceived as one of the major actors in the evaluation process . In addition, the role of the institutional and regulatory actors in shaping the evaluation process cannot be overstated .

Substantial benefits can be obtained from the integration of actor perspective and ontology. In terms of the Student Evaluation Processing Lifecycle (SEPL) in the context of higher education institutions, the integration of these two aspects allows the representation of actor roles, permissions, and responsibilities in pre-evaluation, evaluation, and post-evaluation phases of SEPL. Furthermore, ontology-based actor models can also be used to facilitate reasoning, policy compliance, and adaptive evaluation in digital learning environments . This is an important step towards the development of transparent and extensible Student Evaluation Processing Lifecycle (SEPL).

However, the importance of actor perspective-based ontological frameworks can be seen more clearly in digital and technology-enhanced learning environments. For example, learning management tools, evaluation tools, and analysis tools play an important role in facilitating the process of evaluation and assessment. In this regard, the importance of ontology-based models can be seen with regard to integrating these tools while ensuring the pedagogical intent and stakeholder responsibility .

However, no studies have been carried out within the specific context of higher education in Sri Lanka to examine the Student Evaluation Processing Lifecycle (SEPL) with an ontology-based modeling approach in conjunction with the Business Motivation Model (BMM). This fact highlights a research gap in formally and systematically describing student evaluation processing. The use of such standards enables the structured identification of actors, their roles, responsibilities, and motivations in the Student Evaluation Processing Lifecycle (SEPL).

Methodology

This research employed a hybrid complementary research paradigm, which combined Design Science Research (DSR), Behavioral Science Research (BSR), and Interpretative Phenomenological Analysis (IPA). DSR was employed to ensure the systematic development of the proposed Actor Perspective Ontological Framework (APOF) and its application in the Student Evaluation Processing Lifecycle (SEPL) domain. On the other hand, Behavioral Science Research was employed to ensure an in-depth analysis of organizational behavior related to the application of the proposed Actor Perspective Ontological Framework (APOF) in the Student Evaluation Processing Lifecycle (SEPL). IPA was employed as a qualitative research approach to gain an in-depth understanding of stakeholders' perceptions and meanings of their lived experiences within organizational contexts, based on phenomenological, hermeneutic, and idiographic research traditions . Data collection for this research was conducted through semi-structured interviews with purposively selected domain experts from the Sabaragamuwa University of Sri Lanka, including the Examination Registrar, Deans, Heads of Departments, and Subject Management Assistants. Documentary research was employed as a systematic approach to ensure the development of the proposed framework. Documentary research was employed to ensure an in-depth analysis of faculty handbooks related to examinations and the Manual of Procedure for the Conduct of University Examinations issued by the University Grants Commission on 1st September 1983. The integration of documentary research and IPA ensured an in-depth understanding of institutional roles, processes, and issues, which informed the development of the proposed ontological framework.

Results

The objectives of this research are to develop an Actor Perspective Ontological Framework (APOF) for Student Evaluation Processing Lifecycle (SEPL) in Higher Education Institutions (HEIs), particularly in the context of Sri Lanka. The proposed framework is aimed at formally defining and structuring the actors of the Student Evaluation Processing Lifecycle (SEPL) and their interrelations within the overall evaluation processes of HEIs.

The proposed framework of an ontological nature is aimed at facilitating the transition from traditional manual evaluation processes to technology-based processes. Consequently, this research seeks to examine the feasibility and potential benefits of SMART IT-based solutions in Sri Lankan university contexts. As electronic systems are essentially organized around actors, activities, and information artifacts, developing an effective and standardized Electronic Processing System (EPS) necessitates the systematic identification of all actors, activities, and collaborations, and information artifacts inherent in traditional manual processes. Consequently, this research focuses on identifying and developing an ontological framework of actors and activities of the Student Evaluation Processing Lifecycle (SEPL). Due to the complexity and time-consuming nature of university-level Student Evaluation Processing Lifecycle (SEPL), which comprises numerous interrelated processes, it is extremely difficult to comprehend the overall process. Consequently, in order to effectively comprehend and develop an IT-based solution, the overall evaluation process is divided into numerous stages.

SEPL Actor Ontological Perspective

The university system comprises a diverse number of staff members, each having specific functions during the Student Evaluation Processing Lifecycle (SEPL). An examination of the university’s Student Evaluation Processing Lifecycle (SEPL) indicates that diverse actors take part in this process. Each actor in this process has specific roles, privileges, and responsibilities during different phases of this process.

The academic staff members contribute to different pedagogical and assessment phases of the SEPL, whereas the university’s administrative staff members are mainly responsible for managing different administrative and financial phases of this process. Furthermore, academic and non-academic support staff members play different specialized roles during this process, which contribute significantly towards completing this process efficiently.

Considering different levels of responsibility and privilege, university staff members participating in the SEPL can be categorized into four main actor groups: academic actor, administrative actor, non-academic actor, and candidate actor. Identifying different actor groups is significant in comprehending different role and privilege distributions during the SEPL and developing the proposed Actor-Perspective Ontological Framework (Figure 1).

The diagram illustrates the conceptual classification of actors involved in the Student Evaluation Processing Lifecycle (SEPL) at the higher education level within the Sri Lankan university system. The Student Evaluation Processing Lifecycle (SEPL) Actors are represented as the primary ontological concept, which represents all the stakeholders involved in the student evaluation process.

At the most abstract level, the actors can be classified into four major categories on the basis of the functional roles, responsibilities, and authority levels within the SEPL. These four categories define the overall composition of the Student Evaluation Processing Lifecycle (SEPL).

Academic Actors encompass lecturers, examiners, moderators, coordinators, and other stakeholders involved in the academic activities of the SEPL. The overall responsibility of the academic actors falls within the domain of academic decision-making and student examination procedures.

Administrative Actors encompass the overall administration of the SEPL, including the actors involved in the management of the entire process. The roles of the administrative actors fall within the domain of examination registrars, administrators, and other staff members involved in the execution of the SEPL process.

Non-Academic Actors encompass the actors involved indirectly in the SEPL process, providing support to the entire process through the facilitation of examination activities. The roles of the non-academic actors fall within the domain of examination support and facilitation of the overall student evaluation process.

Candidate Applicants are students or examinees who are actively engaged in the SEPL as recipients of assessment. Although they do not hold any administrative and academic power, they are considered principal actors whose performance results are assessed within the system.

By defining different yet interconnected categories of actors, this diagram presents a framework for the development of the proposed Actor-Perspective Ontological Framework (APOF). Such categorization helps in better clarifying role boundaries, privileges, and relationships between actors, which can lead to the development of a standardized, transparent, and technology-based Student Evaluation Processing Lifecycle (SEPL).

Organizational Hierarchy Of Student Evaluation Processing Lifecycle (Sepl) Actors

The organizational chart shows (Figure 2) a hierarchy of different roles in the Student Evaluation Processing Lifecycle (SEPL) in the Sri Lankan university system. Such a hierarchy can be considered a basis for understanding the distribution of power and responsibility among different actors in the SEPL. At the top level, power is divided under the Vice-Chancellor into two main administrative sections: the Dean of the Faculty and the Registrar of the University. Such a bifurcation is significant in ensuring proper control and management of the Student Evaluation Processing Lifecycle (SEPL). Each of them addresses different aspects of the SEPL.

In this faculty, the Dean is considered the highest position in the hierarchy and is responsible for overseeing and regulating the activities of the Student Evaluation Processing Lifecycle (SEPL). The staff members, including academic staff, administrative staff, and non-academic staff, report to the Dean. The academic staff are responsible for handling matters related to examinations, while the administrative staff are responsible for handling procedural, regulatory, and logistical operations. On the other hand, the non-academic staff members are responsible for handling essential operations. It is, therefore, evident from this hierarchy that each actor plays a specific role in different phases of the same Student Evaluation Processing Lifecycle (SEPL). Each actor is responsible for handling specific tasks in different phases of the SEPL, depending on their expertise, power, and operational responsibilities. For example, senior academic staff members play multiple roles in different phases of the evaluation process of the SEPL. At the pre-evaluation phase, senior academic staff members play the role of examination paper setters, which involves setting and preparing questions for examinations. At the same phase, senior academic staff members play the role of question paper moderators, which involves reviewing examination papers from an academic perspective.

Similarly, lecturers and senior lecturers assume the responsibility of examination supervisors during the evaluation execution phase of the SEPL. At the same time, non-academic staff assume the responsibility of hall attendants during the examination execution phase of the SEPL. Thus, the collaboration of academic and non-academic staff is vital in ensuring the successful execution of the SEPL.

It is also vital to note that one of the distinguishing features of the SEP is that each responsibility within the SEPL has several responsibility channels such as accountability, consultation, and information flow that are distributed among various actors within the hierarchy of the SEPL. In this regard, the informing channel is often hierarchical in nature, such that the individual involved in the activity of paper setting or marking is expected to inform their immediate supervisor of the activities being undertaken within the SEPL.

In addition, in instances when the Dean of the Faculty is unavailable to assume responsibility within the SEPL, the responsibility may be delegated to the Head of the Department of the University with the aim of ensuring that decision-making and responsibility oversight are not interfered with within the SEPL. Furthermore, some responsibility actors within the SEPL may be required to inform and consult various levels of authority within the SEPL. In this context, non-academic staff within the SEPL may be required to inform and consult various levels of authority such as the Dean of the Faculty and the Registrar of the University, depending on the nature of their responsibility within the SEPL.

To further illustrate this point, the paper printer within the SEPL may be required to consult with the Senior or Assistant Registrar of the Examination Branch of the University before printing the examination papers within the SEPL.

Academic Staff Actors In The SEPL

Under this actor category, academic staff actors are mainly differentiated based on their appointment status in the university, which comprises permanent and temporary actors. Temporary academic actors and academic support actors are mainly engaged in invigilating examination halls, and where necessary, assisting in examination and evaluation processes under the supervision of permanent academic staff. For example, during the execution of summative evaluation, temporary and academic support staff assist in carrying out examination processes under the guidance of subject senior academic staff. This helps in ensuring the smooth execution of the evaluation process while providing temporary and academic support staff with proper guidance and exposure.

Permanent academic staff actors can be further divided into senior and junior academic staff, each having different privileges and responsibilities during different phases of the Student Evaluation Processing Lifecycle (SEPL). Some of the responsibilities during the SEPL, especially those requiring high subject expertise and academic judgment, are usually given to senior academic staff. However, when a senior academic staff member from a specific subject area is not available, a junior academic staff member can perform this role independently or under the guidance of a senior academic actor. For example, moderation of examination papers and second marking are usually performed by senior academic staff, but this role can be given to junior staff members. On the other hand, supervision of examinations is usually performed by senior academic staff, although this role can be performed by junior academic staff when senior staff members are not available.

It is worth noting that academic actors, including Professors, Senior Lecturers, Lecturers, Temporary Academic Staff, and Academic Support Staff, may assume different responsibilities within various actor categories at various stages of the SEPL. These actor categories, including their responsibilities, are presented in Figure 3, which shows the various stages of the Student Evaluation Processing Lifecycle.

In addition, specific responsibilities are assigned to each actor category. For instance, Invigilators are important in ensuring that the Student Evaluation Processing Lifecycle (SEPL) is executed effectively. The responsibilities of Invigilators include monitoring the conduct of examinations, maintaining examination records, managing student-related data, ensuring accuracy in examination documents, among other responsibilities, which are overseen by an examination supervisor.

The academic support staff, on the other hand, offer technical and administrative support to the evaluation process. Their responsibilities include setting up audio-visual equipment or computer systems to support examinations, managing examination-related documents, ensuring smooth communication between students and academic staff, handling other issues that may occur during the examination, among other responsibilities. Through these responsibilities, academic support staff play an important role in ensuring that the Student Evaluation Processing Lifecycle (SEPL) is executed effectively.

Administrative Staff Actors In The SEPL

The administrative staff that are part of the Student Evaluation Processing Lifecycle (SEPL) can be grouped into three main categories: general administration staff, financial administration staff, and examination administration staff. The general administration staff (Figure 4) are responsible for various roles that include registration of candidates, hostel allocation, issuance of student identification cards, and maintaining records of academic and administrative information. Financial administration staff are responsible for handling all financial issues regarding the Student Evaluation Processing Lifecycle (SEPL). This includes examination fee transactions. Examination administration staff are key actors in the examination department and are responsible for coordinating key roles that are part of the evaluation process.

Administrative staff are key actors in the success of the Student Evaluation Processing Lifecycle (SEPL), ensuring that the roles are well defined and implemented while taking into consideration the relevant regulations of the institution. Their roles and responsibilities include planning and scheduling examination dates and examination centers, disseminating relevant information to students and academic staff, responding to queries from various stakeholders involved in the evaluation process, and maintaining records of examination scripts and results in accordance with relevant university guidelines and regulations.

In addition to this, administrative staff members are responsible for managing examination logistics and resources, such as preparing examination venues and distributing examination scripts. They also handle student records in such a way that confidentiality is maintained. The administrative staff members also help academic staff members in managing administrative tasks in relation to examination setting, examination moderation, and processing examination results.

The administrative staff members of an educational institution also handle examination management systems and databases. They also handle financial aspects of Student Evaluation Processing Lifecycle (SEPL). During examination periods, administrative staff members provide critical support to examination venues to resolve any issues that may arise. They also contact security personnel to maintain a secure examination environment. The above-mentioned roles of administrative staff members in managing examination processes highlight their pivotal role in ensuring that Student Evaluation Processing Lifecycle (SEPL) is executed efficiently.

Non-Academic Actors In SEPL

The non-academic staff is an integral part of the student assessment process in the university system. They are mainly responsible for handling all logistical requirements to facilitate the smooth running of student examination processes. The non-academic staff is responsible for preparing examination materials such as printing and distributing examination papers to relevant candidates. They also ensure that all examination materials needed by students and invigilators are in place beforehand (Figure 5).

The non-academic staff is also responsible for preparing examination venues by setting up examination halls to meet procedural requirements. During examination periods, they also assist invigilators in handling examination duties such as supervising students writing their examinations. They also cater to students’ needs by providing relevant examination requirements. They also respond to examination operational issues that may arise during examination periods.

In addition, the non-academic staff is responsible for distributing examination papers to relevant examination marking centers. The logistical requirements of this process form an integral part of their duties. They are responsible for coordinating student movement into and out of examination venues in an orderly manner. They also handle student examination needs such as providing emergency services or facilitating examination accommodations.

The non-academic staff is also responsible for security requirements that need to be put in place to maintain examination integrity. They monitor student movement into and out of examination venues to comply with examination regulations.

The non-academic staff is also responsible for maintaining cleanliness and order in examination venues. They report any examination issues that may arise in terms of maintenance that may affect examination conditions.

The non-academic staff also acts as a link between students and relevant university staff in terms of examination needs. They convey information to students and respond to their concerns during examination periods.

The non-academic staff is also responsible for performing clerical duties such as filing examination-related documents. They maintain student attendance records and process examination materials.

The comprehensive roles of non-academic staff in examination processes make them instrumental in creating an enabling examination environment that ensures student examination efficiency and effectiveness.

Candidate Actors In TheSEPL

Students, or candidates (Figure 6) , form an integral part of the student assessment process within the university system. The students or candidates should take an active part in the examination process, abide by the stipulated examination procedures, and effectively demonstrate their knowledge and understanding of the course material. The major roles of the students or candidates include adequate preparation for the examination through diligent study of the stipulated course material, familiarization with the examination procedures, rules, and requirements, and punctuality to the examination venues with the necessary identification and other relevant materials.

During the examination process, the students or candidates should strictly comply with the stipulated examination procedures as directed by the examination invigilators, exhibit academic integrity through the avoidance of all forms of examination misconduct, and complete the examination process within the stipulated time. The submission of the examination scripts and materials should also be done appropriately, ensuring the examination scripts are clear, well-organized, and clearly identifiable as required. The students or candidates should also, where possible, clarify the examination questions and raise any examination procedure concerns to the relevant authorities.

After the release of the examination results, the students should reflect on the academic performance, identifying the strengths and weaknesses, and seeking positive feedback from the instructors or academic advisors. The students should also uphold academic integrity through the assurance of the originality of the work and the reporting of any examination misconduct, which is fundamental to ensuring the integrity of the Student Evaluation Processing Lifecycle (SEPL).

Conclusion

This study outlines the development of the Actor Perspective Ontological Framework (APOF) for the Student Evaluation Processing Lifecycle (SEPL) in Higher Education Institutions, with particular emphasis on the Sri Lankan Higher Education context. By taking the actor-centric approach, this study systematically identified and categorized key actors involved in the Student Evaluation Processing Lifecycle (SEPL), namely academic actors, administrative actors, non-academic actors, and candidate actors.

In this study, the hybrid methodology of Design Science Research, Behavioral Science Research, and Interpretative Phenomenological Analysis was used to ensure that the proposed framework is theoretically sound and contextually relevant to the Sri Lankan Higher Education context. The proposed framework is also relevant and applicable to the global Higher Education context due to its general nature.

The proposed framework is expected to improve the transparency and accountability of the Student Evaluation Processing Lifecycle (SEPL) by providing a more explicit representation of actors’ roles and authority relationships. In addition, the proposed framework is also expected to provide a foundational structure for the development of SMART IT and electronic systems that can cater to policy requirements and scalability needs.

Although this study is conducted within the Sri Lankan Higher Education context, the conceptual nature of the proposed framework allows for its adaptation to other Higher Education environments.

Future studies can further build on this study by implementing the proposed framework within real-world IT systems and testing its efficacy in real-world settings and further expanding the ontology to incorporate features of automated reasoning and analytics and adaptive evaluation systems.

Research Limitation

In spite of this, this research has several limitations that need to be addressed. Firstly, this research proposes an Actor Perspective Ontological Framework (APOF) that has been developed and validated in the context of the Sri Lankan university system with empirical data collected in one higher educational institution. As such, the generalizability of this research to other national/international contexts of higher education systems may be limited.

Secondly, this research has also adopted a qualitative research methodology that has relied on semi-structured interview research with a purposive selection of domain experts. Although this has provided rich insights into the Student Evaluation Processing Lifecycle (SEPL) in terms of depth of information gathered, this research may not have fully captured all aspects of stakeholder viewpoints.

Thirdly, this research has focused on the conceptual/ontological modeling of actors and their process rather than the implementation of an operational system. As such, this research has not tested the effectiveness of such an operational system in terms of its practical application in digital systems of student evaluation.

Lastly, although documentary research has been conducted with official examination guidelines and manuals, university practices may vary in terms of their operational contexts. As such, adaptations to this research framework may be required to make it consistent with local contexts.

Acknowledgement

This section of the report acknowledges the individuals and institutions whose support and cooperation were essential to the successful completion of this research. The author extends sincere gratitude to the academic and administrative staff of the Sabaragamuwa University of Sri Lanka for their valuable guidance and assistance throughout the study.

Special appreciation is extended to the Examination Registrars, Deans, Heads of Departments, subject management assistants, programmers, and technical staff who generously contributed their time and expertise through interviews and discussion sessions. Their insights and active involvement were instrumental in conducting and completing the Interpretative Phenomenological Analysis.

The author also expresses heartfelt thanks to all research participants who willingly took part in the study. Their meaningful contributions significantly enhanced the rigor, relevance, and practical applicability of the proposed framework.