Redefining evaluation: Programmatic assessment in Indian medical education

Introduction

Programmatic assessment involves the ongoing collection and analysis of routine data about a learner's skills and development. When necessary, additional targeted assessments are conducted to provide a comprehensive understanding. This approach aims to inform both the learner and their mentor continuously and supports making significant decisions at the conclusion of a training period. Programmatic assessment was developed to better fit competency-based curriculum.^[1], ^[2]

For many decades, medical education in India has relied on a traditional curriculum that primarily emphasized knowledge acquisition. This approach often neglected the affective and skill components of education, giving minimal attention to the practical application of learned knowledge. Additionally, the traditional curriculum had several shortcomings in terms of assessment, focusing heavily on summative evaluations. These assessments were predominantly criterion-referenced and did not effectively measure students' overall competencies.^[3]

With programmatic assessment, fewer arbitrary decisions about a student’s level of competence and progress need to be made, and at the same time it provides the students with more meaning-ful feedback to help them direct and manage their own learning.^[1], ^[2], ^[4]

Programmatic assessment encourages the deliberate and longitudinal use of diverse assessment methods to maximize learning and assessment and at present can be utilized optimally as it ensures the collection of multiple low-stake assessment data which can be aggregated for high-stake pass/fail decisions by making use of every opportunity for formative feedback to improve performance.^[5]

Medical education in India is undergoing significant changes to meet global standards and ensure high-quality healthcare. Traditional assessment methods often focus on summative evaluations, which may not provide a comprehensive picture of a student's competencies. Programmatic assessment offers a holistic approach by combining multiple assessment tools and continuous feedback. This review examines the principles of programmatic assessment and its relevance to medical education in India.

Methods

A comprehensive literature search was conducted using databases such as PubMed, Scopus, and Google Scholar. Keywords included "programmatic assessment," "medical education," and "India." Articles published in English between 2010 and 2023 were included. The review also considered guidelines and reports from relevant educational bodies in India. A narrative synthesis approach was used to integrate findings from diverse sources. This involved Summarizing key themes and patterns across the selected studies, Comparing and contrasting findings to identify commonalities and differences, highlighting gaps in the current literature and areas for future research.

Overview of Programmatic Assessment

 Programmatic assessment is being implemented in medical schools globally and is gaining traction in graduate medical education and continuing professional development. Unlike traditional assessment programs that primarily rely on module-based tests focusing on assessment of learning, programmatic assessment emphasizes continuous feedback for the learner. This feedback can be quantitative, qualitative, or both, and is intended to help learners analyze their performance, set specific learning goals, and achieve them. Individual assessments are initially low-stakes and are collected, often in a portfolio, to be reviewed by faculty or committees. These reviews generate a comprehensive diagnostic picture that informs high-stakes decisions. Periodically, an assessment committee reviews all collected information to make summative decisions by integrating data from various sources in a meaningful way.^[6], ^[7], ^[8], ^[9]

Programmatic assessment equally emphasizes both the assessment process and the curriculum, ensuring that no single evaluation determines a student's outcome. This method employs multiple assessments throughout the training phase to holistically evaluate a student's competence. A variety of assessment methods are used, such as Multiple Choice Questions, Essay Questions, Short Notes, Objective Structured Clinical/Practical Examinations, and Workplace-Based Assessments. Each method has its own strengths and weaknesses concerning validity, reliability, educational impact, acceptability, and feasibility in medical education. Therefore, these methods complement one another, helping to achieve complex educational objectives. Additionally, this approach enhances the learning function of assessments, supports student certification, and ensures the process is both valid and reliable.^[10], ^[11], ^[12]

Programmatic assessment has garnered significant attention because it enhances learning through assessment and helps develop a robust system that allows educators to make data-driven decisions about students during high-stakes evaluations. In programmatic assessment, the quality is evaluated based on elements such as coherence, continuity, comprehensiveness, feasibility, purpose-driven approaches, acceptability, and transparency. Overall, programmatic assessment offers numerous advantages for students, teachers, and administrators. However, to fully achieve desired learning outcomes, it is crucial for educators to implement measures that improve the quality of all components within the programmatic assessment framework.^[13]

Programmatic assessment is closely aligned with competency-based education models, focusing on the demonstration of specific skills and competencies rather than merely assigning grades. This approach emphasizes continuous learning progression. Assessments are strategically planned and conducted throughout the course duration, allowing for consistent tracking of student progress. This comprehensive evaluation encompasses students' knowledge, skills, attitudes, and professionalism. Through regular assessments, students are encouraged to take ownership of their learning journey, setting specific goals for improvement based on their performance. By actively participating in the assessment process, students become more motivated and engaged in their studies, leading to active learning. Individual weaknesses identified through assessments enable tailored support and guidance, which are crucial for each student's personal and academic growth. Furthermore, programmatic assessment promotes reflective practices, fostering self-awareness and professional development. Students learn to critically evaluate their own performance, identifying areas for improvement and growth. This reflective approach not only enhances their learning experience but also contributes to their overall professional maturity. In addition to individual benefits, programmatic assessment strengthens collaboration and teamwork among various health professionals. By integrating different assessment methods and involving multiple stakeholders, it creates a platform for interdisciplinary cooperation. This collaborative environment encourages health professionals to work together, share insights, and collectively improve existing practices. Overall, programmatic assessment provides a holistic and dynamic framework that supports the continuous development of medical students. It ensures that assessments are meaningful and contribute to the comprehensive growth of students, preparing them for their future roles in healthcare.^[14], ^[15], ^[16], ^[17], ^[18]

In programmatic assessment, every assessment activity is treated as a single data point. These data points are designed primarily to enhance learning rather than solely to make decisions. This approach ensures that each assessment provides valuable insights into learner performance and meaningful feedback, encouraging learners to utilize this feedback for their development. Various assessment methods can be employed, whether they are traditional or modern, objective or subjective. The selection of methods is guided by the educational purpose at a particular stage of the curriculum and its role within the overall assessment program. For instance, an oral examination might be used to assess and promote verbal communication skills, while a written essay could require students to synthesize literature and demonstrate critical thinking and writing abilities. Combining elements of both types of assessments could be used to evaluate a learner’s capacity to communicate scholarly insights effectively. In this way, different assessment methods can be integrated to provide a comprehensive evaluation, similar to how various laboratory tests and pathology reports are used together to diagnose a patient. Each assessment activity is meant to be informative, and no high-stakes decisions are made based on a single assessment. Consequently, the reliability of any single assessment activity is not a primary concern when choosing methods. Traditionally, exams are classified as either summative or formative, with pass/fail outcomes. In programmatic assessment, this binary approach is replaced by a continuum that ranges from low stakes to high stakes. Each individual assessment activity is considered low stakes, but as the stakes increase, so does the number of data points required. Higher stakes decisions necessitate a greater number of data points, and very high stakes decisions demand an even larger accumulation of data points.^[18]

Programmatic assessment represents a contemporary approach to assessment and learning. While it is acknowledged that assessment influences learning, programmatic assessment is driven by the learning process itself. This method aligns with modern educational principles, optimizing both the learning and decision-making aspects of assessment. More and more universities and postgraduate institutions worldwide are adopting programmatic assessment, contributing to its development through experience and research.^[18]

Conclusion

Programmatic assessment presents a promising paradigm shift in medical education within India, emphasizing continuous improvement and a thorough evaluation of student competencies. Unlike traditional methods that focus on isolated assessments, programmatic assessment integrates multiple evaluation tools throughout a student's educational journey. This approach not only provides a more holistic view of students' knowledge, skills, attitudes, and professionalism but also supports their ongoing development. Furthermore, programmatic assessment promotes a culture of self-directed learning among students. By actively engaging in the assessment process and receiving timely feedback, students become more motivated to identify their learning gaps and set personalized goals for improvement. This approach not only enhances their academic performance but also prepares them for lifelong learning and professional practice in healthcare settings.

Source of Funding

None.

Conflict of Interest

None.