Blog

Assessing Assessments

With the policy focus on foundational learning and increased emphasis on data-based decision making, the debate has shifted from ‘To assess or not to assess’ to how to utilise assessments to drive learning (Chan et al, 2021). This piece aims to introduce the different approaches to assessments and the purposes they serve using the Government of Tamil Nadu’s flagship Ennum Ezhuthum programme (EE) as an exemplar. 

Assessments are often classified broadly into two types: school-based assessment and large-scale standardised assessments. The difference between the two modes of assessment has been succinctly captured in this table from Wesolowski (2020): 

 

Figure 1.1: Table reproduced from Wesolowski (2020)

 

Prior to the launch of EE, regular practice in government schools was for teachers to put together the question papers and the overall performance of students not being tracked (except within the school). Assessments served an extremely limited purpose as it did not inform instruction or provide insights for teacher training or curriculum design. While comprehensive and continuous evaluation (CCE) was deemed integral through section 29 of The Right of Children to Free and Compulsory Education Act, 2009, the mandate needs to be contextualised and implemented effectively. The EE programme seeks to build on this mandate by focusing on interdisciplinary learning, experiential learning, and 21st century skills (as defined in Joynes et al, 2019). Hence, the themes referred to in CCE are fine-tuned, contextualised and made tangible, visible and achievable at scale through the EE programme. It builds upon the Constructivist Learning Theory which states that learners construct knowledge for themselves instead of taking in information passively.

Upon the launch of the EE programme, three types of assessments have been taking place in government schools since 2022: two types of Formative Assessments and a Summative Assessment. Formative A refers to assessments wherein students are assessed based on the projects or activities that they do; while in Formative B, students are assessed on the content that they have been taught. Formative B happens every week, wherein the idea is to test the students on topics covered in that particular week. Summative assessment refers to an assessment where students are assessed at the end of the term (quarter) on the entire portion taught in that term. An interesting lens to understand the role of these different types of assessments is Stobart’s (2008) assessment for learning (AfL) and assessment of learning (AoL). In their study on assessment in the digital age in the early childhood classroom, Neumann et al (2019) equate AfL with Formative Assessments while AoL is equated with Summative Assessments. As per Neumann, the assessments are meant to serve different purposes and the distinction between AoL and AfL is further elaborated upon below:

 

Through the EE programme, the benefits of both school-based assessments and large-scale testing are sought to be leveraged by conducting large-scale, teacher-led testing in classrooms for students in grades 1, 2 and 3. This assessment includes components of AfL and AoL by virtue of the fact that both summative and formative assessments are carried out. As the digital tool is utilised for testing, learning outcomes from the weekly formative assessments can be used to make modifications to lesson plans and even provide student-specific insights to assist teachers. 

For example,  below is a snapshot of the learning journey chart as envisaged for the EE application to provide an example of the usage of data collected by way of Formative assessments. Through data collected on performance in formative assessments as illustrated in the figure below, the teacher may draw an inference that Student 4 needs to focus on LO4 or that the entire class has performed poorly on LO 5 which may need remediation. 

Figure 1.2: Table contains fictitious data, for illustrative purposes only

 

Summative assessments, on the other hand, can be used to glean larger insights, gauge performance and track progress overall. The programme focuses on level based learning wherein Arumbu equates to learning at the level expected of a Class 1 student, Mottu equates to that of a Class 2 student and so on. The student workbooks are designed in such a manner that students use workbooks suited to their level irrespective of their class. Progress between levels is gauged from one summative assessment to the next. The adjacent graph has been created using sample data for illustrative purposes only which does not reflect actual data. It is assumed to be representative of the performance of students in Class 2 in two different subjects. From the graph above, we can draw inferences pertaining to the effectiveness of the methodologies, teacher learning materials, and books wherein Subject A may be assumed to be performing better than Subject B on the basis of progress made from Term 1 to Term 2. 

 

Figure 1.3: Table contains fictitious percentages, for illustrative purposes only

This action of collecting data at a large scale involving over 37,000 government schools with tests being administered by approximately one lakh teachers is a massive undertaking in itself involving multiple challenges. The intersection of technology and assessment is an exciting field where new inroads are being made with researchers experimenting on new and innovative methods of test administration and assessment design. While assessments cannot be deemed a silver bullet that can solve all the challenges impeding foundational literacy and numeracy, it can nevertheless help in guiding instruction and remedial intervention at the school level and evidence-based policy making at the systems level. 

 

References

    Cecilia K.Y. Chan; Nai Chi Jonathan Yeung, (2021), To assess or not to assess holistic competencies – Student perspectives in Hong Kong, Studies in Educational Evaluation, ISSN: 0191-491X, Vol: 68 c.f. DOI:10.1016/j.stueduc.2021.100984

     Joynes, C., Rossignoli, S., & Fenyiwa Amonoo-Kuofi, E. (2019). 21st Century Skills: Evidence of issues in definition, demand and delivery for development contexts (K4D Helpdesk Report). Brighton, UK: Institute of Development Studies

  Neumann,Michelle M.; Anthony, Jason L.; Erazo, Noé A.; Neumann, David L., (2019) Assessment and Technology: Mapping Future Directions in the Early Childhood Classroom , Front. Educ., 18 October 2019, Sec. Assessment, Testing and Applied Measurement, Volume 4 c.f.  https://doi.org/10.3389/feduc.2019.00116

   Schildkamp, Kim & van der Kleij, Fabienne & Heitink, M.C. & Kippers, Wilma & Veldkamp, Bernard. (2020). Formative assessment: A systematic review of critical teacher prerequisites for classroom practice. International Journal of Educational Research. 103. 101602. 10.1016/j.ijer.2020.101602. 

  Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26, 482–496 c.f. https://doi.org/10.1016/j.tate.2009.06.007

    Stobart, G. (2008). Testing times: The uses and abuses of assessment. London: Routledge

 Wesolowski, Brian. (2020). Validity, Reliability, and Fairness in Classroom Tests, c.f. DOI:10.4324/9780429202308-5 

Posted by Preethi Sundararajan
Blog Uncategorized

Investing in our teachers: Narratives from our experience in Tamil Nadu

The United Nations defines capacity-building as “a process of developing and strengthening the skills, instincts, abilities, processes, and resources that organisations and communities need to survive, adapt, and thrive in a fast-changing world. An essential ingredient in capacity-building is a transformation that is generated and sustained over time from within; transformation of this kind goes beyond performing tasks to changing mindsets and attitudes”. In the context of schools, capacity building plays a critical role in the learning and development of school leaders, teachers, and students to adapt to the changes in the learning ecosystem.

Challenges in the Indian Education System:

In India, there are 1.5 million schools and over 8 million teachers, yet educational quality is dismal due to inadequate infrastructure and a shortage of qualified and well-trained educators. Teacher training is essential for quality education, influencing both pedagogy and technology. 

Ennum Ezhuthum Mission in Tamil Nadu:

With the vision of enhancing the quality of foundational education, the Department of School Education, Government of Tamil Nadu, initiated the Ennum Ezhuthum Mission, focusing on improving foundational learning among primary school children. This project, introduced in the academic year 2022-23, aims to ensure that by 2025, all students aged eight can read and write with comprehension, and acquire critical basic arithmetic skills. One of the key components of this multi-faceted programme is providing in-service teacher training. 

Structured Teacher Training Programs:

The Ennum Ezhuthum Mission impacts over one lakh primary school teachers across grades 1, 2, and 3. To ensure effective teacher training, the School Education Department adopts a cascaded training model conducted at State, District, and Block levels. The Academic Resource Group of Ennum Ezhuthum Mission is responsible for designing teaching-learning materials and conducting training programs at the State level. Following a substantial refinement of curricular materials under the Mission, the focus shifted to orienting teachers on the content. This approach facilitates the dissemination of content to a large, geographically dispersed group of primary teachers within a compressed time frame of 2-3 days.

Data-Driven Evaluation:

The effectiveness of the training programs is rigorously evaluated through a tiered data collection model. Observation forms, participant feedback, and quizzes are used to assess the facilitation quality and content absorption at all three (district, block and cluster) training levels. This data-driven approach allows for effective diagnosis of breakdowns and enables course correction.

Addressing Content Dilution:

A significant challenge in cascaded training programs is the potential dilution of content quality as it reaches the last workforce level, where its impact is most crucial. To maintain uniformity across the three levels and address facilitation capacity gaps, a video covering essential knowledge and skills is played at each training centre. Yet, it’s essential to recognise that the video’s ability to influence teachers’ mindsets depends on the quality of subsequent activities, demonstrations, and discussions. State and district-level participants, serving as resource persons for the next level, significantly impact the learning experience through their facilitation skills. Fostering mindset changes and skill development hinges on their ability to pose impactful questions, moderate discussions, and facilitate reflection. In this transformative journey, the question arises: can we build the capacity to instigate attitude and mindset changes?

Building Mindset Change and Skill-Building:

Shifting teachers’ mindsets to align with new classroom learning approaches is imperative. Hence, honing pedagogical skills tailored to such learning is crucial. For instance, cultivating a growth mindset through effective feedback can boost student motivation and create a positive learning environment. While the teacher handbook outlines activity procedures, teachers excelling in subject-specific competencies enhance learning by understanding the rationale behind these steps. Skill-building sessions, incorporating learning, discussion, application, and reflection, can significantly impact the quality of instructional delivery in the classroom. Orientation to better understand the use of teacher handbooks and student workbooks is vital, especially in a state-wide mission where changes are continuous during the first few  years of mission roll-out. Striking a balance between orientation and attitude building is key.

Continuous Improvement through observations:

Observation data serves as a lever for identifying effective execution strategies, and training administrators play a key role in this process. Developing a quiz that evaluates the absorption of knowledge, depth of understanding, and application skills is essential. In cases where feedback data and observation form data diverge, the quiz results can provide valuable insights into areas for improvement. Administering prework to participants before the training program, based on the session’s subject, reduces the need for extensive guidance, allowing more room for practice, reflection, and planning. Designing spaces for teachers that encourage group or individual learning rather than just instruction ensures effective processing and application of the training inputs. The utilisation of Cluster Resource Centres can play an active role in creating this conducive learning environment.

By adopting a holistic approach to teacher capacity-building, the Ennum Ezhuthum Mission in Tamil Nadu exemplifies a transformative initiative. Emphasising continuous improvement, data-driven evaluations, and a focus on changing mindsets, this program sets a benchmark for effective teacher training in the Indian education system. Through strategic interventions and a commitment to professional development, the Ennum Ezhuthum Mission aims to bring about a positive and lasting change in the landscape of primary education in Tamil Nadu.

Posted by Sreepriya Jaisankar