Comparative analysis of results from tools measuring student learning levels

Attaining foundational literacy and numeracy for ALL children is both a state and national priority. Carrying out regular student assessments to measure learning levels of children is essential to tracking progress towards achieving this goal.

In light of the latest Annual Status of Education 2024 Report (ASER) and anticipation of the upcoming National Achievement Survey (NAS) results, this op-ed has 3 main goals.

First, this piece highlights key learning outcomes a child should be able to master by grade 3 in both language and numeracy that would enable them to transition effectively from ‘learning to read’ phase to ‘reading to learn’ phase. Experts recommend this to be necessary towards setting up children for future academic success.

Second, it provides a quick overview of the current efforts to measuring student learning outcomes, particularly for students in classes 1-3 in Tamil Nadu (such as NAS, ASER, State Led Achievement Survey (SLAS), Madhi Led Assessment (MLA)); we further triangulate recent results and articulate salient observations from them.

Finally, the piece identifies limitations to existing assessment approaches and calls for support towards more regular, relevant, representative and rigorous assessments geared to inform current programming efforts around Foundational Literacy & Numeracy in the region.

Foundational Literacy and Numeracy: Building blocks for success

Mastery of foundational literacy and numeracy skills by grade 3 is crucial for a child’s future academic success. This ‘learning to read’ stage sets the foundation for the subsequent ‘reading to learn’ phase, enabling children to effectively engage with more complex academic content.

Recognizing this importance, the Government of Tamil Nadu has launched initiatives like the Ennum Ezhuthum mission. This program aims to address learning gaps in the foundational years, ensuring all government school students attain these critical skills by 2025.

A non-comprehensive summary of essential competencies that experts believe is necessary for a child to master by the time they complete Grade 3 are below:

Foundational Literacy	Foundational Numeracy
Listening Comprehension Oral Vocabulary Sound Identification Recognize letter Decode words Familiar and non- familiar word reading Oral Reading Fluency Demonstrate comprehension	Identify Numbers Perform basic arithmetic (addition/ subtraction) Understand patterns Compare Numbers Solve problems Familiarity with 3 digits Numbers Understand Place values

National Council of Educational Research and Training (NCERT). (2022). National Curriculum Framework for Foundational Stage 2022

“Foundational literacy and numeracy should be treated as non-negotiable milestones, requiring a comprehensive policy focus and regular progress monitoring.” (FLS 2022)

Assessments: the GPS for tracking progress on learning outcomes

Carrying out regular student assessments to measure learning levels of children is essential to tracking progress towards achieving the urgent need for mastery of foundational skills by ALL children. Measurement of learning outcomes builds awareness of the status quo and is critical to guiding policy decisions.

Think of a child’s education as a long journey by car to a distant destination. The car represents the education system, the child the passenger, the first milestone destination being the mastery of foundational skills, regular student assessments are alike to GPS check-ins during the journey. Periodical measurement allows for tracking progress, course correction, estimating arrival and creating awareness.

Given that true learning levels are undisputedly the ‘vital signs’ necessary to closely monitor the success of our programs, these are critically measured through a few different representative student assessments.

Assessments geared towards measuring learning outcomes

Various assessments aim to measure student learning outcomes in frequent intervals including the Annual Status of Education Report (ASER), National Achievement Survey (NAS), State Learning Achievement Surveys (SLAS), and Madhi Foundation’s own assessments for impact measurement (MLA).

These assessments cater to different objectives and populations, offering complementary insights. For example: “ASER’s household-based design allows for a broader understanding of out-of-school children’s competencies, while NAS offers detailed, curriculum-aligned data.” (Johnson & Parrado, 2021).

The table below summarizes key features of these periodic assessments.

Feature	ASER	NAS	SLAS	MLA
Frequency	Bi-Annual (Mostly)	Periodic (Every 3 Years)	Varies by state	Annual
Grades Covered	1-8	3,5,8,10	Varies by state	1-3
Sample Covered	Rural households	Schools (National/ State)	State Schools	Schools within selected 4 districts in TN
Focus Area	Reading, Arithmetic	Curriculum based performance	State- specific curriculum	Foundational Literacy and Numeracy
Year Started	2005	2001	2019	2022

In addition to the above periodic assessment, one-off assessments such as Foundational Literacy and Numeracy Study of 2022 (FLS 2022), State Educational Achievement Survey (SEAS) have been conducted in the past to evaluate the foundational literacy and numeracy skills of students across India.

Given that both periodic and non-periodic assessments are time and resource-intensive efforts, it becomes critically important to allow for some standardization in these assessments & synthesize learnings across these studies. This synthesis is crucial for meeting their intended purpose of informing programmatic shifts aimed at addressing learning gaps.

Making sense of the results in Tamil Nadu: A tale of progress and persisting challenges

The data from various assessments conducted between 2021 and 2024 tells a story of incremental progress but also highlights the monumental task that lies ahead in achieving universal foundational literacy and numeracy.

The good news first: we’re seeing consistent improvements. Both the Annual Status of Education Report (ASER) and Madhi Foundation’s assessments indicate an upward trend in literacy and numeracy levels among grade 3 students in government schools in Tamil Nadu. ASER 2024 reports an 18.3% jump in numeracy proficiency to 27.6% compared to 2022. ASER reports an increase in literacy proficiency from a dismal 4.7% in 2022 to 13.2% in 2024. Similarly, Madhi’s assessments show literacy indicators improving from 27% to 36%. These gains, while modest, are significant considering the setbacks caused by the COVID-19 pandemic.

However, the stark reality is that these improved figures still paint a grim picture. When only 13.2% of third graders in Tamil Nadu can read at grade level, the long road to mastery of foundational skills stretches far into the horizon.

Measure	NAS 2021	FLS 2022	ASER 2022	MLA 2024	ASER 2024
Literacy: % of Grade 3 students in sample being able to read paragraph/ grade 2-level text.	62%1	20%	4.7%	6%	13.2%
Numeracy: % of grade 3 students in sample being able to do 2-digit subtraction/basic arithmetic.	52%	23%	9.3%	34%	27.6%

Adding to this complexity is the variation in results across different assessments. How do we reconcile the National Achievement Survey (NAS) 2021 showing a 62% literacy rate with ASER 2022’s 4.7%?

It’s important to recognize that these assessments use different methodologies and serve different purposes, making direct comparisons challenging. Such discrepancies underscore the challenges in educational assessment and the need for standardized, reliable measurement tools.

Areas of improvement to current approaches

While the periodic assessments are useful dipstick for understanding learning levels of student, extra efforts need to be made in how the results are interpreted given a few limitations recognized by experts. The absence of standardized methods across different assessments further complicates cross-comparison and analysis.

A few salient limitations include:

1. Limited scope of assessment: The most common criticism of some of these approaches has been that the tests focus on floor-level reading and math levels, which may not provide a comprehensive picture of a student’s overall learning abilities.
2. Inconsistencies Across Tools: Variability in sampling and methodology leads to data discrepancies. For example, a study by Johnson and Parrado 2021 found that changes in state averages in one year are often reversed in the following year, indicating potential reliability issues in year-to-year comparisons
3. Resource Constraints: Reliance on volunteer enumerators and limited funding to conduct the assessments hampers quality.

The path forward: strengthening assessments to drive change

To ensure assessments serves as a catalyst for educational outcomes in Tamil Nadu, a more cohesive and collaborative approach is needed. Here are a few recommendations:

1. Collaborative Efforts: Foster partnerships between governments, NGOs, and schools to align goals and interventions.
2. Enhanced Methodology: Standardize frameworks across studies for consistency and reliability.
3. Transparency and Accessibility: Simplify results for policy makers and communities to drive informed action.
4. Leveraging Technology: Use digital tools to improve data collection and analysis.

Call for Support: Enhancing Assessments for Foundational Literacy and Numeracy

As we aim to strengthen educational outcomes and bridge learning gaps for children across Tamil Nadu, it is vital to recognize the transformative potential of robust assessment systems in achieving foundational literacy and numeracy (FLN). While several assessments—such as ASER, NAS, SLAS, and Madhi-Led assessments—have contributed valuable insights, there is room for improvement in effectively addressing students’ needs. Further efforts are needed to enhance these frameworks, ensuring they can better adapt to the evolving challenges in education and provide more tailored support to students.

References:

1. Johnson, D., & Parrado, A. (2021). Assessing the assessments: Taking stock of learning outcomes data in India. International Journal of Educational Development, 84, 102409. https://doi.org/10.1016/j.ijedudev.2021.102409
2. National Council of Educational Research and Training (NCERT). (2022). National Curriculum Framework for Foundational Stage 2022. https://ncert.nic.in/pdf/NCF_for_Foundational_Stage_20_October_2022.pdf
3. Ministry of Education, Government of India. (2020). National Education Policy 2020. https://www.education.gov.in/sites/upload_files/mhrd/files/nep/TS10_1.pdf
4. Foundational Learning Study 2022 https://dsel.education.gov.in/sites/default/files/FLS/National/National_Report_on_Benchmarking_for_ORF_and_Numeracy.pdf
5. National Achievement Survey (NAS) – Key Facts and Objectives.” Testbook, https://testbook.com/articles/national-achievement-survey.
6. RISE Programme. (2023). Focus to Flourish: Five Actions to Accelerate Progress in Learning. https://riseprogramme.org/publications/focus-flourish-five-actions-accelerate-progress-learning.
7. Times of India. (2023). Tamil Nadu’s learning outcomes plunge. https://timesofindia.indiatimes.com/city/chennai/tamil-nadus-learning-outcomes-plunge/articleshow/101586369.cms
8. The Hans India. (2023). ASER survey shows reading skills of Tamil Nadu students affected by pandemic. https://www.thehansindia.com/tamilnadu/aser-survey-shows-reading-skills-of-tamil-nadu-students-affected-by-pandemic-778387
9. Times of India. (2022). Tamil Nadu school students’ learning outcome below national average. https://timesofindia.indiatimes.com/city/chennai/tamil-nadu-school-students-learning-outcome-below-national-average/articleshow/91822925.cms
10. Institute for Competitiveness. (2021). Report on the State of Foundational Learning and Numeracy. https://www.competitiveness.in/wp-content/uploads/2021/12/Report_on_state_of_foundational_learning_and_numeracy_web_version.pdf
11. Institute for Competitiveness. (2024). Panel Brief: The Challenge of Foundational Learning and Numeracy. https://www.competitiveness.in/wp-content/uploads/2024/04/Panel_Brief_Day2_The_Challenge_of_Foundational_Learning_and_Numeracy.pdf.

Annexure:

Tracing the Evolution of Key Educational Assessments in India

Approach	History	Purpose Purpose	Key features & Evolution
Annual Status of Education Report (ASER)	Launched in 2005, ASER has been conducted annually to assess the status of education in rural India. It began as a grassroots initiative by Pratham and has grown to involve various stakeholders.	To Provide a comprehensive picture of children’s learning levels in rural areas, focusing on basic reading and arithmetic skills.	Evolved from basic assessments to include detailed indicators of learning outcomes and school infrastructure. Expanded coverage and refined methodology over time.
National Achievement Survey (NAS	Initiated by the ministry of Education, NAS was first conducted in 2001, but it gained prominence with its structured approach in 2017. Conducted every three years.	To evaluate the effectiveness of school instruction across different grades and provide insights into educational quality at a national level.	Transitioned from periodic to systematic assessments, aligning with international standards Emphasized competency- based evaluations, with a recent focus on foundational literacy and Numeracy.
State Level Achievement Survey (SLAS)	Introduced as part of a broader educational assessment strategy, SLAS aims to evaluate state specific educational outcomes since 2019	To assess students learning outcomes at the state level, providing data to inform policy and improve educational practices.	Developed state- specific methodologies aligned with national standards. Focused on improving local educational frameworks and addressing unique challenges.
Madhi Led Assessments (MLA)	Established in 2022by the Madhi Foundation to address foundational learning gaps in Tamil Nadu, particularly after the disruptions caused by COVID-19	To assess foundational literacy and Numeracy skills in Grade 1-3 to inform targeted interventions like “Ennum Ezhuthum “mission	Utilize one- on- one assessments for precision in Tamil, English, and Mathematics. Developed using a cross- referencing methodology aligned with TNCF, EGRA, and EGMA. Focus on critical skills: reading fluency, listening comprehension, vocabulary, and number operations. Highlight gaps between local and global benchmarks in foundational skills to inform interventions.

Overview of Current Efforts in Measuring Student Learning Outcomes in Tamil Nadu (Classes 1–3)

Tamil Nadu employs several assessment tools to gauge student learning outcomes in early grades. Below is summary of the most recent results from these assessments:

Assessments	Key Findings
National Achievement Survey (NAS) 2021	NCERT conducts the National Achievement Survey (NAS). Last survey in November 2021. Tamil Nadu students performed below national averages. Particularly low in science and mathematics. Learning losses attributed to the pandemic.
Annual Status of Education Report (ASER)	ASER is a household-based survey. Latest report shows challenges in Tamil Nadu. Struggles with basic literacy and numeracy persist. A substantial 42.7% of children aged 14-18 cannot read simple sentences in English, and 26.5% cannot read a Class 2-level textbook in their regional language.
Madhi- Led Assessment (MLA)	Lower-order skills, like listening comprehension, showed more progress. Higher-order skills, such as reading comprehension, showed minimal progress. Tamil reading fluency improved in Grade 3 (2 to 12 words per minute). Still below the global benchmark of 50 words per minute. Only 25% of Grade 3 students could perform grade-appropriate subtraction.