Reforming Assessment - Time for Change

The recent CBSE paper leak presents an occasion to consider wide-ranging reforms in the process by which we assess student learning in our schools to make it more pedagogically appropriate, less stressful, and relevant to modern society. At a time when public discourse on education commonly references 21^st century skills, a Board examination that only emphasises rote learning and the ability to memorise and reproduce facts is an anachronism that has outlived its utility.

The purpose of assessment should be to illuminate where a learner is on her path of learning at a given point in time. This implies that learning is a continuous, ongoing process that does not end at a particular stage, in keeping with the internationally accepted principles of lifelong learning. Equally importantly, the assessment of learning should lead to an understanding of a student’s ability to construct and apply learnt knowledge as opposed to her ability to memorise factual information that will not be retained after the examination.

A conscious move away from the current situation was made when the Right to Education Bill, 2005, was drafted on the basis of recommendations of a sub-committee of the Central Advisory Board of Education (CABE). While recommending the removal of all Board examinations at grade 8 level and a no-detention policy, the sub-committee also recommended the introduction of Continuous Comprehensive Evaluation (CCE) to ensure that learning was assessed regularly through the school year in order to track student progress. This came from the belief that labelling children examination failures at such a young age was harmful to their self-esteem and their ability to stay engaged with learning. The sub-committee recognised that the majority of children in the schooling system come from less privileged backgrounds, many of them being first generation school-goers, and failing them would have the immediate consequence of pushing them out of school.

As the then Director in the Ministry of HRD who assisted the sub-committee and worked on the draft, I can vouch for the fact that CCE at the elementary stage (grades 1 through 8) was expected to be driven by NCERT and customised by states based on their own specific requirements. It was never envisaged that CBSE, a Board of secondary education, would be responsible for its design and implementation, as eventually happened. The manner in which CCE was implemented carried within itself the seeds of its own downfall, increasing pressure on teachers and establishing a complicated assessment system that was never fully understood by those most affected by it.

Assessment of children’s learning should be an in-built mechanism, with teachers undertaking regular, ongoing monitoring of what has been learnt by the individual child so that corrective action can be taken in time – in other words, assessment for learning. This is of course, easier said than done, given the conditions under which many teachers function, especially in government schools, with multi-grade classrooms, and pupil-teacher ratios in excess of 100:1 in many states. Yet a portfolio containing work done by each child over a period of time remains the best indicator of student progress and can help to scaffold and support individual learning.

This in-classroom assessment should be accompanied by diagnostic, low-stakes, external assessments to indicate the health of the education system. The results of such assessments, such as the National Achievement Survey (NAS) recently carried out by NCERT, provide policy makers with clear information about the functioning of the system and help to identify shortcomings for correction. After the poor showing of Himachal Pradesh and Tamil Nadu in the Programme for International Student Assessment (PISA) 2009, India has stayed away from international learning assessments like PISA, TIMSS, and PIRLS, but there has been some discussion lately around the possibility of participation in PISA 2021. In such a situation, undertaking national diagnostic assessments that provide comparative data over time and across regions acquires renewed importance.

Using Board examination results for selection to higher education institutions has become an annual farce, with students scoring unrealistically high marks, pushing up the cut off levels each year. Eliminating the Board examination altogether and replacing it with an individual entrance test or a standardised assessment like an indigenously developed GRE- or GMAT-like test would remove stress and provide a reliable mechanism for selection. Combined with a portfolio describing a student’s progress over the years, this would provide a far more accurate picture of abilities than the current Board results.

With the establishment of the new National Testing Agency announced in the last Budget, there is an historic opportunity to initiate far-reaching reforms in the way in which we assess student learning, and to drag our education system into the 21^st century. It is unfortunate that the committee set up to review the CBSE leak has its mandate confined to the leaks only, instead of focusing on the big picture. The earlier experiment of CCE may not have been fully successful, but that should not deter us from putting in place more carefully designed assessment systems that help all stakeholders, and most of all, our children.

Redefining Access

The debate about school education in India has focused on many things – enrolment, retention, learning outcomes, teacher preparedness, vocational training, etc, but over the last few decades, we seem to have taken one thing for granted.  This is our definition of the term “access”, i.e., the availability of a school to every child.  Since the 1970s, when the 2^nd All India Education Survey took place, this concept has been defined in terms of physical proximity; in other words, the measure of a school’s availability to a child lies in its distance from her home, with the idea being that a school should be provided within the immediate neighbourhood.  Every government policy or law thus prescribes access in such language – a primary school within 1 kilometre of the residence of a child, an upper primary school within 3 kilometres, and a secondary school within 5 kilometres.

In a soon-to-be-published NUEPA/World Bank study on teachers in the Indian education system, Vimala Ramachandran et al[1] analyse DISE data to show that about one-third of all elementary schools have less than 50 children enrolled, while 11 percent of primary and 14 percent of secondary schools have two teachers or less.  At secondary level, only 3.3 percent schools have all the five subject teachers and head teacher prescribed.  Worse, fully a quarter of all government secondary schools operate out of two classrooms or less.

The exponential increase in the number of schools over the last decade as a result of the impact of programmes like Sarva Shiksha Abhiyan (SSA), Rashtriya Madhyamik Shiksha Abhiyan (RMSA) and the Right of Children to Free and Compulsory Education Act, 2009, has led to an increase in the number of schools with less than optimal infrastructure or numbers of children and/or teachers.  What the foregoing numbers indicate is that the rapid expansion of the school system in the last few years has not been fully efficient on the ground.

Additionally, learning outcomes continue to remain a matter of concern despite the substantial increase in infrastructure, teacher appointments and enrolment.  Each succeeding ASER[2] report has highlighted the continued decline in learning levels and the increasing shift to private schools in rural areas, notwithstanding the opening of many more State schools.  Worryingly, the number of children in Class 5 unable to read a Class 2 text has actually increased from 47 percent in 2006 to 52 percent in 2014, even though the number of elementary schools increased from 1 million to 1.45 million during the same period!

This single minded expansion of school infrastructure has also contributed directly to the non-availability of enough trained and qualified teachers for all the schools and classes being opened under various programmes.  At the elementary school level alone, there are 500,000 teacher vacancies, with another 500,000 being needed to meet the norms stipulated by the Right to Education Act; existing teacher training colleges are simply unable to produce the required number of graduates.  Subject teachers are increasingly hard to find.  When combined with the well-known issues of teacher absenteeism, sub-contracting, etc, this effectively means that enough skilled teachers are simply not available to children in the classroom.

For the teachers themselves, this unprecedented expansion of the school system has not been without grief.  Apart from the personal difficulties of being posted to remote and inaccessible areas, those posted to single or two-teacher schools find themselves deprived of any opportunity for peer learning and thus, professional and personal growth.  And taking leave, attending training programmes away from school, or being deputed for several miscellaneous government duties, nearly always means leaving students without a teacher, further impacting already poor learning outcomes in the classroom.

All things considered, it appears that in practice the concept of access has been reduced to the ability to access a building instead a fully functional school.  This implies the need to re-examine our fundamental definition of access, and define it instead as the ability to attend a school with appropriate infrastructure and adequate numbers of qualified teachers, and to be enabled to achieve predefined learning outcomes, even if such a school is not within the existing norms for distance. Such a definition would compel us to look beyond availability of mere infrastructure, and focus on all the additional conditions that are essential to improve learning outcomes, thus changing the focus of the debate from mere physical proximity to school provisioning, empowerment of staff and administrators, and learning achievements.

Such a discussion is neither new nor unknown; in a policy brief published in 2009, the DFID-supported Consortium for Research on Education, Access, Transitions and Equity (CREATE) had defined “meaningful access” as “…access (that) requires high attendance rates, progression through grades with little or no repetition, and learning outcomes that confirm that basic skills are being mastered”^[3].  Writing in 2012, Rashmi Diwan of NUEPA pointed out how “…poor connectivity, lack of interaction with peer group of teachers and lack of need-based training to teachers coupled with inadequate and poor quality of school facilities”[4] made teaching in such schools far more complex.

If a new parameter of access is accepted, it would be possible to consolidate scattered existing schools into whole and viable institutions that are fully staffed and infrastructurally complete, yielding economies of scale as well as gains in efficiency.  In cases where the school is located beyond distances as currently understood, transport facilities or transport vouchers could be provided to eliminate associated costs.  Clearly this is not an option that would be appropriate for very young children – it may still be relevant to retain the 1 km norm for primary school.  However for children enrolled in classes 6 onwards, when the need for specialised subject teachers kicks in, it could potentially change the nature of schooling, while significantly impacting learning outcomes.

At the same time, while government accounts for just over three-fourths of all elementary schools, at secondary and higher secondary level, schools under private management are 37 percent and 40 percent respectively.  In other words, there are far more secondary schools available outside the government system, accounting for nearly 40 percent of all students enrolled.  It may also be possible to explore the use of these schools to admit children who would otherwise have accessed ill-equipped government schools, through the use of government-issued vouchers that cover the cost of tuition, books, uniforms, etc.  Some pilots involving such vouchers have been tried in India, notably by the Centre for Civil Society (www.ccsindia.org), with some success.

The ongoing review of the National Education Policy by the Ministry of HRD offers an opportunity to reconsider some of these issues. In a country that aspires to take its place on the world stage, education remains one of the most important sectors and needs immediate attention; we cannot continue to plan on the basis of concepts and philosophies that were relevant four decades ago.

---

[1] Vimala Ramachandran, Prerna Goel Chatterjee, Nikhil Mathur, NUEPA; Aparna Ravi, CLPR; Toby Linden, Tara Beteille, Sangeeta Dey, Sangeeta Goel, Chiraag Mehta, The World Bank.

[2] Annual Status of Education Report, brought out by the NGO, Pratham. www.asercentre.org

[3] http://tinyurl.com/oorz25a, accessed 28.04.2015

[4] Diwan, Rashmi (2012), Indian Small Schools -  A Review of Issues and Related Concerns, NUEPA Occasional Paper 40, National University of Educational Planning and Administration, New Delhi.

Acknowledging The Other Half

In the morning newspaper, I came across this article by my old friend and colleague, Rukmini Banerji of Pratham.  Rukmini makes the very strong case that having addressed the issue of providing access to most children, we need to focus on ensuring that children receive quality education; merely enrolling children in school is obviously only half the job done.  More importantly, all children must receive quality education as part of the Right to Education Act, and not just the ones who go to government schools.

Of the 1.3 million schools in India, only about 300,000 are in the private sector; the rest are run by government.  Yet according to some estimates, these 300,000 schools account for fully 40 percent of all enrolments; the schools themselves range from unrecognised budget schools to very high-end premium schools.  Clearly, the non-government sector accounts for a very significant chunk of the total child population.

Rukmini’s argument is a simple one – in a country where such large numbers of children attend schools managed by agencies other than government, can we afford to focus only on the needs of only those who attend government schools?  This is why the Annual Status of Education Report (ASER) brought out by Pratham makes it a point to survey children at home and not in the classroom, so that as many children as possible can be covered, whether enrolled in any school or not, whether attending or not.  On the other hand, the NCERT’s Assessment Survey (NAS) includes only children enrolled in and attending government and aided schools.  Naturally, the results of both surveys are vastly different.

But the point of this piece is not to get into the debate about survey results or methodology.  Rukmini’s article set me thinking about the way education, particularly at school level, is viewed by the political establishment.  I decided therefore, to take a closer look at the manifestos of the major political parties to understand their respective positions.

Not surprisingly, I found a singular lack of vision or understanding across the board.  Almost all the manifestos repeat tired old clichés about generally strengthening the education system, revamping the Sarva Shiksha Abhiyan (SSA) scheme, universalising secondary education or improving skills development, without providing any details.  And only the Grand Old Party talks about a possible role for the private sector, committing itself to “exploring possible partnerships with the private sector in the delivery of education”.  Perhaps because they remember that the 12^th Five Year Plan categorically states that “...private providers (including NGOs and nonprofits) can play an important role in elementary education.  Their legitimate role in expanding elementary education needs to be recognised and a flexible approach needs to be adopted to encourage them to invest in the sector”.

The other major party, which released its manifesto yesterday, promises to increase budgetary allocations to 6 percent of GDP (a long standing demand of activists), but makes no mention of the private sector at all.  And for the rest, the fact that nearly half the children in this country attend a school under private (including NGO) management is clearly not good enough reason to improve the environment in which they function.  Indeed, two of them – the Party-Not-On-The-The-Right, and it’s competitor, the newest entrant into electoral politics – actually talk about increasing supervision and control over private schools, as if things weren’t bad enough already!

In many ways, this is a reflection of the way in which education has been viewed within the establishment all these years.  In a reaction to the opening up of the economy post-1992 and the acceptance of international aid for funding the District Primary Education Programme (and similar projects in the States), the education establishment moved to the far left, arguing that it was the sole responsibility of government not only to regulate and supervise, but also to deliver education services to its citizens.  This position has hardened further in recent years, even as growth of the economy resulted in the creation of a middle class with increased aspirations for their children that caused them to move away from poorly performing government schools to private ones that were more accountable.  Even the judiciary has succumbed to the persuasion of this argument, holding in innumerable judgments that private schools have little or no entitlements at all.  The result has been increasingly stringent regulation and the imposition of unreasonable standards and rules, all leading to increased rent collection at various levels, but little actual improvement on the ground.

The trouble is that the so-called private sector in education is mostly disorganised and disunited, as is normal when agencies are in competition with each other and not colluding.  In such circumstances, they lack a strong enough voice that would enable their concerns to be heard.  But does this mean that parents who choose to send their children to these schools – and indeed, the children themselves, all 120 million of them – also have no rights and can be ignored?

The school education landscape today has been divided into two distinct camps – the ‘us’ and ‘them’ of the government and non-government sectors, which allows government policies or national bodies like NCERT to target only children enrolled in the former.  In turn, this is mirrored in the manner in which policy is decided, with decisions affecting private schools taken with almost no contribution from them.

As we look forward to the formation of a new government at the Centre, this is an opportune moment to consider the manner in which we would like to heal the school sector, and get both the government and private agencies to work together.  The country urgently needs a vision for education that enables rather than restricts, treats the private sector as an equal partner, and leads to the fulfilment of every child’s right to quality education.  Indeed, this would be a good time for the new government to articulate a national education policy, given that the last one was published in 1986 (and amended in 1992).

The manner in which the new government addresses the problems of school education, and in particular, the privately managed parts of the sector, will also be an instance of resolving that classic dilemma of modern democracy – how do you conduct majority rule while protecting the rights of the minority.  In education, as in other sectors, the question before the new government will be one of safeguarding those rights, while “avoiding the tyranny of the majority[1]”.  Sadly, I do not see that vision in any of the manifestos I’ve read so far.

Disclaimer – It is not my intention to express a preference or otherwise for any of the parties contesting the elections; this blog post does not endorse any candidate or party for the ongoing General Elections.

---

[1] John Adams

Assessments - Clearing the Exam

Over the last few years, there have been increased calls for more stringent assessments of learning outcomes, notwithstanding the elimination of conventionally understood examinations thanks to the decisions to do away with Board examinations in Classes 8 and 10; the former because of the RIght of Children to Free and Compulsory Education Act, 2009, the latter thanks to the CBSE imperative to reduce pressure on school children.  Like much else about our education system, the few national assessments carried out so far reveal contrary results - while the Annual Status of Education Report (ASER) carried out by Pratham with the help of volunteers and other NGOs has consistently indicated falling levels of learning, NCERT's own studies seem to contradict this data; the former's survey shows that despite increased government spending on education, more than half of Class 5 children unable to read a simple Class 2 text, while the latter in its most recent report claims that two out of three children in Class 3 were able to correctly answer grade appropriate questions.  And the performance of the two historically most advanced States in PISA 2012 was nothing short of dismal.

On the other hand, a couple of weeks ago, there were reports of Class 12 children in UP who committed, or attempted to commit, suicide because they were unable to pay the bribes demanded by their respective Principals (no less!) to allow them to cheat in the exam.  The cases of teacher-assisted cheating in Atlanta, Chicago, New York and other cities in the US have already been documented; in every case where teacher performance or compensation were linked to student performance, there was a strong incentive for those teachers to "help" their students do well in assessment tests.

In principle, it is hard to argue with the need for outcome measurement; at the end of the day, what gets measured, gets done.  In a recent article, Prof Lant Pritchett and Gulzar Natarajan argue for different types of measurement reports, depending on the end user of the information (see http://indianexpress.com/article/opinion/columns/one-report-card-is-not-enough).  The trick lies in calibrating systems to ensure that while they provide an  accurate picture of our children's achievements, they don't end up either discouraging everyone, or lend themselves to gaming.

Ideally, we need to find a method of low stakes testing that serves as a diagnostic tool, enabling teachers and administrators to take corrective steps well in time; paraphrasing Robert Stake, we need more tasting of the soup during preparation, i.e., more formative assessments.  This means a middle path between the extremes of the high stakes testing of the No Child Left Behind Act in the US and the no-tests-at-all philosophy of the Sudbury Schools.

What should worry us all is the fact that learning levels are consistent across school managements; regardless of whether a school is run by the government or a private entity, learning outcomes seem to be uniformly poor.  The difference, as Prof Kartik Muralidharan of the University of San Diego has shown, is that private schools achieve these poor results at about one-third the government rate because their input costs are much lower!  Clearly it is time to more beyond the public vs private school debate to look at the best way to help our children learn.

A country that claims to be on the verge of benefitting from the demographic dividend needs to make greater effort to ensure that all its young get an equal opportunity to participate in building the nation.