Search This Blog

Monday, June 10, 2013

School Board Results: Moderation or Tampering

Last week, Debarghya Das shocked us by writing a blog on quora, Hacking into the Indian Education System. He revealed two things. One, at least one of the boards did not care for privacy of its students. Two, the marks awarded to the students were not as one would expect in a public exam. The dust had hardly settled (which just means that we were about to forget the story, not that there were any explanations from the board) when Prashant Bhattacharji wrote yet another blog, Exposing CBSE and ICSE: Statistical Insights into the True Lies on your Marksheets. This blog shows that similar things happen in CBSE as well.

When I read the two blogs, I said to myself, there must be some academic reason for the boards to change marks, to moderate the marks as they call it. The search on CBSE site leads me to the following moderation policy.:

Prior to declaration of results of Senior School Certificate (Class XII) and Secondary School (Class X) the Board adopts the Moderation Policy in the following manner: 
  1. To compensate the candidates for the difficulties experienced in solving the question in a specified time due to misinterpretation/ambiguity of questions anderrors, if any, leading to multiplicity of performance and causing constraints on consumption of time for other questions.
  2. To compensate the vagaries and to bring uniformity in the evaluation process.
  3. To bring parity on account of element of subjectivity involved in the evaluation process.
  4. To level up the mean achievements in the set-wise performance of the candidates attributable to the difference in the difficulty level of different sets of question papers in the multiple sets scheme.
  5. To maintain a near parity of pass percentage of the candidates in the current year vis-à-vis preceding years, subject-wise and overall.
The moderation policy is too vague and does not help us understand how the marks are actually changed. The two blogs, particularly the first one, has been covered extensively by media. A lot of questions have been raised which are very uncomfortable and indeed are leading to the public at large losing whatever little faith that had in the boards. This also happens to be the time when the government is asking the public to have faith in the boards, by including the performance in the boards for admission to engineering programs across the country.

Given the timing of these two blogs and the ongoing admission process, one would have hoped that the government would act swiftly and ask the boards to explain the moderation policy at the earliest. They would have explained their policies. We would have pointed flaws in those policies and would have requested them to change those policies in future, but at the end of the day, we would have been happy that the policies were only increasing the marks without any explanation (like fitting the marks on a curve), and the whole process was not completely random. As time passes by, the faith in boards keep reducing and the belief that the changes in the marks are for non-academic reasons keep increasing. Even if there is a good academic reason to do such moderation, it must be made public. We all know what lack of transparency could do even in IIT JEE in the past.

The two blogs strengthen the already strong case that consideration of board marks for engineering admissions was always a bad idea. And notice that CBSE is considered to be amongst the best managed boards in the country. One wonders if that reputation is well deserved, and if it is, then what would be happening in the other boards. But what is clear is that unless boards come clean on their academic processes, they can't be trusted to the extent of using their marksheets in the admission process for highly competitive colleges and universities.

There are other issues that the data is pointing to. There appears to be an extremely liberal policy on grace marks. Do the boards provide an outcome of a course. Do they say things like, "someone who passes an English test can write one page in English on a topic of current interest without too many grammar and spelling errors." Do their passing criteria meet those outcomes. Or are all boards trying to compete with each other in increasing the pass percentage (and making their political masters happy as a result of such a result).

I wouldn't want to use terms like fraud, tampering, lies, etc., for now. I hope the boards will explain all this soon and we will have a chance to discuss their policies as academicians.


Prashant said...

To give the benefit of doubt to the government, one can't even be sure if someone in the HRD has seen these blogs. Do you know someone in HRD whose attention could be drawn to this issue. Mine was published just 2 days ago. For a couple of days I wasn't really sure whether to go public with these findings (which I had suspected for a long time). Then I figured, these boards will never release these curves anyway, so might as well put it up for the broader academic community to scrutinize. Terrible decision to use these scores without first having inspected their distributions. The marking in "blocks" might randomly inflate everyone's result, but someone's rank (hence percentile) might be affected due to this.

gautam barua said...

It is a very good that the marking systems of the Boards are coming under scrutiny. This will improve transparency, make the marking systems of the Boards, fairer, and will therefore help school education. I would only like to point out that it hardly seems a coincidence that such scrutiny is taking place after Board marks have been included in admissions to JEE based Institutions. After alll grace marks have been around for many years. One of the arguments of including Board marks was precisely to take Board results seriously. Such scrutiny is but a first step in this direction.
Gautam Barua

Prashant said...

"It is a very good that the marking systems of the Boards are coming under scrutiny. "

Really? You think this is going to be fixed anytime soon. If at all there's a fix to it. Also, please note that this scrutiny is being done by us private individuals and not in an organized manner! In fact we might even have to face the music for going public with what we did.

I am extremely sorry to say this Dr. Barua but most of these problems cannot be fixed. Today they might artificially fix the curves to make their results look good, and the leakage and cheating will still remain. Even last year I had blogged about various issues related to board exams and sent it around to IIT faculty members and other than Dr. Sanghi, no one seemed to care or even reply to my mails.

People who gave way to please their bosses in HRD should have insisted on looking at this data first. Wasn't that fairly obvious? Specially when everyone was shouting themselves hoarse that there are serious issues with these examinations, it is a very lame excuse to say that "now having these scores matter for JEE will benefit school education". Now what fair normalization can be applied to these scoring patterns. This scheme is playing a lottery with careers of students who appeared for the entrance tests this year, regardless of how it is normalized.

Ajit R. Jadhav said...

1. First of all, I want to thank the following two parties involved:

(i) The two hackers, who have exposed the two supposedly "high quality" central boards, viz., CBSE and ICSE, and in doing so, have provided a wonderful service towards the cause of (high quality) education.

(ii) Prof. Dheeraj Sanghi himself, for bringing the matter to a wider attention, via this blog. I, for instance, was not at all aware of this expose. Even the national-level English language newspapers, in their Pune editions, did not at all mention (at least, did not adequately highlight) this issue. I became aware of it only today, and only through this blog.

Prof. Sanghi is especially to be lauded, because during the discussions at this very blog last year, he had seemed to be inclined to believe that the central boards should be of higher quality as compared to the state boards. (One only wonders what kind of statistical curves the state boards end up with!)

2. Now, another point. Dheeraj said (in the main post above):

"We all know what lack of transparency could do even in IIT JEE in the past."

Have no idea what you mean by that, Dheeraj. ... Would it be possible for you to provide some explanation or pointers? Thanks in advance.


andy said...

respected sir
i respect ur views. u have written in this article and verified also that these board marks should not be included in the prepration of final ranking of jee mains.
sir because of inclusion of board mraks and faulty formula of mapping ,students who have got very good marks in jee mains and little lesser marks in board ,their life will be ruined.
please do some thing for us.

Siddharth Jain said...

Looks as though Dr. Barua will be more good in Politics. He is giving lollipup of improvements like our poltical leaders.Being in touch with aspirants I know the pain of a repeater who has done his best to get 198 marks and just because in XII he couldn't perform well(75 odd percentage) is seriously killing his bright future prospects. There are numerous such cases I encounter personally or through my blog where students don't know what the future has in store for them. Similarly some ill informed students(which are plenty in number) who don't know much about this normalization are still taking admissions for next year JEE MAIN preparation in institutes without knowing that their 60-70% is leaving them no hopes next year as well. But the greedy institutes won't counsel them right way

gautam barua said...

Siddharth: Please do not mix up two issues and use it to label me a politician.
1. The first issue is about the Boards marks coming under scrutiny. I stand by my remark in an earlier comment in this post. Some of you may be sceptical that this will lead to any change, but I am not.
2. The normalisation procedure used for JEE (Mains) - I am not responsible for it and I have voiced my opinion against it in a comment in an earlier post. I repeat - the normalisation should be as per the "original" proposal : 0.6* (JEE(Main) marks/360) + 0.4* (Board percentile) (or normalise to out of 360 as someone else had suggested).
Gautam Barua

Siddharth Jain said...

Dr. Barua sorry for hurting you but you are a better judge; do you still feel after such insights and the knowledge we all have about our school and board culture its an adventure worth taking on this scale so early. Like Dr Sanghi has expressed before these things must first have been tested over previous datas and should have been implemented in a phased manner.Seeing the good students suffering hurts and that's the prime cause of anger. On a lighter note sir there's no harm in having educationalists like you in Active politics.

Vaibhav said...

Siddharth Jain - Did anyone file a PIL?

Siddharth Jain said...

If courts have to decide all policies from coal to telecom to education then what's need of other institutions let the CJI be the pm and rest judges his ministers. Its really painful to see that people expect a judge to be efficient in all spheres of policy making. He is not superhuman.The CBSE was already making a mockery of RTI acts as they never disclosed answers even of aieee or Aipmt. Its only because of court orders they are doing. Everyone doesn't have the luxury of affording sibbal and jethnalani's let them be busy in defending some raja or mulayam or shreesant.our common student is not that important.

Vaibhav said...

Siddharth please provide your email id.
Its really shameful that the executive body has so much follies despite hving ppl with best minds in the country. but they are no utility as mindless politicians have been ruling them. I want to ask Mr Sibbal if any child in his family was reluctant to study for competition and get away by doing good in boards only, or may be buying marks ?
How can he take such a decision? Such a foolish and insensible decision. i m astouned to see a student promoting the cause against the inclusion of board marks , as he himself got only 80 marks in jee main and 92% in CBSE. This person told me that he wanted to help his brothers and sisters knowing that the proposed formula will actually help him get a good prvte colege. He calls this unjust and unfair! Learn something from these people. The person who started the petition himself is a passout 6 years elder. He has created a platform for students like me to voice against the unjust. Respected Professors - are you not moved by all these things? the practical aspects makes you discuss it on this blog, but its consequence deserves a more stern act against this crap!

Kartik said...

This system is worse then ever.I don't know what does top 20 percentile means ?a student who is not in top 20 percentile can also crack the iit exam.I am seriously suffering from this formula because I got 77.28% in my 12th class and as I belong to the CBSE board so I can't take admission to any of the iit as I am not in top 20 percentile of my board.And the top 20 percentile of my board is 77.2 just because I got 5 marks less in my board exam I have to stop dreaming of iit next year also as am dropping this year.I request professor Dheeraj Sanjhi and all other to do something to change this faulty system so that the future of many students like me can be secured. Thanking you Kartik Joshi