So NEET is back in Supreme Court, this time a candidate asking for normalization between NEET I and NEET II. Here is a news item:
NEET I and II combined results challenged in Supreme Court
First of all, why is it that no one challenged JEE. They too have multiple papers, and by no means the papers are of equal difficulty levels. The JEE Mains has not been challenged, because someone who gave a slightly tougher exam but still still got shortlisted for Advanced JEE, does not care. And someone who didn't get shortlisted for the Advanced JEE, knows that even if s/he was shortlisted, the chances of getting through to IITs was remote. And frankly, people don't quite care for admission to NITs as much as they care for admission to IITs and for admission to MBBS programs.
I recall a meeting in CBSE regarding JEE. Before the meeting started, out of curiosity, I asked the officer handling JEE Mains whether the averages of the paper version and the computer version were same. Without blinking an eyelid, he said, yes they are identical. I kept quiet.
At the end of the meeting, the CBSE Chairperson was wondering why so few candidates give online JEE, even though there is significant benefit in giving online JEE. The paper exam is just after your board exams are over, while the online exam gives you additional time to revise everything. And the conversation went something like this:
Me: Do you recall the question I asked before the meeting about the averages of the two papers.
CBSE Chairperson: Yes.
Me: I can guarantee that the officer was lying. He has never seen the two averages.
CBSE: How can you be so sure.
Me: He has a laptop in front of him. Ask him to check the information now, and let us know the exact numbers.
JEE Officer: Sorry, Sir. I don't know the averages.
CBSE: (Now very curious), But how could you guess?
Me: Averages of lakhs of students can be close but not identical. If he had seen the numbers and they were indeed very close, he would have said that they are within 1% of each other, or something like that.
CBSE: But what does this have to do with students not taking up online exam.
Me: The perception is that the online exam is much tougher. Your processes are very opaque and you don't give out any statistics. So you do nothing to tell people that either the exam is not much tougher, or yes, it is tougher, but then we normalize.
At this point, the officer who handled JEE Mains exam informed us that the way they handle the differences between the two exam is that they ask the group who prepares the question papers to ensure that the two papers are of equal difficulty levels, and we get a certificate signed by them that it is indeed so.
This was, frankly, very shocking. Think about it. There are persons out there who are willing to vouch in writing that the two question papers are of exact same difficulty level without any data, and just on the basis of their experience. I don't know who are these persons with no understanding of testing, but I can only say that these people don't deserve to be anywhere close to that question paper. And for CBSE to involve such persons in preparing the most important question paper for over 10 lakh students shows how CBSE lacks competence about testing. And if, doing this was expedient in the beginning, wouldn't you at least look at data later on. And if the data miraculously pointed to similar distribution of marks, make that information public so that the public perception about your processes become more positive. But once the exam is over, everything about it is best forgotten.
I am extremely happy that someone is challenging the lack of normalization in NEET. I only hope that there is no miracle here and NEET I and NEET II indeed have different distributions of marks. Otherwise, we will be strengthening the argument that it is possible for a man to look at two question papers and guarantee same distribution of marks.
Posts
Mr. M. K. Gandhi and Nathuram Godse
2 weeks ago
8 comments:
This is the issue which has been puzzling me for these years that how can some experts certify three or four papers as of equal difficulty level and why no one has even tried to question this practice of CBSE. The reason for CBSE to be caught wrong footed this time is correctly summarized in your post. To add to this I think this time it's noted so prominently becahse student gave both papers in real exam time and they can make out that the papers are not of same difficulty level. In JEE Main hardly any student try attempting the other papers in same real time conditions. Overall I feel CBSE is a complete failure body in respect of testing at any level. Plus someday I would like to see your post in the CCE crap introduced by CBSE. I have noted the serious ill effects due to student getting too relaxed with this system. There ability to handle vast syllabus of JEE/ AIIMS has seriously got affected due to this.
Having the same difficulty level does not mean identical for sure! What I don't understand is why they need to be! Is someone with a rank of 1001 so much better than 999? If not, a merger of ranks between two tests should cause no serious complaints from anyone as long as they are "roughly" at the same level. Yes, a person with rank 999 in one test may have got a rank of 1001 in the other. This is as unfair as the accident of birth---in other words, like life itself.
The bottom line is that there is nothing absolute about "merit". IITs are interested in getting very good students, a minor permutation of the ranks will not affect that at all. Students are interested in getting to an institute that is within their academic capability, a minor permutation of the ranks will affect that only to the extent that luck plays a role in things --- which is undeniable.
I am so happy to read this comment you and from the pbrevious comment a bout the level of difficulty of questions. I used to set the paper for a science talent exam and judging the"level of difficulty" has always foxed me. I never understood how one does that
@Unknown, I don't think these students and parents are complaining about minor differences in difficulty level. Let CBSE come out with data. If the difference is small, there is no issue. But if the difference is large, then we need to do something. And I suspect that the difference is large, since otherwise, they would have been the first one to share data.
Will somebody throw light on the procedure adopted in such situations;I mean when there is a difference between two modes of the same examination?
Ksrao
Though normalization is not easy to do in these cases. Since, the difference in let's say mean of both the distributions depends not only on the difficulty of the examination but also on the average preparedness of the candidates. Unfortunately, currently the two candidate populations might differ in the average preparedness ( since most of the candidates taking online examination will be from the cities and hence their average performance purely because of access to better resources might be better ).
Ideally, we would want that the different school systems provide students with almost similar preparedness and the preparedness curves of the two student populations is similar.
If both the student populations were almost similar in terms of preparedness, doing straightforward normalization makes sense ( i.e under the assumption that distribution of marks is depended only on difficulty of examination ). However if the preparedness assumption is incorrect, then normalization will negatively affect the online takers.
So maybe, the current system of somehow setting similar examinations has maybe a valid argument.
@Rahul, we do not know data, but the perception is that online tests are less scoring. If the data shows that online tests are less scoring and the expectation is that they should be higher scoring (since the students are better prepared), then there is a problem. And this is exactly what I have been saying. If the evidence is that there is no problem with online tests, then release data to show that. And if there is a problem with online tests, then solve that problem.
Agreed, the data must definitely be released and your conversations with people who actually conduct these examinations is really alarming.
Post a Comment