Common Exam for all Engineering Admissions

For last five years, we have been hearing about the government's interest in holding a common exam for admission to all engineering colleges. Recently, a news item said that the success of NEET is encouraging the Government to take a similar decision on a common engineering test for engineering.

First of all, one wonders what this success of NEET means. I am noticing so many legal challenges, so much stupidity in organizing NEET (different papers in different languages), and so many state governments unhappy since they see this as interference in their domain. They also have fears that NEET discourages against students following a state board, and hence indirectly forces a common syllabus across the country. But I guess success at the government level is to be measured more in political terms than in academic terms, and certainly, there is a perception among the people that giving less exams somehow helps them, and therefore, all politicians would want to reduce the number of exams.

There is a legal aspect to it too. The entire medical education in India is supposed to be supervised by Medical Council of India. The status of All India Council for Technical Education is not equivalent. Supreme Court has said that AICTE can not control universities, but only affiliated colleges, and hence AICTE has no say on IITs, NITs, IIITs, State universities, private universities, and so on. So what will be the legal process through which a common engineering exam will be introduced is not clear. (Of course, a dummy case in SC which demands common exam and which is not opposed by Govt/AICTE, etc. could be a simple way. Alternatively, a law can be passed by parliament. Or simply there is this method of arm twisting which is extremely effective in India - remember how everyone was told to get rid of 4-year undergraduate programs.)

But I am not a legal expert. I will only talk about academics.

Does a common exam help students? The perception, of course, is that it does. The naive argument is that every exam causes stress, and more the number of exams, higher the stress. People supporting this argument includes IIT directors and professors (but only if there are two exams, one for IITs and one for everyone else). The same people who inside IITs would sing praises for continuous evaluation and vote against just one big end-semester exam. There, the argument will be, that one exam puts all eggs in one basket and hence causes stress. We must have multiple evaluations with small stakes.

What is also very strange is that no body ever is happy with his/her performance in an exam. I get hundreds of emails from JEE (Adv) passed students every year, and all those mails will start from "I made a silly mistake or two, otherwise, I was expecting better performance." If students are unhappy with their performance in the big exam, shouldn't they be supporting multiple exams. Also, look at the number of students who drop a year because in that one big exam, they think they could have done better. What a huge waste of resource for the country. If they had multiple options and had received some alternate good education, many of them would not have dropped a year.

Stress happens not because of the exam, but the stakes associated with the exam. If I were to tell students that I am going to give an exam in an hour, but it has only 1% weight on the final grade, do they get stressed. Of course, not. Reduce the stakes, and you will reduce the stress. So multiple exams, each with lesser stake, is so much better for stress management than a single exam with huge stakes.

People who accept the logic so far will then argue, but having 100 different exams is still stressful even if each exam has less stress. And my answer is: How many students take 100 or even 10 different entrance exams. A few who may be taking 10 different exams, do they need to. Why are they taking 10 exams. Shouldn't they instead do some research about the kind of places that one wants to join and has the academic preparedness to get admitted to. 10 exams would open thousands of doors already. Frankly, we cannot reduce the number of exams because there is this person somewhere who refuses to do introspection, who refuses to do some research into various colleges and programs, and then insists on trying his/her luck with everything under the sun.

There is another problem with one exam. It is sorting you based on your preparedness in one dimension. People are not identical. If someone is really good at Physics and Maths, s/he will do better in an exam where Chemistry has less weight or is easier. Is it fair to banish this guy to a low rank for all educational institutions in the country. IITs may consider Chemistry to be important, but should every university be forced to consider Chemistry as important. What if you are good at languages. There is huge amount of research which shows that being good at a language increases your chances of success. Now, government institutions may be afraid to get into language politics of the country and not have language component in the entrance test, but should private universities also be forced to admit students without considering their language abilities. (India is the only nation in the world where admission to higher education is done without considering language abilities.)

If there is a single test, it really amounts to asking fish to climb a tree, and then saying that we were fair since we asked everyone to climb a tree.

From the university point of view, if I am offering only courses in Maths and Computing, I may argue that I want to admit students who knows maths and programming, and I am willing to teach them whatever little Physics they need to know, and I don't care about Chemistry. Should that not be the prerogative of the university to admit students based on different capabilities. I may even say that as a university, I want to admit students who are aware of their surroundings, and will have a general knowledge test for that. Should that be illegal. As a university, I may want to acknowledge different achievements of the student, including sports and give them some bonus marks. Should that not be allowed.

So a single test is taking away the most important component of the autonomy that a university must enjoy, and is being unfair to most students since most students' capabilities and interest would not align with any one particular test.

There is more to it. What should be the syllabus of this exam. If the syllabus is the intersection of syllabi of all boards, then it is going to be pretty less and may not be acceptable to most universities in the country. But if we add some topics other than the common part, then we are being partial. This is exactly the point of several states in case of NEET. By choosing one syllabus for NEET, you are discriminating against state boards whose syllabus is significantly different from that chosen syllabus. Same thing will now happen for engineering education, if a common exam is forced on this country. Having state level exams were allowing students to get admission to top colleges of their respective states without being forced to go through coaching of topics that they have not learned in schools. Now that option will be taken away.

I know many readers of my blog would argue that uniformity is a good thing, and let this common test force all state boards to change their syllabus to the CBSE syllabus. Many people just love uniformity, but they either want mediocrity or they do not know that uniformity can only lead to mediocrity or worse.

The suicides in Kota happen due to high stakes and thus high pressure from parents to succeed in JEE. If there is only one exam, brace yourself for a much higher pressure and many more suicides.

So a common test (like NEET) increases stress by increasing stakes, forces a linear ranking among students who have widely different skills and training and thus is unfair to most students, is unfair to students who have studied in state boards, takes away the most important component of autonomy of universities, bars any innovation in admission process by universities, and really gives us nothing in return of all these costs.

JEE Advanced 2017: Case simplified

Yesterday, I wrote a blog on the mess that IITs have created regarding JEE Advanced and the petition that has been filed in Supreme Court. From the mails that I have been getting and the comments on the blog, it is apparent that people have not understood the case properly. Here is an attempt to simplify matters.

IITs have declared several questions from JEE Advanced 2017 as wrong, and decided to award marks to everyone who took the exam. (It is equivalent to canceling those questions and not awarding marks to anyone, for all practical purposes.) The petition (and I am told that many other students have joined this petition with their own petitions) claims that same treatment to all questions is not fair.

So let us look at different types of errors that seem to have happened.

First, the question had such a serious error that there is no right answer (or at least all the options given in the question are wrong).

There is no dispute about what to do with these questions. Presence of such questions is unfortunate as it wastes precious time of students. And those who don't attempt these at all get benefit in terms of extra time. And there is no way to distinguish between those who attempted and those who did not attempt this. The petition is also not talking of such questions. And whether IITs give 0 marks to all students, or 1 mark or 100 marks to all students, it makes no difference in ranking. But it is desirable that IITs look at their process of paper setting and see if there is a way to minimize the chances of such questions since it does bring in an element of unfairness without any solution. But there is no legal issue as of now, and they are not part of the court case.

Second, a couple of questions had some typing error in some versions of Hindi paper. So 99.5% students have received one question. About 0.5% students have received another question.

IITs argued that since the question is different in two versions, we are canceling it and give marks to all. The petition is saying that a fairer way of doing things is to check 99.5% papers according to their question, and 0.5% papers according to the other question. And it hugely helps that despite the misprint, the question remains a valid science question. I really don't understand why IITs did not do it this way to begin with. Indeed this is the right way to do things. IITs know exactly who had asked for English paper and who had asked for Hindi paper. IITs know exactly who received those Hindi papers where there was a misprint. So where is the issue? This is simple and absolutely fair.

Well, it seems that some of those students who were supposed to read Hindi questions asked for English version during the exam, and if there was an extra English paper with the same code, they were given those English papers. So theoretically it is possible that in IITs' database the student is shown as someone who received that Hindi paper which had a misprint, but actually had taken an English paper. And for some strange reason fairness to such students is much more important for IITs than fairness to all the remaining students.

Third, a few questions which were supposed to have a single answer, it turned out that due to some ambiguity in the language can be interpreted to have an alternate answer.

IITs have declared such questions as invalid and given marks to everyone. The petition in SC is saying that if there can be two or even three potential answers to a question, then those marking those answers should get credit and those who have not given any answer or given the wrong answer should not get credit. Again, sounds pretty reasonable to me.

Most students while trying to solve that question would interpret it in one way or the other, and will solve accordingly. That there are multiple possible interpretations would not bother a student busy giving an exam. And hence it makes sense to award marks to both answers. On the contrary, there is a possibility that someone would be able to figure out that there are two interpretations and decide not to pursue this question because of negative marking. Also, there would be students who have marked an answer through guess work. Their probability of getting full marks would go up if this is accepted.

The question to be asked here is how many students may have left the question after realizing the ambiguity because of the negative marks? If that number is expected to be small, then one can be unfair to them rather than be unfair to thousands others. Also, note that such students actually had more time to solve other questions. So the level of unfairness to them is rather low. And yes, someone doing guess work will have a higher chance of success but giving marks to everyone is much more unfair to those who have solved the question. And this is exactly where the 2005 SC judgment may become relevant which says that only those students who have attempted a wrong question be given bonus marks.

Fourth, there are questions where IITs have declared a unique answer but many coaching institutions have come up with an alternate answers and IITs have decided not to consider them as correct. (These are questions beyond the 18 marks of bonus that IITs have given.)

IITs had sought response to their answer keys, and many coaching institutions and individuals had written to them about the alternate answers. IIT experts have rejected these answers without any explanation. (I wonder if IITs even considered those responses. I had also responded in an earlier year on a question which is in my area of research. Never got an answer either personally or on the GATE website, and my objection was not accepted.) The petition is asking that alternate answers be considered.

I think in all fairness, IITs must either explain why those alternate answers are incorrect or award marks for them. After all, those answers have been written by experts too, and there are only a few questions of this nature. Note that these are questions which are beyond the questions on which everyone has been given the bonus marks. So the total amount of confusion is actually beyond 18 marks.

What has been IITs response so far in the court?

I was not present in the court. So I am only reading the news papers who are notorious for not reporting fully. So take all this with a huge pinch of salt.

None of the news papers have mentioned anything about these questions. I do hope that IITs have an answer to these questions. All the newspapers have mentioned that IITs hired the highest law officer of the land, the Attorney General himself for defending it, and he told to the highest court of the land that since 30,000 students have already taken admission, and since it will take a long time to regrade 2.5 lakh copies, and because IITs have done everything on the basis of expert advice, the petition is not maintainable.

First of all, if the Attorney General claimed that it will take a long time to regrade 2.5 lakh copies, he was misinformed. 2.5 lakh students did not give the exam, and regrading is a matter of changing the key and running a software, which should take a few minutes to give the result.

But let me decode this legalese for you:

If Supreme Court decides to intervene in the matter, IITs have enough arguments to delay admissions by several days if not several weeks. That would cause lots of problems to lots of people. Since IITs are better at media communication than Supreme Court is, one can be sure that everyone in the country will blame SC and not IITs. So the best thing for the court to do is to look the other way, and let IITs continue with its erroneous decisions.

And the courts can easily suggest that since the matter is very technical and the courts are not equipped to deal with such technical matters, and because IITs are our crown jewels who can be trusted with anything technical, we would leave IITs to take a decision.

The mess called JEE Advanced

Supreme Court, yesterday, stayed the process of admission to IITs, NITs, and several other institutes. A student has filed a petition a couple of weeks ago, in which she has claimed that awarding marks to all students for wrong questions was fundamentally flawed and has resulted in significant change of merit, and that there were alternative ways in which the problem of wrong questions could have been sorted out. The writ petition can be read on the same page as this news report.

I read the petition. I don't know if everything being claimed there is correct or not. It is for IITs to decide on how to respond to those points. However, if even ONE of those claims is correct, I think it is a matter of shame for the IIT system. The news paper reports suggest that IITs responded yesterday to the petition by arguing that the process of admission has already started, that about 30,000 students have already accepted admission, and that it will take a long time to recheck everything, etc. I hope this is incomplete response and that they have said more things. I hope IITs are able to argue that the suggestions made by these students are somehow wrong or unimplementable. Because if they have not argued on those lines and cannot argue on those lines, then we can have no trust in JEE any more. And I believe that a fair JEE was always one of the most important reason behind brand IIT. If JEE is no more fair, then it won't take long for brand IIT to go down as well.

But I have my doubts. People who have followed this blog would know that my confidence in fairness of JEE and in the competence of IIT system to conduct a large public exam has always been very low. Every year, we see wrong questions. Every year, we see that IITs don't respond to alternate answers by external experts (or even internal experts).

Let us look at the main issue. Apparently there are questions which got misprinted in Hindi medium. The claim in the petition is that because the question was misprinted, IITs decided to award marks to every single student. Now, what is the percent of students who sought Hindi question paper. I believe that the number is 3-4%. Only a couple of variants out of 10 variants of the question paper had a misprint. Which means that about 0.5% of the students received a different question. Now the petitioner is claiming that it was possible to identify these 0.5% students who may have given a different answer, and for these 0.5% students, one could do different grading. There is no reason to give full marks to all students. Makes eminent sense. If you incorrectly identify (why would you) 5-10 students, say, and 1-2 of them could have been in the top 10,000 ranks, then you are unfair to 1-2 students out of 1.6 lakhs. But by giving marks to all 1.6 lakhs, you have been unfair to may be 10,000 students who could solve that question. Now, if you are faced with a difficult situation, and the only two options are that you be unfair to 10 students or you be unfair to 10,000 students, what would you choose. I am sure the answer is obvious. You would rather be unfair to 10 students than 10,000 students. But IITs don't think like that. (And I have been inside the room with JEE Implementation Committee in the past, so I know the culture there.) The IITs would argue that let us give bonus marks to everyone. It does not matter how many students you are unfair to, since no one would do such back of the envelope calculations. If you give marks to everyone, there is a greater chance of happiness all over, and less chance of someone going to court. And the only thing that matters is a court case, and not fairness in the system.

At another place, the petitioner has pointed out that while the IITs have given out one single answer to a question, a large number of coaching places have given alternate solutions which result in different answers out of the four choices. Now, let us remember that some of these coaching classes have some of the finest Physics/Chemistry/Maths faculty, some times better than the paper setters. If these people are saying that there is an alternate solution, and showing that alternate solution on their websites, IITs must respond to them. They must point out why those solutions are not acceptable. I know what IITs will argue. How can they respond to each and every answer given by any tom, dick and harry. But sorry, they are not tom, dick or harry. There are typically, only a handful of answers which are different from IIT answers, and there is no reason why IITs can not respond to those handful of answers, and that too, as I said above, by some of the finest brains of the country. Again, I have had experience of being in the room. Typically, it happens because the question is ill framed. The language is ambiguous. And no paper setter would ever admit that the language is ambiguous. And if they were to respond they will have to admit in writing that a particular statement only means something and that in their opinion alternate meanings are wrong. Now, this interpretation can be easily challenged in a court of law. So, as I said above, the fairness does not matter. Only court cases matter. So we won't agree to those alternative solutions, since our ego is hurt, and we won't explicitly respond saying that, since fairness does not matter. Only court cases do. Now, in this particular case, either IITs must respond as to why multiple answers are incorrect or agree to regrade as per multiple answers.

I also want to know if there is any accountability of paper setters. Does their substantial remuneration depend on their performance. Are they told that they will get x amount of money, if there is no dispute about their answers, but will progressively get lesser and lesser money, if it turns out that they made a mess. Is there a process of black listing professors who set erroneous questions. Why does it take weeks to publish the answer key, when the coaching classes normally publish them within an hour. Wasn't a key prepared several months in advance. Why is that key not announced within an hour of the exam.

IITs have argued (as per media reports) that it is too late, that about 30,000 students have taken admission, that regrading will take too much time. First of all, regrading will take a few minutes. Yes, a few minutes. If IITs are claiming that it will take a long time, then they are lying, and someone should be persecuted for lying to the court in an affidavit. All the answers have been scanned and stored. Now, only a key needs to be decided and that needs to be applied to the stored answers. We don't need to scan the answer sheets again. And applying the new key will take a few minutes.

If the scale of the problem is really as serious as this petitioner is claiming than it does not matter if 30,000 students have taken admission. We must restart the process, and if that means that the semester will be delayed by a week or two, so be it. Let us not forget that in 1997 when the JEE question paper was leaked, the semester started 5 weeks late and the heavens did not fall.

My only concern is that the petition seeks as the last resort (page 20, prayer 'e' of the petition) a direction to the IITs that all those who took JEE Advanced this year be allowed to take JEE Advanced next year. I suspect that IITs will latch on to this demand and agree to it, if they are allowed to continue with the admission with the current merit list. This particular petitioner may be ok with such a decision, but it would be grossly unfair to thousands of hard working students (again assuming that what is written in petition is correct).

It is becoming increasingly clear that conducting large public exams is not a core competence of IIT system. I am sure some of you would disagree and point out that in an earlier era JEE was conducted wonderfully well. Well, what was done in the pre-RTI era can not be termed as wonderful. It is just that we don't know what happened then. A few years ago, when the previous government wanted "One Nation, One Test" and we successfully resisted that, one of the commitments that IITs had made was that besides the JEE Implementation Committee (consisting of all JEE chairpersons and vice chairs), there will also be a new standing committee which will look at the longer term horizon and advice on how JEE could be made better. That committee does not exist today and all the issues of JEE are left to be resolved by the current chairmen and vice chairs who are under pressure to somehow manage this year's JEE with as few court cases as possible. I think it is high time IIT system has a standing committee to look into the longer term.

Added on 8th July:

A few questions have been asked offline, and I thought I should address those questions.

Q1: IITs appear to have claimed (see the ToI news where this is mentioned) that it is not possible to identify the students who solved the specific Hindi paper with that misprint. Is it possible.

Ans: JEE knows exactly who applied for Hindi question paper and who applied for English question paper. JEE also knows which code paper was given to whom. Now, the only issue is that a student could possibly have both Hindi and English version of the question paper and looked at the English paper and answered it accordingly, even though in JEE data base s/he is shown as a student who received the Hindi question paper. There may be 5-10 such students in the country, but theoretically, they could be 100 or 200 students. So technically, IITs are right in saying that they do not know who have looked at only the Hindi paper before answering.

However, this is just an argument to not do anything to solve the problem. Remember the principle at this stage should be that out of all bad options, we should select the least bad option. So what if JEE were to say that we will grade you according to the language you were supposed to be looking at. This would perhaps be unfair to 5-10 students. The current solution is unfair to 10,000 students. There are other options. What if we consider both 5 and 6 as the right answer for those few students and consider only 6 as the right answer for everyone else. Or even, consider 5 or 6 as the right answer for all students. So let us understand that the only options with IITs are not binary (as Attorney General, the highest law officer of the country is claiming in the highest court of the land). They have multiple options with varying impact on fairness.

Q2: Isn't there a Supreme Court judgment in 2005 that bonus marks in case of wrong questions can only be given to those candidates who attempt the question. Why did IITs award marks to all.

Ans: I have not read that judgment. But I doubt if that would be relevant in the current case. Earlier, the exams required long answers to be written. So if someone has written something in the answerbook which has anything to do with the question, it could be taken as a proof that the answer has been attempted. But now with fully objective type question, students try to solve the question in rough sheets and then fill in a bubble. If they couldn't reach any answer, or if they have a doubt about their answer, they will not fill in the bubble. Since those rough sheets are not kept, and sometimes rough work is done on the question paper itself which is allowed to be taken away by the student, there is no identification possible of students who have "attempted" the answer. Filling in the bubble is certainly a proof of attempt, but not filling in a bubble is not a proof of lack of attempt. But note that this petition is really not dependent on 2005 ruling. This petition is arguing that a better solution key would be more fair than giving bonus marks to all.

Q3: What happens to a student who has rejected BITS/IIIT-Delhi/IIIT-Hyderabad, etc., and is now told that his JEE Advanced rank is much poorer. Wouldn't revising the grading scheme be detrimental to his/her interests.

Ans: Yes, it will be unfair to such a student. However, remember the philosophy that I am suggesting. If you have to be unfair to some, prefer a system with least unfairness. And would you rather be unfair to someone who did not deserve an IIT seat or would you rather be unfair to someone who deserved an IIT seat. Also, it is not as bad. If there is a new ranking and new counseling, the student may not get IIT but will get NIT/IIIT etc. So unfairness to an undeserving student is going to be rather small.

Also, even this unfairness can be handled. The court can order a 10% increase in all IITs in all programs as a one time measure to reduce this kind of unfairness. A single batch being 10% larger will not be a disaster for any IIT. And it will give relief to large number of students whose ranks change. Note that if someone's rank changes drastically downward, there may still be cases where they will not be fully compensated, but heck, these guys didn't deserve to be in IITs by a wide margin.

Added on 9th July:

As a comment on the next blog, Mr. Kandasamy Subramani has sent the link to a petition on change.org. There the issue of bonus marks for supposed misprinting has explained very nicely. Please read that.