Are Competetive Exams (like NEET, JEE) Fair?

There is a big debate about the fairness of NEET given all the allegations of paper leak, favoritism by some center superintendents, grace marks for loss of time, two correct answers for one question, and so on. They are all implementation issues which I am sure will keep improving year to year.

I think the bigger question is whether such exams like NEET and JEE are fair even if the implementation is perfect and there are no charges of wrongdoing. Almost everyone in this country (except a few in Tamilnadu) seems to believe that if the same question paper is given to two students (may be in their respective languages) then it is a fair exam. And the believers include most academicians as well.

What is fairness. What type of student should, in general, be ranked higher? Do we have an answer to this question? No. In the past, I have asked IITs whether they have a definition of what kind of student they want to select and why, and whether they have at any point in time checked if JEE (and now JEE Advanced) is selecting those kinds of students. If you don't even know what kind of students you are trying to select, how can you declare fairness.

Let us assume a definition of fairness. An exam will be fair if a student who has learnt what has been taught to him/her very well because this gives a university hope that the student will learn well in the programs to which s/he will be admitted.

Now think about it. A student from board of state 1, and another student from board of state 2 are given the same exam which is based on the syllabus of the board of state 1. Is this a fair exam.

This will be a problem with any exam under the concept of "One Nation, One Test." The syllabus of the exam cannot be the intersection of all state boards since that would be too small a set. And if you are going to test it on any other syllabus, its overlap with different state boards will be to a different extent causing unfairness.

Such a test is not just unfair to students, but it is also unfair to universities. Every program is slightly different and this may need different inputs. A university may have a program which requires a greater focus on communication and hence they may want to admit students with decent language skills. Should they not have the right to do so. Admission is basically a system of predicting who will succeed in the program or benefit most from the program. Can we say that the same test will predict success of all programs in the country.

Unfortunately, in a country with very few good colleges, the competition for those few colleges is so intense and the incentive to game/hack the system is so high, that fairness takes the back seat. Any agency conducting the exams will only care that the exam can result in a ranking of students and the process passes the judicial test of fairness. Who cares if it does not pass the academic test of fairness.

Given the reality of few good seats being chased by large number of students, is there no hope of at least improving fairness. There is. One can go back to having multiple exams, may be one based on each state board syllabus. May be there is an exam conducted by a private party which will include language and other skills. A student can try taking those exams which are closest to examining his/her strengths. And a university can choose the exam that it wants to use for admissions. It can even have some seats through one exam and other seats through other exam and so on.

There are two perceived problems in this. One is that students will have to take a large number of exams, and two, the earlier system prior to NEET was roughly this and it had a lot of corruption including capitation fee.

My answer to this will be that students will need to take 2-3 exams only, may be 4, which is not a problem. In fact, it is good that they will have something to fall back upon if they don't do well in one exam. Second, the corruption happened because the admissions took place at the college who could manipulate the merit list based on capitation fee. We can easily have a centralized counseling for admission. Each university can tell their criteria of ranking applicants and a centralized application server will get applications from students for all colleges they are interested in, and the seats will be allocated centrally. In fact, universities may be allowed any parameter which is objective in nature (to stop manipulation). They may even use grace marks for under-privileged students as long as the criteria for deciding under-privilege status is objective.

So the real solution to NEET is to have several smaller exams. That will be more fair to both students and universities.

Problems with NEET 2024

We have serious allegations about the NEET exam this year.

Police has arrested people who have shared an exam paper one night before with some students who paid hefty sums for a look at the questions and answers and they are agreeing that the questions that they crammed the night before indeed appeared in the exam next day. (Media Report here)

We have seen police action on a center where some students were asked to leave the difficult questions which will be marked by the person before packing up the OMR sheets. Several students from around the country somehow knew about this and gave the option of this center. But, of course, when several students knew about this, it also means that someone will leak the information to Police and that is what seems to have happened. (Media Report here.)

The next problem is that of "wasted time." This happened because in many centers, there was an error in distribution the correct question paper (language) and by the time, the correct paper could be distributed, significant amount of time was wasted. And NTA decided to compensate for the loss of time by giving grace marks (which they later withdrew). This is probably the most ridiculous thing that has happened and points to incompetence of NTA. I have been involved with IIT JEE in the pre JEE Advanced days, when several lakh students took the exam with OMR sheets. We almost never had a situation where wrong question paper is being distributed, and if by chance, such an error indeed happened, the center representative was authorized to compensate for that time by extending the closing time. If the exam started 10 minutes late, it will end 10 minutes late. It is as simple as that. Did NTA not have such a simple SOP? Also, when this issue of wasted time came into limelight, and a decision was taken to compensate for the time, they did it only for those students who went to court. If a student did not go to court but only complained to the center, no compensation. What happens to poor students. Is it obvious to everyone that they should go to court for every small thing which can be handled at a center level.

I wonder why they can't use the same model as JEE Mains for NEET. There are lots of sessions, which means that there are lots of question papers. So any one paper getting leaked, the person accessing it would not know whom to sell and hence the incentive for leaking the paper is minimized. Also, it is online and hence there are no physical copies to leak. Further, if there is any problem with the machine, the exact time and every keystroke is recorded, and hence it would be easy to figure out what was the loss for each individual student. And, there are two groups of sessions and every student can take the exam twice. This way, if some thing indeed happened with a few individuals, they have another attempt. Not every minor incident would need to be handled by re-examination.

Of course, this is not to say that JEE Mains is above all suspicions. People are very creative and there is technology to hack every system. But, by and large, JEE Mains is a better managed exam than NEET and we can learn from it. And both are conducted by NTA only.

Having said all this, I think the biggest problem with NEET is not the poor conduct of the exam, but the notion of "One Nation, One Test." This notion puts all the eggs in one basket, makes it super important for people to do well in that one exam. When the competition is that high, some people will use their creativity to hack into the system and sell their service to students and parents and there will always be some students/parents who will agree to get ahead through these means. And of course, "One Nation, One Test" may sound fair, but it is not fair, and I will write another blog around it soon.

I don't know what solution will NTA or Supreme Court will come up with. But I have one request. All those who have received 720 marks out of 720 (some of them removed from the list after grace marks were removed), they should all have the right to which ever college they want to study in. I know NEET had pre-announced the tie breaking rules. I am OK with those tie breaking rules for everyone but not for those who have got 720 marks. For everyone else, one can say that you should have worked harder, gotten another question right, but you can't say this to someone who has got all questions right. What more should s/he have done to get admission to their top choice.

An 18 year old who have worked really hard and got 100% marks is now being told, "Sorry, you can't get into AIIMS, Delhi, because of things that are not in your control." This is unacceptable. If there aren't as many seats in a college as are the number of 100% students are interested in, then for this particular year, we should allow super-numerary seats.

Return-on-Investment in Education

During this time of the year, when millions of students are deciding which college to go to and what program to enroll in, one question that is often asked is, "What is the RoI of the program?" RoI means "Return on Investment.

Now, RoI of education is a bit complicated, because it is not just the additional money that you can potentially earn, but also the quality of job, and the quality of life, happiness, and the privileges of connections and so on. But let us assume that everything can be reduced to a number, the financial returns.

What is the financial return of a program. Well, one has to see what would be the estimate of lifelong earnings of the person without doing that program, and compare that with the estimated earnings of the person after doing the program. The difference between the two can be attributed as return due to that program. Now, you can sense that this is not easy. One can perhaps model an average person but it does not work for an individual because your return on investment could be very different from my return on investment simply because you are more prone to doing a corporate job after the same degree while I am interested in becoming a teacher. The financial returns over the next 50 years in an uncertain world under two hypothetical conditions, impossible.

But still we need to decide whether to spend a very significant amount of money or not. Is there a proxy for RoI calculation. Yes, of course. Without this, what will all the consultants do. It is a simple proxy. You take your expense over the program, and look at the first month salary of an average student in the last batch graduated and computed in how many months one would get as much salary as the amount spent in the program. So, if the fees for a BTech program is 15 lakhs over 4 years and the average package in the university for that program is Rs 50,000 per month, then the RoI is 30 months (never mind that RoI in financial world is in terms of percentage, here months will do). One can question the exactness - you haven't taken into account the hostel/mess expense, you haven't taken the cost of time, you haven't taken into account the interest, and all that, but well, it was always supposed to be a proxy, an estimate.

But this has several problems. (And I am not even talking about how do you know these numbers. Everyone seems to present these numbers in the most confusing way.)

First, should you consider the highest salary in RoI computation or the average salary or the median salary. What is your expectation from the program - do you expect to be closer to the highest or average or what. One would say that what is the expectation of an average student. That should determine RoI of the program. But the expectation of an average student is the median placement offer and not the average placement offer. And I have never seen anyone talk about median. Everyone talks about average which could be significantly higher than median. Next, should you only consider jobs in the core engineering or all jobs of students graduating with that degree. If you look at all students, then you will get biased result. The RoI will be biased by the societal bias (good students getting to a program causing higher placement and not something that you learn in the program causing higher placement). On the other hand, if you look at only core jobs that is not right for the undergraduate programs because it is expected that you are learning to be ready for a non-core job.

OK. So we will toss a coin and take a decision of what jobs to consider for RoI. But then this method has other serious concern. If you have a choice between a 4-year BTech and a 5-year BTech-MTech dual degree (or integrated MTech), what would you choose. Your cost has gone up by at least 25% (if you don't include interest and the fees is not going up by inflation) and unless the median/average salary offer is also higher by 25% or more, it does not make sense to do MTech. In certain industries, and certain kinds of jobs, MTech is still considered very valuable and over 50 years of career, you will be compensated for that one year extra education.

Similarly, there are disciplines which have a slower start but you get to the same point in due course. So if you are taking a decision based only on first month salary, you will not choose disciplines with slower start. The reason why some of the engineering disciplines are less popular today is that first people tend to over-estimate earnings based on highest salaries offered in certain popular disciplines, and second, the core engineering have a slower start of the career.

There are other slow-start careers like Design (this is changing though). In disciplines like Design where one works with other designers in small firms, the importance of working with better known designers is so much that you may have to work for a much smaller salary initially till you establish yourself. And, therefore, the number of students interested in design are less and as the opportunities for earning better salaries right in the beginning are improving, the number of students getting interested in design are increasing.

The obvious question then is this. If estimating lifelong earnings is impossible, and just considering the estimated first month salary is deeply flawed, how do we choose between the two programs.

My answer is this. If you are taking education loan, and you don't have family support to pay that loan, then you really have no option but to look at the first month expected salary. (But even in this situation, you should only decide whether you will be able to pay the EMI, and not compare programs on where you will have more savings or less savings after paying EMI.)

In general, if you are being funded through parental savings, then think like this. What are people doing 5-10 years after doing that program. If it is a new program, can you guess based on a "similar" program. If it is a new college, can you guess based on similar program in other colleges where you think the quality of education is likely to be similar. And if that profile is to your liking, both in terms of job content and reasonable financial gains, it is ok to do that course. And if you have a choice between several such courses, choose the discipline you are interested in, or choose the college where you think the quality of education (broadly defined) will be better.

This advice is based on my life principle that while it is important to have financial security and enjoy the good things of life, beyond a certain level of prosperity, one should look for happiness, impact, time for family, and such, and money should become secondary. And if money is primary for you, go back to previous paragraph and see which program leads to best jobs after 5-10 years and one way to check that is to check linkedin profiles of alumni. So check out 5-10 years and not one month.