JEE 2013: The Compromise

Apparently, we have a compromise. What it is, we know only from what media is telling us. I guess minutes will come out soon (to be followed by revised minutes), and we will know if media is right. Of course, there will still be many implementation details to be worked out, and it is already July.

So, here is what the media is telling us:

1. To be eligible for admission to IITs, one will have to have a minimum of 80 percentile marks. (Note, it is percentile and not percent. It means that you should have marks more than 80 percent of the students in that board, or in other words, you should be in the top 20 percent of the students in the board. The marks will be different for each board. If media is to be believed, the marks are 78% for CBSE and Tamilnadu board, 65% for UP Board, 70% for Rajasthan Board, etc. Also note that this eligibility is checked only at the time of admission. Anyone can give JEE Mains. And candidates shortlisted as below can give JEE Advanced, irrespective of their board results since they may not even be available at the time of the exam.)
2. Every student interested in getting admission to IITs must give JEE-Main (similar to the current AIEEE). You will have to have a rank within the top 1.5 lakh in that exam.
3. After the result of the JEE-Main is out, JEE-Advanced test will be conducted. Only the shortlisted 1.5 lakh students will be able to take this exam.
4. Ranking for admission to IITs will be strictly based on the ranks in JEE Advanced.
5. JEE-Mains will be handled by CBSE, with help from IITs (and several others). In particular, IIT faculty will prepare the question paper.
6. JEE-Advanced will be conducted by IITs only. There will only be some administrative coordination with CBSE. The format of JEE Advanced will be decided by Joint Admissions Board of IITs. (Though some media is saying that it will be same as JEE 2012.)

Let us first discuss this percentile issue. As I have already said in my previous blogs, percentiles are not comparable and comparing them is unfair to students belonging to good quality boards. Academically speaking, there must be a way to compare multiple boards, and come up with different percentile scores for different boards. However, admission to government funded institutes is not allowed to be just an academic decision in our country.

However, the contention is that percentile scores are at least less unfair than the marks. I would really like to study this contention with data from 2012 or earlier, but 2013 is sacrosanct for some "important" people.

After we heard the decision of the Joint Admissions Board (JAB), we have communicated to our Directors that in the transition year (2013), we should keep a relaxed eligibility criteria. We wanted a criteria which will reject JEE qualified students in the rare case, just like the current 60% does. In particular, we wanted 70 percentile (so that 30% of board students are eligible). And once we have exact information on percentile of selected students next year, we could increase it slightly.

There are several implementation issues yet to be resolved. What is the meaning of top 20%. Do they count all those who register for the 12th class exam. Or do they count only those students who have passed the 12th class exam. Do they consider all students, or do they consider only those students who have studied Physics, Chemistry, and Maths in 12th class (besides other subjects). Do they consider the total of 5 subjects or 6 subjects (in case of boards which have 6 subjects) or do they consider only 3 subjects to rank students for the purpose of computing percentile.

Assuming that they will consider percentile score only amongst those students who have passed 12th class exam with PCM subjects, let us do a back-of-the-envelope calculation. I am told that there are about 20 lakh such students in the whole country. Top 20% means that 4 lakh students are eligible. (Since two batches are considered for JEE, 8 lakh students are eligible for admission to IITs.) The current 60% rule makes about 6-7 lakh students eligible. (Anyone with better numbers?) Top 30% would have made 6 lakh students (per batch) eligible, and hence close to current numbers.

When we are making too many changes together, it results in a lot of confusion, and a lot of stress. So, my suggestion to JAB will be to only change the process this time, and not change other parameters in any significant way. Let us make it 70 percentile for 2013, and if the same two-stage process continues for 2014 onwards, we can increase the cutoff slightly every year, if the analysis of the data supports such an increase. In particular, the goal should be to reject less than 1 percent of JEE qualified candidates because of 12th class performance. Media will highlight the stories of these 50-100 students enough that from next year, everyone will take 12th class seriously (more coaching). You don't need to reject thousand JEE qualified candidates to force students to do 12th class coaching.

Another implementation issue is regarding students who passed 12th class in 2012. It is being reported in media that they too will be required to have a cutoff of 80 percentile. It is being said that if they do not have 80 percentile, they can give the board exams again in 2013. I hope this is not JAB's position. In 2006 (or was it 2005), when we limited the number of attempts to two, we gave a one time exception whereby everyone who had given 12th class earlier was allowed to give JEE. I think JAB can allow 2012 students with old eligibility, that is 60 percent. Again, be liberal in the transition year.

Some people have written to me saying whether an eligibility is better than the earlier model of screening. The contention of these students/parents is that in the earlier proposal, if someone got less marks in 12th class, s/he could make up for that in the JEE Mains exam. Theoretically, you are right. But let us look at the reality. Only 1.5 lakhs (or less) are being selected for JEE advanced. These are from two batches, and hence in percentile terms they represent 3.75 percent or 96.25 percentile of 20+20 lakh students passing 12th class with PCM marks. How much better could you possibly do in JEE Mains to compensate for a huge loss of 16+ percentile in 12th class board performance. If we had agreed to 50-50 formula of the earlier decision of IIT Council, almost everyone shortlisted for JEE advanced would have been someone with 95 percentile and higher, though an odd person with lower percentile but with excellence JEE Mains would have gotten through. Even that odd person would not have gotten through with less than 80 percentile.

The second major issue is that of selecting 1.5 lakhs from JEE Mains. Most IIT Senates had suggested a number between 50K and 100K, so that we can experiment with newer formats, including tests with written answers. IIT Council had earlier said that long answer tests are not possible because even grading 50K to 100K is very difficult in a consistent way. This is where the difference between administration and academicians come in. They are looking at the issue in black and white. Either fully machine graded, or fully manually graded. We were looking for a test which will have a few questions which require at most a few lines of answer. 150K would make it difficult for us to consider even a few such questions, and it will limit our abilities to do innovation with this exam.

But what we also wanted was that if the change has to happen in 2013 (as a compromise), then JEE Advanced in 2013 should be exactly same as JEE 2012. As I said, we should put in changes slowly, giving time to students to adjust. JEE Advanced in 2013 should be a machine graded exam. It means that the limit of 50K or 100K was not important for 2013. In such a situation, I would have actually liked a much larger number. We would like to study the correlation between JEE Mains and JEE Advanced. We would like to know if there are students who could get in top 10,000 in JEE Advanced but would not be in top 50K or top 100K or top 150K of JEE Mains. This will give us important information to design JEE Mains, and to decide how many students should be shortlisted for JEE Advanced. Since the number in the second stage in 2013 is not important (and we have the bandwidth to conduct an exam of even 5 lakh students - we just did it in 2012), we should actually select a much larger number for 2013 only.

The students are not really used to this new format. They will mentally be preparing for JEE Advanced (since they cannot prepare for it after the results of JEE Mains is out), and telling them just before JEE Advanced that they cannot give the exam for which they have prepared for such a long time is painful for them, at least the first time this happens. Now, JEE in 2012 has been given by 5 lakh students. One would assume that about half of them were serious candidates who had some real hope of getting through (basically students who got non-zero marks in all three subjects). This set could be allowed in 2013. Once this process of two-stage selection is established in the minds of people, non-selection will hurt, but students will be mentally prepared for this possibility. So, if the 2-stage process continues in 2014, we can reduce the number to 100K or less based on data analysis of 2013. But let us have 2.5 lakhs in 2013.

Also, when we say same format, it means the same format. Two exams of 3-hours each, with equal questions from PCM in both exams, etc. I am hearing some stories that it would be only a 3-hour exam. Not in 2013, please.

By the way, I am really hoping that JEE Mains, one day, will have aptitude part as well, and it will be given multiple times in the year. And attempts will be made to standardize this test so that scores in different versions of the test can be compared.

Once again, many students have written to me asking what is the utility of the 2-stage process. Let me take this opportunity to explain that here. JEE Mains will be a test where ranks have to be given to several lakh students, since it will be the primary exam used by many colleges (like AIEEE). It means that several lakh students must get non-zero marks. The bunching of students on the same score will be very high in this exam. At the middle part, a lot of colleges are of similar quality, and hence students with same score can go to one or the other without affecting their quality of education. However, at the top, the quality of education drops substantially after 20-30 institutes. Hence discriminating amongst the top 20-30 thousand students is more important. The current JEE is designed to have less bunching at the top, and a large number of students getting very low score to be able to discriminate amongst the top 20-30 thousand students. Hence admitting students from JEE Mains would have resulted in a sort of lottery and would not have been fair to really good students.

And if we have to have JEE Advanced for this reason, a smaller number for JEE Advanced allows us to experiment and innovate with the exam itself to really select the best for ourselves, and for anyone who would care to look at the merit list of JEE Advanced.

Recall that all earlier proposals have required everyone to give both JEE Mains and JEE Advanced, and that too on the same day. So the compromise formula does not increase the number of exams for students compared to what was being thrust upon us, indeed it reduces an exam for about 10 lakh students. So, it is an improvement over the earlier proposals.

I think, overall, it can be called a reasonable compromise, if some of the issues raised in this blog article are taken care of. I am certainly not excited about it. I do not believe that the 2013 admission process is any better than 2012 admission process. But we have to have a compromise since time is running out for students preparing for 2013 admissions, and the uncertainty must end. However, the compromise does create a platform for positive changes in 2014. I hope that JAB can quickly settle all pending issues so that there is complete clarity on what students can expect in 2013. Also, JAB should immediately form a committee to look into 2014 and beyond, and I hope that this time we will work together from bottoms up to come up with a good admission process.

I request JAB to do the following:

1. Keep the eligibility for students giving 12th class in 2013 as 70 percentile.
2. Keep the eligibility for students who gave 12th class in 2012 as 60 percent.
3. Select 2.5 lakh students from JEE Mains for the JEE advanced.
4. JEE advanced in 2013 should be same format as JEE 2012, including machine graded, two 3-hour tests, same weight for Physics, Chemistry, and Maths, same syllabus, etc.
5. Set up a committee who should be charged to improve upon the process for 2014.
6. Of course, joint counseling with NITs (and indeed anyone else who wants to join, since a software can easily handle multiple merit lists).

Sanghi and Sanghi

This is a special post. With this post, I complete my century of posts on this blog. I thought what should I write about as the 100th post, and decided to postpone my comments on the compromise reached by IIT Council yesterday. I am yet to make up my mind fully about its consequences. Will wait for a couple of days.

During this entire saga, a lot of people have asked me whether me and Prof. Sanjeev Sanghi of IIT Delhi are brothers or at least related, and I tell them the story of how we met for the first time thirty years ago. I thought it would be interesting to narrate that story on this blog. I hope Sanjeev does not mind my narrating this story.

I had received the admission offer to the BTech program of IIT Kanpur. Obviously, the whole family was very happy and we had to brag about it in the entire neighborhood. It so happened that Sanjeev's aunt was our neighbour, and close family friend. She told us about the existence of Sanjeev in IIT Kanpur, who had taken admission there two years earlier. I learned everything about Sanjeev from his aunt, and armed with this reference, came to join IIT Kanpur.

I had read a lot of stories about ragging in universities at that time, and I was quite scared. On joining IITK, I quickly found out that Sanjeev was a fairly popular character. He was famous for two things. He would take tutorials in the evening for weak students, and was considered an excellent teacher. Everyone knew that he is going to be a professor one day. And he was very good at reciting poems in "Veer Ras." No cultural program could be complete without his poem recitation.

It occurred to me that I could make use of his popularity in avoiding ragging at IIT Kanpur. Since "Sanghi" is such an uncommon surname, everyone would invariably ask me whether I knew Sanjeev, and my answer to them would be, "Of course, he is my brother." In reality, I had never met him.

I noticed that I was ragged very little. Most of the time, it would be asking some embarrassing questions. Some times, I would be asked to sing a song, which I hated. But this never lasted more than 30 seconds, since they hated my singing more than I did. The worst that happened to me was that one particular student asked me questions for almost an hour, and I was told to keep standing till his questions are exhausted. So I had to keep standing for full one hour in his room.

Many of Sanjeev's friends were confused. How come Sanjeev never told them about his brother who was giving JEE. But I could answer all questions about Sanjeev's background, his family, and all that. So they had no option but to believe me.

I was thinking how smart I was in avoiding ragging. One day, I was in Lecture Hall 7 (L-7), the largest lecture hall, where movies were screened in the evening in those days. I was waiting for the movie to start, and a student comes up to me. The conversation went on like this:

He: Fresher?
I: Yes, Sir.
He: Intro?
I: Gave my name, city I came from, JEE rank, department, etc.
He: Do you know Sanjeev Sanghi?
I: Of course, he is my brother.

A pin drop silence, and then he said: "My mother never told me that I had a brother lost in Kumbh Mela." (In Hindi, of course, with a dialogue delivery better than any Hindi Film actor.) He was none other than Sanjeev himself.

There was no way, I could continue to claim to know him. And then I noticed that ragging had changed. I was not embarrassed any more. I was not asked to keep standing for long. Just a friendly chit chat.

It later occurred to me that there was no ragging in IITK even in those days. What I went through for the first 4-5 days was actually more than what most of my batchmates went through in terms of ragging. Everyone was keen to rag Sanjeev Sanghi's little brother.

Moral of the story:If you think you are smart, you do not understand the situation.

JEE 2013: On Percentile Ranking

Consequent to my open letter to Prof. Barua, he has written on his blog about percentile ranking, an issue that I had raised in my open letter.

Unfortunately, he has once again missed an important point. ISI report talks about two primary assumptions. They are:

1. Aggregate scores are expected to increase from less meritorious to more meritorious students in any particular subject.
2. Merit distribution is the same in all boards.

Now, Prof. Barua, raises this issue of what is merit, and very cleverly uses different definitions of merit for justifying the two assumptions. For the first assumption, he says, "marks are not given randomly. The student who does better, gets more marks.   Clearly, if this is questioned, Board marks cannot be used for anything." For the second assumption, he says, "I cannot do much more but to suggest synonyms for merit in this context: innate ability, intelligence."

What do marks represent. Do they represent innate ability or do they represent academic preparation. I have never seen any teacher or any board or any educationist ever claiming that marks in the board exam represent (even in statistical sense) innate ability. They are supposed to represent academic preparation (san corruption, cheating, copying, etc.). Just try giving the 12th class board exam paper to a 5 year old, who is brilliant in everything s/he does, and you are not likely to get 100 out of 100.


So, if the definition of merit is "innate ability" then assumption 2 may be correct (as he says, there is no way to prove it, but let me give it to him). But no educationist will ever agree that whatever that undefined entity called "innate intelligence" is, that has a linear relationship with board marks, not just in total, but individually in each subject. Prof. Barua wants us to believe that someone with higher "innate intelligence" is likely to score higher marks (in statistical sense) in every subject, irrespective of the subject.

On the other hand, if the definition of merit is "academic preparation" (and this is the only logical definition that can be used when we talk about board marks in various subjects), then I have already shown in an earlier blog and many others have pointed out other studies that merit distribution is not same across all boards, and in particular, ICSE and CBSE have higher concentration of academically well prepared students. And hence the assumption 2 behind percentile based normalization does not hold.

 Hence, simple percentile ranking is not good. It is unfair to students of higher quality boards.

He then says that if the board performance across 42 boards have to be compared, then percentile ranking is the best. I can challenge that too, but there is an "if" in his statement. I think he needs to answer first, why board performance needs to be compared at all. Isn't there any other method to encourage students to go back to schools.

Percentile ranking is the easiest to explain to masses, and in particular to bureaucrats, politicians, and administrators (like IIT Directors). So, if you are taking a political decision, then percentile ranking is the best. But if you are taking an academic decision, there are ways to study the performance of boards, and hence give higher and lower weight to boards. It should be possible to say that 90 percentile student of a good quality board has a similar level of academic preparation as a 95 percentile of a poorer quality board (for a hypothetical case).

But are we talking academics here, or are we talking politics here.