JEE 2013: Will it revive the school system

The Directors of IITs have been telling us that the new JEE is a magic bullet. If we bite it, miraculously, our schools will improve.

What is the evidence?

Oh! We believe it. Our Minister believes it. Isn't our collective belief a good enough evidence? We are the Directors of the best institutes in India. We are smart people, much smarter than you are. If we collectively believe in something, that is evidence.

And I thought if they don't learn from me, that is ok, but they should at least learn from their boss, who is one of the finest lawyers, and what is law without evidence.

But I persisted. Belief is not evidence.

That is exactly why we are the Directors and you are a professor. Belief is evidence, if boss accepts this as evidence. The sooner you learn this, the better it will be for your career.

But, why not look at states which started using board marks for engineering admissions either fully or partially. Have we noticed the improvement in performance of students of those states in a quantifiable way. (For example, is there a study which shows that school education in Tamilnadu has improved in any significant way, since they started using the board marks for admission to engineering.)

One honourable member of the IIT Council from Tamilnadu told me that the school education has suffered tremendously since they started using board marks for admission to engineering. He told me that there are model question papers, question banks based on last 5 years question papers for each subject, and different coaching institutes will train you for answering all those questions, and one is reasonably sure that if one just remembered a few hundred questions and their answers in each subject, then one will get excellent scores. Since the "value" of board exam has increased tremendously, learning has suffered, and the students are only concerned about marks in the board.

I do not know whether to believe this or not. But what I find strange is that the same IIT Council member does not suggest to the Council that we go slow and do some study before repeating Tamilnadu experience in the entire country. When this happens, it is difficult not to think of motives.

But Directors are not impressed by these questions. Their belief is more important than any data, facts, and analysis.

In research, it happens many times that I have an idea, but I can't mathematically prove it. The system is too complex to come up with an equation. What do we do. We do some simulations. We make some simplifying assumptions and then try to analyse it. We learn about the system better, and then we make a more complex system. Basically, we go in steps.

But one of the IIT Directors have famously said that the methodology for deciding education policies of the country do not have to follow the methodology of doing research. He is essentially admitting that when one decides education policy, it is alright to be arbitrary and ad hoc, and one can play with the careers of millions of people, based on the "feelings" of a few smart Directors. No research is needed to decide education policy.

Most educationists tell us that schools will not improve unless they have teachers, unless there are class rooms, unless there are black boards and chalk, and so on. But our Directors believe that if 12th class marks are considered for admission, this will improve the schools. I find it hard to believe, and hence am asking for scientific validation of this belief.

JEE 2013: Discrimination against quality and law of large numbers

Here is the Wikipedia page on Law of Large numbers. It defines the law as: "the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed." A variant of this (again from the same Wiki page), called Borel's law of large numbers states that if an experiment is repeated a large number of times, independently under identical conditions, then the proportion of times that any specified event occurs approximately equals the probability of the event's occurrence on any particular trial.

So, here is a question that IIT JEE would expect you to know the answer of.

If  two persons, "A" and "B" pick up the same telephone directory, and randomly select 10000 persons each, and find out the average age of their group of 10000 persons. Which of the following statement is true:

1. Average age of "group A" would be approximately same as that of "group B"
2. Average age of "group A" would be substantially higher than that of "group B"
3. Average age of "group A" would be substantially lower than that of "group B"
4. The two random groups can not be compared.

I do understand that in a JEE question, I would need to be formal about various words, but please let that pass.

If you are like one of our students, you would have marked the first statement as the right answer. Congratulations. You have learned the law of large numbers. But wait a minute. Let me test you further.

Let the person "A" pick up the telephone directory of Mumbai, and person "B" pick up the telephone directory of Delhi. Would that make a difference. If you answered that it won't make a difference, you are being reasonable, though someone must test it.

 Now, let us consider a hypothetical city, SanghiNagar. The city has made a few interesting laws. It does not allow a phone to be in the name of anyone less than 21 years of age. Further, it wants all old people to have a phone, and hence it gives a tax rebate on all phone connections in the name of the people whose age is more than 60 years.

Now, let person "A" pick the telephone directory of SanghiNagar, and person "B" pick the telephone directory of Delhi, and select randomly 10000 names each. Would you expect the average age of the two groups to be same in this case.

If you are a student of an IIT, you would have answered it in negative. The reason is that the two populations are statistically very different. You would expect the average age of the selected group in SanghiNagar to be higher than the average age of the selected group in Delhi.

Now, ask your Director the same question. He will tell you how wrong you are. He will tell you that the law of large numbers will operate over two separate samples and give out the same statistical quantities, that once the quantities become really large, they cannot be statistically different.

When we teach in our classes, we encourage our students to question us, find errors with what we have taught in the class, and that is how our students learn, and we learn too. But, we can't question our directors.

If you take the group of CBSE students and the group of a state board students, can you say that the 90 percentile of CBSE group has similar academic learning as the one at 90 percentile in the state board.

Notice that state governments put in very minimal amount of money into school education. They don't have good infrastructure, they don't have enough teachers, and there is very little accountability. On the other hand, CBSE affiliates a large number of private schools, which are more serious in providing education, at least better than what a typical government school in most states would. CBSE also affiliates several central government funded schools (like Central Schools) which are much better managed, endowed, etc.

Are the two groups statistically similar. Let us look at the statistics. If you consider the number of "PCM" students in the country, CBSE only has a little more than 15 percent of them, but if you see the IIT students, more than 45% of the students are from CBSE board. Recently, I was in a meeting in one of the states. I was told that about one percent of 12th class science students in the state are from CBSE, while 98% are from state board. But if you look at the entrance exam results, the number of CBSE students in the top 1000 is several times more than 1%. (This is when a large number of CBSE students would not give the entrance exam, since they know that only 1% seats would be for them - they have a board based quota for admission.) We can check the same thing across the country in a variety of tests. And note that I am not talking about minor differences here and there. We are looking at a performance which is a multiple of others' performance.

And the reason is easy to see. As I said above, state governments are simply not putting enough resources in school education.

But our Directors have a different take. Every board is a large board. And their understanding of the "law of large numbers" is that all large populations will give the same statistical results, and any proof to the contrary will be put in to the waste paper basket.

And they can, therefore, go ahead and pronounce to the world that a 90 percentile in CBSE is same as 90 percentile in a state board, and therefore, the percentile marks can be used for admission purposes.

What is the impact of this. A board which has done reasonably well in terms of having a good curriculum, inculcating a decent pedagogy, insisting on minimal infrastructure in every school that they will affiliate, ensure a better attendance of students than other boards, carry out exams in somewhat better conditions, less cheating, and so on, and the board works to attract good schools to affiliate with itself, is going to be discriminated against, just because they deliver better quality.

The message being sent is simple, but yet effective: We want to reduce the quality of education in this country. If you are going to join a school affiliated to a better board, we will make sure that your chances of getting admission in IITs, NITs, and other CFTIs reduce. You still have time. Leave CBSE. Join your state board. With the same effort, you will be able to get a much higher percentile in a low-quality board than in a high quality board. So why join the high quality board. After all, you shouldn't be worried about school teaching any way. Whatever you need to learn, you will learn in a coaching class. It is just a matter of giving some exam. Give the simplest exam.

I am sure the Directors don't deliberately want to attack quality. They had good intentions when they were appointed Directors. The only problem is that they don't know the law of large numbers, but they are not willing to admit it.

JEE 2013: Why I feel cheated

We all feel cheated. We had a meeting with the Minister in April. He asked all the seven Senates (of the older IITs) to give suggestions on JEE 2013 and beyond. He suggested a methodology for incorporating their views into the final decision. He said that once all the Senates have given their views, there will be an IIT Council meeting which will come up with a proposal that is based on the feedback given by the seven Senates. This proposal was to be shared with all IITs. This was to be discussed in a second meeting of IIT Council. For this second meeting, each Senate was to nominate 2 persons each who would be invited to this IIT Council meeting. In addition two persons from All India IIT Faculty Federation were to be invited as well. The second IIT Council meeting was to discuss the proposal in presence of these 16 faculty members, and differences were to be ironed out with the help of these 16 faculty members.

What really happened is a very different story. The first meeting came up with a mechanism which most Senates had opposed to. In particular, most of the Senates felt that IITs should continue with their current process of JEE for 2013 and the new process should be adopted only from 2014. This was not only fair to students who have given 12th class already, or have already spent a lot of time in preparing for the JEE 2013, but this was also felt necessary to test various assumptions that the new system makes, and many of those assumptions are simply ridiculous (like every board being equal) and contrary to the data that we have. Even this request was denied. Informally, several Directors have told me that 2014 being the election year, no major changes can be done in that year. Now, are we going to adopt an admission process with so many questions about it, based on political expediency.

There was no invitation to Senates to send two representatives to the next meeting.

I am sure a lot of persons pushing their agenda are congratulating themselves on their smart moves and strategies, and they would have stories to tell their grand children, how they made fool of more than 1000 IIT faculty members. I have only this to say to them: It is possible to win by cheating. That is what you have proven. A system created by cheating, cannot but allow cheating. Come 2013 admission time, a lot of hard working, intelligent, honest students of this country will find out, it is possible to win by cheating.