Normalization of Admission Tests

So NEET is back in Supreme Court, this time a candidate asking for normalization between NEET I and NEET II. Here is a news item:

NEET I and II combined results challenged in Supreme Court

First of all, why is it that no one challenged JEE. They too have multiple papers, and by no means the papers are of equal difficulty levels. The JEE Mains has not been challenged, because someone who gave a slightly tougher exam but still still got shortlisted for Advanced JEE, does not care. And someone who didn't get shortlisted for the Advanced JEE, knows that even if s/he was shortlisted, the chances of getting through to IITs was remote. And frankly, people don't quite care for admission to NITs as much as they care for admission to IITs and for admission to MBBS programs.

I recall a meeting in CBSE regarding JEE. Before the meeting started, out of curiosity, I asked the officer handling JEE Mains whether the averages of the paper version and the computer version were same. Without blinking an eyelid, he said, yes they are identical. I kept quiet.

At the end of the meeting, the CBSE Chairperson was wondering why so few candidates give online JEE, even though there is significant benefit in giving online JEE. The paper exam is just after your board exams are over, while the online exam gives you additional time to revise everything. And the conversation went something like this:

Me: Do you recall the question I asked before the meeting about the averages of the two papers.
CBSE Chairperson: Yes.
Me: I can guarantee that the officer was lying. He has never seen the two averages.
CBSE: How can you be so sure.
Me: He has a laptop in front of him. Ask him to check the information now, and let us know the exact numbers.
JEE Officer: Sorry, Sir. I don't know the averages.
CBSE: (Now very curious), But how could you guess?
Me: Averages of lakhs of students can be close but not identical. If he had seen the numbers and they were indeed very close, he would have said that they are within 1% of each other, or something like that.
CBSE: But what does this have to do with students not taking up online exam.
Me: The perception is that the online exam is much tougher. Your processes are very opaque and you don't give out any statistics. So you do nothing to tell people that either the exam is not much tougher, or yes, it is tougher, but then we normalize.

At this point, the officer who handled JEE Mains exam informed us that the way they handle the differences between the two exam is that they ask the group who prepares the question papers to ensure that the two papers are of equal difficulty levels, and we get a certificate signed by them that it is indeed so.

This was, frankly, very shocking. Think about it. There are persons out there who are willing to vouch in writing that the two question papers are of exact same difficulty level without any data, and just on the basis of their experience. I don't know who are these persons with no understanding of testing, but I can only say that these people don't deserve to be anywhere close to that question paper. And for CBSE to involve such persons in preparing the most important question paper for over 10 lakh students shows how CBSE lacks competence about testing. And if, doing this was expedient in the beginning, wouldn't you at least look at data later on. And if the data miraculously pointed to similar distribution of marks, make that information public so that the public perception about your processes become more positive. But once the exam is over, everything about it is best forgotten.

I am extremely happy that someone is challenging the lack of normalization in NEET. I only hope that there is no miracle here and NEET I and NEET II indeed have different distributions of marks. Otherwise, we will be strengthening the argument that it is possible for a man to look at two question papers and guarantee same distribution of marks.

IITs as accreditators?

We have problems in the country, lots of them. But we also have solutions in the country. And thanks to successive governments' indulgence, lots of them too, 23 of them at the last count. And there is a promise that if these 23 can not solve all problems of India, we will create more of them.

One of the problems is the poor quality of higher education. It was hoped not too long ago that India will reap the immense benefits of a young population, the demographic dividend, as they said. But without a good quality education, that dividend seems to be becoming a liability. One of the ways of improving quality is to insist on accreditation by a bunch of supposedly quality conscious individuals following some guidelines and checking things in each college. NAAC was born out of this requirement. But NAAC has not been able to grow fast enough. It is said that at the current rate of accreditation, they will take 38 years to accredit only the existing institutions, and in these 38 years there will be many new ones. So in the foreseeable future there is no hope of making accreditation compulsory for all.

The new HRD Minister is quick to figure out that we need multiple accreditation bodies. This is, not a new revelation, of course. A lot of people have been saying that we need accreditation agencies in the private sector. But when it comes to education, private sector is bad, not to be touched with a 10 feet pole, never mind that most of our students are studying in private institutions and that many of these are superior to most government institutions. So the minister has been advised that perhaps IITs could be the accreditation agencies.

Here are the links to some of the news reports regarding this:

Centre plans 10-20 more accreditation agencies besides NAAC, says HRD Minister

Prakash Javadekar for giving IITs and IIMs accreditation body status

Do IITs have the bandwidth. Will this not be a distraction. Remember, compulsory accreditation once in 5 years for 38,000 institutions means 7600 accreditations per year, and if 20 organizations are involved in accreditation, then each of these organizations on an average will have to accredit 380 institutions in a year. That is a huge workload in addition to teaching and research.
 
I am sure some IITs will see a huge business opportunity in this. We can hire an agency to do this job under some bit of supervision from our side. Charge heavily so that a decent profit is made which can improve the quality of education at IITs further. Basically, a transfer of wealth from poor institutes to richer institutes.

But there is a more fundamental question. Should someone who has scant regard for the accreditation process be doing it. Today, accreditation is no longer based on inputs. You don't just look at the infrastructure, number of faculty, etc. You look at what are the vision, mission statements of the institute. What are the goals of each academic program. What is the outcome of each course in that program, and how it contributes towards the overall goal of that program. Having looked at all of it, one then asks questions regarding, how do we know that these outcomes are being attained. We also look at the whether there is a process for continuous improvement, looking at those outcomes and goals and discussing whether they remain relevant, etc.

I can't say about all IITs, but I can confidently say that IIT Kanpur does not do anything of this sort. We do not have a vision statement even 56 years after we were set up. We don't have goals for our programs, we don't have outcomes for our courses, etc. We are perhaps a unique institution which says that while there will be a feedback of all courses every semester, that feedback can not be used for any administrative decision. If you look at the way proposals are drafted, they invariably would not talk about options and what are the pluses and minuses of those options. The way minutes of the meetings are written, one could be forgiven to think that our language of instruction is not English. Overall, they have complete contempt for this process, which they believe is too bureaucratic, and is designed to generate employment. And that is fine. They still have done wonderfully well in academics. They still teach well, and they still do good research. They may fail the accreditation test, but if they don't want to go through accreditation (as long as it is optional), that is their call. And the market has already decided whether they need accreditation or not.

But while, it is alright if IIT Kanpur does not follow what Washington Accord suggests as best practices, or what NAAC will come and check, but it is not alright for such an institution to inspect other institutions and demand that they follow the same best practices which we have contempt for. And I suspect that it is not just about IIT Kanpur, but a lot of IITs are in the same boat.

So while I appreciate that the Minister has identified a very serious shortcoming of our educational system, but asking IITs and IIMs to solve that problem is not correct. Particularly when it is obvious that IITs can not afford to do this with their own faculty, and can at best provide a very slight supervisory role with an outsourced agency. The better thing to do will be to seek private participation in this.