National Testing Agency

For a long time, a need has been felt for a body that can conduct massive public exams that can be used by universities for the purpose of admissions to various programs. Till now, CBSE has been carrying out this task, but there were two problems with this arrangement. CBSE was too overloaded, handling school leaving exams and all the issues related to affiliation of schools. Second, the college admission tests are very different from school leaving tests, and there was a need of different kind of expertise to do research into these tests. Both problems were solvable, of course, but it was best to have a different body altogether.

Last year, the government announced setting up of National Testing Agency (NTA). Today, it was announced that NTA has started working and it is going to conduct three major tests in the coming academic year. The most important announcement was that two of these tests - JEE Mains and NEET would be conducted two times a year.

This is all very good, indeed music to my ears, since my blog has been saying for many years that we need to have exams conducted multiple times. But I also want to point out possible issues in this, and what they may do in addition to conduct of exam.

First of all, an exam being conducted more than once should not just mean that a student can choose his/her date of the exam. It should mean that a student can actually take the exam more than once. Minister has indeed said that the student will be able to take the test on both occasions, if s/he so desires, and I hope NTA will do exactly that.

Second, when a test is conducted twice, one must be prepared for normalization of marks. Unfortunately, CBSE wasn't doing that for JEE Mains. There were multiple tests being conducted even now (online and offline) on different dates with different question papers. But their absolute marks were being considered without any normalization. This is very unfair to students even if the degree of difficulty is different by a small margin. These tests are used to fill 10s of thousands of seats in several institutions, and a few marks can make a big difference.

Third, we need a strong research wing. Testing is a science, and just asking a few IIT professors (or school teachers for that matter) to prepare a question paper is not enough. We must analyse what type of questions are difficult, moderate, simple, etc., and whether there is a reasonable mix of them. We must also analyse when some question is answered wrong by a lot of students whether the topic is too difficult, or whether the language is confusing, or whether most teachers in the country are not teaching it properly. We must also look for performance of different segments of society on different questions. Are there questions on which some particular segments perform particularly poorly. That may yield some inherent biases which if the paper setter is aware can be avoided. I can go on and on. This is really a huge issue in the current exams in the country that we don't analyse them enough.

Research is also needed in newer type of questions. Also, what kind of exams will best predict success in different programs - what exam is a meaningful one for engineering, and what is meaningful for medical, and what is meaningful for BA/BCom/BSc type of courses. The current exams may not be the best predictors of success in the programs to which students take admission in.

During a twitter discussion, my friend, @learning_pt pointed out that there should also be research on interfaces. Is computer interface that was used for JEE Advanced by IITs this year the best one. Did we have the same distribution of successful candidates this year as in the recent past or did we lose out on a segment of society because they somehow could not get adjusted to the new interface. If yes, we may need to do something about it. Hopefully, we will not find such discrimination, but if we do find it then we need to solve this problem. May be we need to continue with the offline version for some more time. May be we need to have better methods of giving them practice in realistic scenarios.

The testing agency should give out detailed information about each test so that different universities can use the same test result in the way they find it appropriate. It is already been done for JEE Mains, and should continue, and may be made even more elaborate. For example, raw scores in all subjects (after normalization, of course), total, percentile (normalized with respect to last few exams), may be even details of sub-disciplines, or how did one perform on different types of questions, etc.

NTA may consider designing an aptitude test (for different programs), communication test (in different languages), and conduct these tests too a couple of times in a year, and universities may use them in addition to JEE kind of exams. All these will require research.

As we move forward, testing may become a continuous process a la GRE, and we may go for adaptive testing based on a large database of questions. There is research going on in automatic generation of questions, which could come handy here.

Much of this expertise may not be available at government salaries. Would NTA remain a body bound by government rules, and only conduct large exams, or would NTA be a body that breaks new ground in testing.

Overall, I am very happy with this development and am hopeful that NTA will do many of these things in due course.