Here is the Wikipedia page on Law of Large numbers. It defines the law as: "the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed." A variant of this (again from the same Wiki page), called Borel's law of large numbers states that if an experiment is repeated a large number of times, independently under identical conditions, then the proportion of times that any specified event occurs approximately equals the probability of the event's occurrence on any particular trial.
So, here is a question that IIT JEE would expect you to know the answer of.
If two persons, "A" and "B" pick up the same telephone directory, and randomly select 10000 persons each, and find out the average age of their group of 10000 persons. Which of the following statement is true:
- Average age of "group A" would be approximately same as that of "group B"
- Average age of "group A" would be substantially higher than that of "group B"
- Average age of "group A" would be substantially lower than that of "group B"
- The two random groups can not be compared.
If you are like one of our students, you would have marked the first statement as the right answer. Congratulations. You have learned the law of large numbers. But wait a minute. Let me test you further.
Let the person "A" pick up the telephone directory of Mumbai, and person "B" pick up the telephone directory of Delhi. Would that make a difference. If you answered that it won't make a difference, you are being reasonable, though someone must test it.
Now, let us consider a hypothetical city, SanghiNagar. The city has made a few interesting laws. It does not allow a phone to be in the name of anyone less than 21 years of age. Further, it wants all old people to have a phone, and hence it gives a tax rebate on all phone connections in the name of the people whose age is more than 60 years.
Now, let person "A" pick the telephone directory of SanghiNagar, and person "B" pick the telephone directory of Delhi, and select randomly 10000 names each. Would you expect the average age of the two groups to be same in this case.
If you are a student of an IIT, you would have answered it in negative. The reason is that the two populations are statistically very different. You would expect the average age of the selected group in SanghiNagar to be higher than the average age of the selected group in Delhi.
Now, ask your Director the same question. He will tell you how wrong you are. He will tell you that the law of large numbers will operate over two separate samples and give out the same statistical quantities, that once the quantities become really large, they cannot be statistically different.
When we teach in our classes, we encourage our students to question us, find errors with what we have taught in the class, and that is how our students learn, and we learn too. But, we can't question our directors.
If you take the group of CBSE students and the group of a state board students, can you say that the 90 percentile of CBSE group has similar academic learning as the one at 90 percentile in the state board.
Notice that state governments put in very minimal amount of money into school education. They don't have good infrastructure, they don't have enough teachers, and there is very little accountability. On the other hand, CBSE affiliates a large number of private schools, which are more serious in providing education, at least better than what a typical government school in most states would. CBSE also affiliates several central government funded schools (like Central Schools) which are much better managed, endowed, etc.
Are the two groups statistically similar. Let us look at the statistics. If you consider the number of "PCM" students in the country, CBSE only has a little more than 15 percent of them, but if you see the IIT students, more than 45% of the students are from CBSE board. Recently, I was in a meeting in one of the states. I was told that about one percent of 12th class science students in the state are from CBSE, while 98% are from state board. But if you look at the entrance exam results, the number of CBSE students in the top 1000 is several times more than 1%. (This is when a large number of CBSE students would not give the entrance exam, since they know that only 1% seats would be for them - they have a board based quota for admission.) We can check the same thing across the country in a variety of tests. And note that I am not talking about minor differences here and there. We are looking at a performance which is a multiple of others' performance.
And the reason is easy to see. As I said above, state governments are simply not putting enough resources in school education.
But our Directors have a different take. Every board is a large board. And their understanding of the "law of large numbers" is that all large populations will give the same statistical results, and any proof to the contrary will be put in to the waste paper basket.
And they can, therefore, go ahead and pronounce to the world that a 90 percentile in CBSE is same as 90 percentile in a state board, and therefore, the percentile marks can be used for admission purposes.
What is the impact of this. A board which has done reasonably well in terms of having a good curriculum, inculcating a decent pedagogy, insisting on minimal infrastructure in every school that they will affiliate, ensure a better attendance of students than other boards, carry out exams in somewhat better conditions, less cheating, and so on, and the board works to attract good schools to affiliate with itself, is going to be discriminated against, just because they deliver better quality.
The message being sent is simple, but yet effective: We want to reduce the quality of education in this country. If you are going to join a school affiliated to a better board, we will make sure that your chances of getting admission in IITs, NITs, and other CFTIs reduce. You still have time. Leave CBSE. Join your state board. With the same effort, you will be able to get a much higher percentile in a low-quality board than in a high quality board. So why join the high quality board. After all, you shouldn't be worried about school teaching any way. Whatever you need to learn, you will learn in a coaching class. It is just a matter of giving some exam. Give the simplest exam.
I am sure the Directors don't deliberately want to attack quality. They had good intentions when they were appointed Directors. The only problem is that they don't know the law of large numbers, but they are not willing to admit it.
Posts
55 comments:
You have correctly pointed out that percentile approach will make students choose state board. One may possibly argue to fix that issue by normalizing the ranking by size of the board. While intuitive and simple, such normalization wouldn't fully solve the problem.
Because I am afraid there is one more serious problem here.
In state boards, graders are paid very less per test copy and usually grade with room temperature 40C. These conditions make them a good random number generator. Rather than grading the answer correctly as per model answer sheet, they tend to select a number between 5 to 9 randomly if question is worth 10 marks (typical case for UP board). You may ask what is the implication of this type of grading on JEE 2013?
If you change the graders, you will get different rankings. That is to say that the percentile ranking for given board itself is not stable. A guy who is ranked 54th in state-board, had he got his social science paper correctly graded he would have been 18th. Note that law of large numbers do not apply here.
I would be very much interested in knowing how we are going to solve this problem?
--Devesh
You made us aware to entirely different aspect ,how a board will be punished for being good in quality.I am extremely worried, my son is going to appear for IIT in 2013,he is studying in CBSE board.
Nice to see a technical argument. Let us take this further. The selection marks for qualifying for JEE-Advanced are to be computed as follows:
Q = 0.5*S + 0.5*M, where S=state board marks, M=IIT-Main marks.
In your experience, for the last qualifying student, i.e. the 50,000th student, how important will the state board marks be in the above equation, to leap over the first non-qualifying student? What do you expect the distributions of Q, S, and M to look like at the 50,000th student mark? Can IIT-Main be set so as to negate any percentile discrepancies?
Please comment, based on your experience with JEE.
Also, I tried to post a comment earlier that your complaints against the new solution seem more procedural than about the solution itself. Please see my blogpost indillect.blogspot.in on why the new solution has a good structure. Of course it can do with some parameter tweaking, as with all solutions. In the first go it is important to get the structure right, parameter-tweaking comes next.
More importantly board exam marks can easily be rigged in all the states of this country. And they will be when the stakes increase.
Also, I disagree with your opinion on the intent of our directors. I do not think they have good intentions. Political alignment has eclipsed their fairness.
I see a sinister attempt here to paralyze the intelligentsia of this country. It will not stop with JEE. Other entrance examinations will follow.
The only one which will escape the wrath of the ministry is the civil services (which in my opinion should have been the first to be changed) because we have a strong IAS lobby protecting that rotten system. They will never allow the quality of graduation to be a parameter in the selection process. In fact, they want less number of people from IIT and other good colleges (remember how the exam was changed once half of the IAS batch started coming out of IITs).
Dheeraj,
Very well articulated article, bringing out why CBSE is not equal to state boards.
One thing I can share from my own experience, that the whole board system is full of rot. The exams are conducted in a totally disorganized manner. Copying is very easy and very broadly supported. Forget about a 90% person in board equal to 90% in CBSE, the 90% student in board may not even deserve 50%, but still gets 90% due to inefficiencies.
It's very pathetic, that our honorable education minister is focusing on the wrong issues, while ignoring to focus on improving quality of education starting from elementary schooling. Problem solving and goal based learning are very important.
If a student doesn't fit into popular tracks of engineering or medicine, then hell for them to choose an off-track career courses, though it is improving a bit recently.
Actions like this from the ministers make me loose confidence in democracy. What majority feels need not be always the right choice, compared to an option by a few intellectuals.
Girish Elchuri
@Devesh, Percentile is by itself normalization of ranking by the size of the board. The corruption and inefficiency of various boards is not something that Directors and MHRD would agree, since their viewpoint would be that it will reduce substantially in years to come, if the board marks are taken very seriously and there is a spotlight on them.
This is a view that I disagree with, but note that they are powerful people, and they decide what is right and what is wrong. As a small individual, I am giving a more scientific argument in hope that Directors will find it more difficult to question.
Very funny article! In the style of Marc Anthony "But Brutus is an honourable man"! But the content? It is not about the law of large numbers, but it is saying that the quality of students in CBSE are much better than in State Boards (or, generally, the quality of students in different Boards are not the same).
Here, the word "quality" is important. What do we mean by it? Is it purely based on what has been taught? Are we saying that there is no "innate" quality at all? How then did a poor boy from Manipur top this year's CBSE, studying in a very average school?
Another point: When we compare students from different Boards for admission to institutes like IITs, will the quality difference of students in the bottom 3/4th have any impact on admissions?
There are no easy answers for either side.
Gautam Barua, IITG Director
@GnaanaMaargi, Your earlier comment is also published on the other blog. I don't have sufficient data to answer your question. And that is precisely what we have been asking as Senate of IIT Kanpur. We have suggested that both JEE and AIEEE should collect a lot of data from the board this year (2012), do some studies of the kind you suggest. After all, as Prof. Barua, Director of IIT Guwahaty, and a strong public supported of Government proposal, has said, the first exam (JEE Mains) is like AIEEE, and the second exam (JEE Advanced) is like current IIT JEE.
The problem in the whole process is that no body is talking about data, but only perceptions. In the last 4-5 months, in so many of my blog articles, I have pointed out the kind of data that we need to really evaluate the scheme. The Senate of IIT Kanpur has said the same thing - we need data, and I believe other Senates have said so too, when they opposed implementation from 2013. ISI, when it was asked to study the normalization process said the same thing - they need more data.
But, neither the Directors nor MHRD are willing to collect and/or share data. It is obvious that their assumptions cannot stand any form of scrutiny. It is much simpler for them to argue on the basis of perceptions than facts, when facts are likely to be unsupportive. In case of perceptions, the "power" of position rules. Their perception has to be valued more than my perception, since they are at a higher post. Data and facts make them and me equal. They don't like that.
Even with the final dispensation that they have come up with, they should, in the first year, take a much larger number from board and mains, say 5 lakhs, to essentially not consider them at all in 2013, and let the merit list be based only on JEE-advanced. They should then do the study of the kind that you have proposed, and reduce that 5L to 4L in 2014, 3L in 2015, and so on, with a commitment that if any negative fall out is seen during this process, they will take corrective action.
@Easy Sea (Girish Elchuri), The corruption angle is very much there and will create problems in implementation. As many people have already pointed out, it is not clear what will happen to the student ho has made a request to the board for regrading or re-totaling, and gets more (or less) marks as a result. There are just far too many issues which have not been thought through. And therefore, they should have first done an experiment with AIEEE on a smaller scale, before forcing it on everyone.
@Gautam, You have been saying everywhere that 90 percentile of CBSE has similar quality to 90 percentile of a state board. I am arguing that there is no evidence for it. In all the entrance tests (including state level CETs, where the syllabus is in line with state boards), the CBSE students seem to perform better.
Yes, there is innate quality. But are you arguing that percentile will measure that innate quality.
Dear Professor Barua,
I think that innate or raw talent either does not exists or is at least highly overrated. There is no evidence. On the contrary, it is rather clear that better training and infrastructure can give better results be it sports or academics. How else would you explain countries that spent enough effort (both in terms of money invested and political will) like the US, Europe before the war and Russia (before the cold war) doing consistently better in
not just academics but also sports. You can quote one Ramanujan from India or in this case a student from Manipur but they are anomalies (as in any system).
Dheeraj's point that a board which is
willing to take the difficult path to
excellence is unjustly discriminated against is very valid.
There is no short cuts to excellence. I hope the minister (assuming he is interested in it) realises.
Piyush P Kurur
One issue with this XII exams marks is that due to grade inflation, everyone's marks approach 'large numbers'. If 1000 people get 100 in PCM, how do you rank them ? Some guy who does pathetically in the Mains can get through because of XII marks.
Sir, on an unrelated note, do you see this whole issue bringing down the quality of faculty as well ? As the quality of students goes down, why would some bright guy with a PhD want to waste his life here dealing with ministry mandarins ?
PS: In 2001 when I sat for JEE, my teacher used to say that prelims are like One-day, where even Prasad can hit McGrath for a six. But in the subjective Mains only Sachin can prevail. I strongly feel that we should have subjective component in JEE.
Prof Sanghi, I agree that there may be a possibility of gaming and selecting boards according to ease of getting marks. This exists in every college admission process; furthermore it exists in the choice of subjects too after admission to college. What is needed to solve this is some degree standardization. But as argument against the new exam pattern, I think it is one of those "higher-order terms" which are outweighed by all the other positives of the new system.
I think this article is relevant.
https://www.princeton.edu/main/news/archive/S33/87/54K53/index.xml?section=
I don't deny your technical argument, but the idea that the good and high quality education should be accessible only to select few is a fatal one. We need to find how to nullify these effects and provide equal access of IIT to all. IIT is a public system and it can not work on the principles of a private company. I am sorry that I am with the directors this time. There are lots of biases in the system and society can not progress if these biases can not be removed for at least providing quality education to most people. Just because some student didn't have chance to study in a good school doesn't mean he/she lacks intelligence and aptitude for engineering education.
@Ankur, you can't change the goal post. All along, it has been said that percentile in two boards can be equated and this system is fair. Now, when I point out that the system is substantially unfair, you tell me, "ok, may be, it is unfair, but there are so many other advantages" without articulating any advantage, without giving any data to support that, etc.
This has been my problem with the new system ever since I started writing about it a few months ago. Every supporter is only willing to say that this will help, this has advantages, etc., without any data, without any analysis.
And, please remember, the onus of having data is on someone who proposes or supports change. Here the persons who are opposing the change are bringing out data.
@Rainbow Scientist, We already have 51 percent reservation (including that for Physically challenged persons). Out of this, 27 percent is for those who are financially weaker sections amongst the "other backward castes."
What you are saying is that we are as yet not inclusive enough. To be more inclusive, we must tweak our admission process to encourage poor people in.
Let us accept that argument for the time being. Can you show me any evidence that the proposed system will achieve that.
The only thing that the system is doing is to say that 90 %ile of good board and 90 %ile of bad board are equivalent. But remember, even in the bad board, the top 10% will be mostly those who are relatively better off in that group.
If the goal of this policy was to have a quota for each board (and frankly, this is what a lot of faculty members at IITs are suspecting is behind all this), then let them come up with an explanation as to why this is in the interest of the society.
For a moment let us assume that the law of large numbers IS magically and universally the fairest thing to do.
Has someone done a check that these numbers are large in the first place ?
http://mhrd.gov.in/sites/upload_files/mhrd/files/ExamResults-2008.pdf
Check table 9-
Many boards have less than 50k students appearing for Class 12 - which means number of students appearing for PCM will be in the range 15k-20k or so. There are even boards which have <10k candidates in all. A few boards even have a few hundred candidates. Are these numbers large enough to apply the law of large numbers ?
Also Check Graph 1-
Check the massive difference between the pass percentages in the board exams. Difficulty of various boards might vary but how does one know how much of that variation is coming from the difficulty of the board examination vs the ability of students in that board ? And if the HRD is so convinced that students across boards are equal, shouldn't their first priority be to come down heavily on the boards which are flunking so many students ?
Sorry but unless a proper analysis and report comes out, one cannot help think of this decision as an example of yes-men pleasing their bosses in HRD in a very irresponsible manner.
Let me share some interesting history for all concerned.
There was a time (in not so distant past) when only entrance tests to engineering education was of IITs and Roorkee . Rest of the colleges admitted students on the basis of Class XIIth marks. Gradually, the colleges started facing the challenge of making a merit list as number of students having first division (I'm talking of good ol' days !) steadily went up. Also, the correlation between Board marks and their performance in the college starting coming down.
It was then that the practice of entrance test caught up. It was essentially to do justice to good students (hardworking or intelligent or both).
I believe the Boards have not gotten any better.
Giving weightage to board's marks will again reverse the Kal Chakra .
I agree with Prof Barua that quality is not the preserve of some chosen few. But , when you have doubt about how to 'measure' this quality, isn't it sensible to jettison it and move with some fair and well designed test (which should be continually validated against testing of students' quality).
Can for a moment IIT faculty stop being a gatekeeper and focus on providing best and challenging education to whoever happen to come by (through quota, board or any other policy). The only problem I see with the system is that it is stagnant and imposed from the top as if the faculty can not decide what kind of students they need for providing their branch of education. The whole focus is at the wrong place. Doesn't matter which side you will take in this argument, the focus will always be on some marks (12th board, CBSE, JEE etc etc) which is a poor indicator of success or creativity in any field.
The only statistical anomaly around here is the very existence of IITs in the vast wasteland that is India, for as long as a few decades. Warts and all notwithstanding, IITs have teachers that largely know their stuff, teach dutifully, evaluate exams diligently, and take measures to reduce or eliminate copying. Grades are published on time and cannot be bought for money. There is some grade inflation but still, the GPA/CPI largely reflects the true academic accomplishment of students. Contrast this with the backdrop: a nation where even electrons carry less charge, and protons, less mass than their Swedish or Danish counterparts. Where you need a delivery challan signed in triplicate, stapled to each quark and muon when you transport a bar of steel. Is it any wonder this is a very unstable equilibrium that will end, soon enough, in the dissolution of IITs as we knew them?
@Prashant, Thanks a lot. This is fantastic. I think you may have meant Table 1 of Section 2, which gives pass percentage of regular students in all boards. I had no idea that one school in Banasthali was a board in itself.
And yes, with this data, how can anyone even talk about law of large numbers. But, as we have seen in some other comments, I think the proponents of the new scheme have no interest in a scientific argument. They are very comfortable with "perceptions," "feelings" and political statements like being inclusive, help the poor, improve the school system, and alike.
And the proponents (the Directors) know that as long as the language of debate is in political terms, they will win it (they have already won it actually), but as soon as the language of debate changes to data, facts, analysis, they lose. You may throw whatever data at them, they are not going to change their religion because of you.
IIT directors are not elected by IIT faculty (or students), and therefore, do not represent the primary interests of IITs. It is the only position that is "plugged from the outside". Amid the vast wastelands of India, one would be very naive, then, to be surprised at how IIT directors behave. I am yet to meet an IIT director that I can regard as a respectable peer with who I'd like to hold a conversation. Any conversation.
Prof Sanghi - I just did a quick analysis on this topic. It shows that your intuition of the inequality across different boards for the same percentiles is right; but the relative difficulty of boards can be taken care of easily. Please bear with me while I explain this below.
Suppose X is the innate ability of a student and the marks obtained by him, M_b, in board b are given as M_b=c_bX. c_b is board-level constant, it dictates the ease of getting a mark in board b for a student of unit caliber. Suppose a utopian board exam has perfect mapping of ability to marks. In this exam the marks obtained are M = X. This exam may be thought of as the alternative you are trying to replace, i.e., the JEE.
What we want is (I think) to find a threshold a for the utopian exam and a_b for each board b such that for an randomly chosen student, the probability of getting marks > a in the utopian exam is the same as the probability of getting marks > a_b in a board exam b.
i.e., we want Prob(M > a) = Prob(M_b > a_b). This means Prob(X >a) = Prob(X > a_b/c_b). Therefore the thresholds have to scale according to c_b: that is, a_b has to be chosen so that a_b/c_b = a for all boards. Else we won't have a match, you have pointed out. However, this can be repaired if can find c_b and do the scaling.
But now, how do we find c_b? Here is where the law of large numbers is useful. We need to assume that the average of the abilities of all students is the same for every board. That is, for any board b one can find the average marks obtained by all students (denote as T_b), then this average T_b is proportional to c_b by the same amount for all board.
That is c_b/c_b' = T_b/T_b' for any two board b, b'. Therefore the thresholds a_b and a_b' must be set in proportion to the average marks obtained in each board. So long as this is done, the relative difficulty of one board with respect to another will not matter.
For this to succeed, we need large numbers and student ability to be distributed identically in each board. If the numbers in board are varying vastly, one has to discard this system. What Prashant has pointed out is worrisome - spurious boards with very small numbers will not have representative outcomes. That is not to say they will be at an advantage, because before LLN kicks in the average can be higher or lower than any other board. But it needs to be addressed for fairness sake.
We need to assume that the average of the abilities of all students is the same for every board.
This is exactly what is being discussed is an incorrect assumption to make and needs some kind of third party validation.
eprints.iisc.ernet.in/25027/1/6.pdf
Check out figure 4. That does give you an idea about how the abilities of students are very different, board to board. ICSE students perform much better in this third party assesment, followed by CBSE. That is a board with only 25-30k students in Class 12 and I suspect those students will be severly disadvantaged by this assumption, same will hold for CBSE students.
Prashant, could you (or may be anyone) please explain what 'abilities' or 'innate abilities' of students actually mean? I'm confused. Sorry to interrupt with an odd question in this thread since I'm not an active participant in this JEE/vs.non-JEE discussion although I've been following it mutely.
If infrastructure, teaching quality, regular classes, etc. define 'innate abilities', then I'm fine with Prashant's comment that it is arguable as to why/how we make an assumption of "the average of the abilities of all students is the same for every board". Because, CBSE/ICSE etc. would fare better than say, Assam state board.
If not - that is - if 'innate abilities' refer to a student's intrinsic talent and learning abilities irrespective of infrastructure etc., then it unfortunately has a racially unfavorable tone to it. Because, in that case, if someone says, for example average 'innate abilities' of West Bengal board students are better/higher than say those of Tamil Nadu board (hypothetically speaking), then it would mean Bengali people have higher intrinsic talent than Tamil people, which would be very, very racist !! ..so my question is: can we place the intrinsic talent of a student from state board X at a higher level than a student from state board Y?
Dheeraj: Your hunch that different boards have significant quality differences is correct. There's one study that I know of which has gone into the question of quality of different school boards. It's called 'Quality Education Study' managed jointly by Wipro and Educational Initiatives. I quote item 3 (page 6) of their Executive Summary: 'Significant differences in schools affiliated to different boards and in different cities: The finding from boards-wise comparison reveals that schools from Council for the Indian School Certificate Examinations (CISCE) and Central Board of Secondary Education (CBSE) performed among the top 2 and differences with other Boards were statistically significant. CISCE also performed significantly better than CBSE. The comparison of scores reveals that students from Kolkata and Delhi performed significantly better than Mumbai, Chennai and Bangalore.'
The CBSE and ICSE boards tend to be patronized by wealthier and more powerful sections of society. So, it is indeed possible that the average student in those boards has more ability (due to better circumstances not any innate difference).
As for the KVPY data, the paper did not mention how many students from each board gave the exam. Without that piece of information, it is not possible to assess the validity of the means obtained for the individual boards.
Professor,
Here is a very astonishing article on what happens if a one-day do-or-die examination is in place as it has been proposed:
http://www.ndtv.com/article/world/for-college-exams-chinese-students-use-iv-drips-hormone-injections-228547?pfrom=home-topstories
The assumption that CBSE students will be disadvantaged by percentile system is facile. In fact, all these years there has been systematic discrimination against state board students. Firstly, a large number of CBSE schools are funded by Central Government, having good all round facilities as well paid teachers. Secondly, the AIEEE paper set by NCERT means the question papers are set within the four walls of CBSE sylllabus, teaching pedagogy and question bank framework. As compared to the children born with a silver spoon in CBSE schools, the new system will hopefully give a better level playing field for state board children. In fact, to further level the playing field, exams should be set by a panel of experts drawn from all state and CBSE boards. The papers should also be adminstered in all major languages of India to avoid discrimination.
PS- Before anybody accuses me of bias, I am a product of both state and CBSE boards and know what a huge head start CBSE students enjoy by virtue of their fortunate cirmumstances.
@manu, I am saying that a person with x percentile rank (not percent marks) in CBSE is better academically prepared than a person with same percentile rank in a typical state board. I am giving some evidence of it, which is the performance of these students in various exams. And I am giving some reason for it (state does not fund its schools adequately).
Now, if you disagree with it, you need to counter this evidence, and you need to provide evidence in support of your claim. Otherwise, it is just a political statement.
While I have allowed this comment, I will not allow this blog to be used for political statements in general. If you have evidence, please share.
@Chandresh, the better performance of CBSE students is not just because of similarity in the syllabus. As I said in the blog, even in a state CET, where the state board people are involved in making the question paper, the syllabus is same as state board, still CBSE students perform better.
And obviously, they perform better because the schools are better. Now, instead of you asking the state governments to put in more investments into school education, you are arguing that there be quota for each board.
We already have 27% seats reserved on the basis of social and financial backwardness. We already have a total of 51% reservation.
But, still, if the Directors of IITs want a reservation based on boards, let them come out openly and say so, and we can discuss whether that is a good idea for the nation or not.
The problem in this whole debate is that the "powerful" side just says that it is their belief that this is good, gives no facts, data, analysis, logic, nothing, and then goes ahead and declares that this will be implemented soon. Whatever little they say in terms of a rational argument can be easily proven to be wrong, but it does not matter to them, since they have the power to do whatever they want to do.
Dr. Sanghi, I am sorry but I dont find myself in agreement on the CBSE matter.
As far as I know the mandate of the CBSE is to serve the employees of the Central Government across India. In my own personal experience (in Mumbai), I was not allowed to join a CBSE school because my mother was a state government employee and my father worked in private industry. Also, a large number of IIT alums I have come in contact have parents who are central government employees. Also, one cannot forget the politics of the CBSE board, where English and Hindi are given much more importance than state languages, and the history is very north centric.
Chandresh has also pointed out other issues, importantly the economic one, states cant simply invest more money in education. They rely on the centre for revenues, and dont have anywhere near the economic power the centre has.
We cannot create an unfair system for 95% of the students, because 5% have been given privileged access to better resources.
@Vikram, First of all, this restriction on central government employees is only on central government owned schools. There are a huge number of private schools affiliated with CBSE.
As I have repeatedly said that CBSE is better because its schools are better endowed. Should that be a disqualification. As I have said, we already have 27% reservation based on social and financial backwardness. Please keep that in mind.
And if you give preference to state boards, you will only be giving preference to "better" schools of state boards, and not village schools.
And again, let the Directors make that point. Let the Directors tell us what is the problem for which this is the solution (and show that indeed this is the solution with some data). And if giving preference to state boards is part of a solution to some serious social problem in the country, then I will accept it.
I think my heart bleeds as much for India as the hearts of all these IIT Directors.
1. Let me give you a different set of numbers. The bunching effect of the percentile ranking scheme needs to be kept in mind. Let us assume that the CBSE Board has a perfect co-relation with merit and "quality" and let us assume that all the other Boards will throw up random ranks for its students. There are 10,000 seats in IITs. With about 3 lakh students in CBSE science, the top 10,000 will be in the top 4% of students. Now along with them, the top 4% from the other Boards (assuming a total of 10 lakh students) will be 40,000 students. So these 10,000 CBSE students will get marks close to these other 40,000 random students from other Boards. Now with 50% weightage, the difference in marks between the first and the last in this sample is 2 marks. So, in order to get the better of these 40,000 random guys, the CBSE brigade has to score 4 more marks in the Mains. Since they are the "best", ergo it follows, this is a "no contest"! So what are we talking about?
2. How many of IIT faculty children study in CBSE schools? I will not be far off the mark if I say 98%.
A much abused and stubborn IIT Director.
@Gautam, After I posted it, I have been told of better numbers. CBSE Science is less than 10 percent of all india science students. So, if we want to make sure that 10,000 CBSE students make it to the next round, then the total students selected would have to be more than 1L. And if you further consider that at least some of those CBSE students are interested in alternate careers like medicine, law, economics, or go abroad, etc., then the numbers will have to be even larger, may be around 2L.
Having said that, I would not want to depend on back-of-the-envelope calculations for something that affects the careers of such a large number of students. I want hard data to understand that this is exactly how things will pan out. And that is why my proposal earlier was that we select 4-5 Lakh students in 2013, and as we collect more data we keep reducing this number every year. So we get to 50K or 1L (whatever) in 3-4 years, and by that time your new baby can learn how to walk.
Why should everything be done overnight according to what IIT Directors "feel" without hard data.
And yes, not all IIT faculty members come from super rich backgrounds, and cannot send their kids to Scindia school or other elite schools. We live with Central schools or at best DPS.
Anecdotal evidence in Mumbai suggests that a large number of intrinsically bright state board students just give up on AIEEE and JEE (daunted by its formidable repuation) and settle for the CET. I suspect it is similar in other states as well.
Forcing students to appear for a 'One Country, One Exam' and giving a marginal boost due to the percentile methodology might actually improve quality levels in state board schools, albeit over a period of time. Equally, the implicit statewise quotas in the shortlisting process could turn out to be the thin end of the wedge to politicise or ruin premier higher education institutes. I don't see any evidence to suggest which way the pendulum will swing. Only time will tell.
We must also not forget that this particular blog article is limited to impact of inclusion of percentile ranks and not overall consequences of the new policy or understanding the risks of the situation degenerating into a scenes like a monkey settling a dispute between two cats.
There is another claim floating around which I am very suspicious about. The claim that IIT entrants were not doing well in school board exams in Class XII. This argument has been used to push through the current system. Dr. Sanghi, is there any data for these trends. In my school days about a decade ago, it was almost always the case that the board toppers went to IIT. Most of those who went to IIT were the high scorers in the board exams. Most of the students I met in IIT were at the top of the class in their school days. Both the ICSE and CBSE Class 12 toppers were there in my batch at KGP ( 2002 entry ). This is however, anecdotal information and it will be interesting to see real stats.
Not to say that board exams are not important, but keep some kind of a loose filter ... like the top 25-30 percentile, which can accomodate various forms of noise. Find out the average percentile of the students currently in the IITs then try to calibrate where the new system should cut off people from admission. Surely these computer programs, data generation, data disclosure etc should not be more than 2 days work.
One IIT Director, I think perhaps from IIT Roorkee, in an interview said that the IIT faculty were just getting worried over "anecdotal" tales of problems with board exams.
Do people know that various board exams are conducted within the premises of the schools itself ? I gave both my board exams from my school premises and the Class 12 exam was supervised by someone from our school itself. Why bother providing hi security for the JEE when half your score comes from a test which has none at all ? You might argue these are small things, the problem here is that there are just too many small things to ignore in the current process. And in our usual Indian style no one has done any pro-active thinking. Think of how much of incentive there is to ignore the use of unfair means now.
That same IIT director whom I mentioned above, went on to discuss how the top 10 percentile in CBSE was swept within a range of 2 percent. Should that statement not be ringing a red flag in itself ? I suspect, that if multiple examiners corrected the same subjective board paper, the standard deviation of the recorded scores for the same paper would be well more than 2.
@Professor Barua,
I have read your articles at other places, and if I understand your views correctly: you eventually want learning to take place in schools. I am sure none would disagree with that vision.
However, Dr Sanghi's objection is that we are rushing into implementing this new proposal, he thinks the system is not statistically sound, while you argue that the differences will be so minor that a detailed statistical data analysis is not needed, and back-of-the-envelop calculation is sufficient.
Why can't we achieve your goal in other alternative ways, which may be less controversial.
Include Social Science, History, Geography, analytical writing, aptitude, comprehension etc. in
a common screening test. So that students do focus on all subjects. This in some way will account for how well they did in board exams. Note that this will happen without having to deal with inefficiency of state boards.
As I mentioned earlier that state boards may be very inefficient. If you change the set of graders and grading room temperature :), top 10% may change very significantly ... so your system currently is quite sensitive to that.
Dr Sanghi suspects that IIT Directors have this viewpoint: "State board inefficiency will reduce in near future, if the board marks are taken seriously and there is a spotlight on them."
I would like to confirm if this is the viewpoint you do hold?
Dr Sanghi's objections have some ground as well. But I believe your vision is very noble too. Why bear so much criticism for such a noble goal? Why not make things more transparent and clear publicly?
--Devesh
Prof Sanghi,
The question I am asking here may have already been answered by you in some other posts/comments so please pardon me for my ignorance.
Since they decided to change the JEE, there must be some reasons for doing the same. One reason can be to fix something that is wrong with the current one and the other reason can be to make it even better. In the first case (to fix a wrong), one needs to tell what was that wrong which is being fixed. Did they list the shortcomings in the current JEE and then suggested the new JEE with explanations on how the new JEE fixes those shortcomings one by one. In the second case (to improve), again the items that are being improved need to be listed and the explanations on how the provisions in the new JEE will improve the specific provisions of the old JEE. If this information is available then I would like to have a look.
If the purpose is to genuinely discourage the coaching culture and therefore to choose students who are actually brilliant and not those who are trained to be brilliant (actually trained to score in a test of a particular format), then an exam that tests the intelligence of a student along with his understanding of some subjects needs to be designed (and then improved over the years). There is no need to reinvent the wheel, current JEE must be trying to achieve this to some extent; additionally, we can learn from the other successful systems in the world. I don’t agree with the common argument that by giving weightage to the board exams marks, student will be forced to take the board exams seriously. JEE’s primary objective is to choose the best students from among those who are taking the test and not to fix the secondary education as provided by various boards. Moreover, board exams are not becoming irrelevant just because of the JEE aspirants. Even if there are few lakhs of JEE aspirants every years, there are still many more who do not take JEE and therefore the objective of making students to take boards exams seriously cannot be achieved by the JEE in any case.
If the purpose is to somehow make IITs more inclusive (in addition to reservations, representation from more state boards) then that objective should be stated clearly. The debate then will be different because in that case the objective is not to choose the best candidates by considering the entire nation as one pool but to make that selection state/board wise. Not a good analogy but something similar to Indian Idol auditions where even if a city has 30 signers that are better than all others in the country, they still will not select all of them and will have auditions around the country to pick few from each area to complete the 30 selections. Many will not agree with it but those debates will be different – one about the merit vs social engineering and the other on whether the percentile based system is good enough to achieve the stated goal – why not just reserve seats for different boards to make the representation certain.
-Atul
I can see valuble tax payers money is going to be spent amassing data from all boards, converting them from percent to percentile etc in a very complicated manner. That is certainly quite unnecessary. What is the advantage of using board marks as weightage as compared to board marks as cut off? will it make the students gain more knowledge? In the cut off method it will not create so much stress. also the malpractice if any in board exams will not affect an honest student because all those scoring above 80% will get to attempt the JEE. This new method will ensure that many who have written a brilliant advanced paper will not have their paper corrected due to the abberations in corrections of the Board papers.
@Chandresh, since no one is sure. We should go slow. That is all IITs are asking for. Most IITs have agreed to many aspects of the proposal from 2014 in writing. But as I said earlier, our Directors are agreeing to 2013 purely on political considerations - 2014 is an election year.
I don't know whether to feel amused or sad for the spat between Dheeraj and Sh. Gautam Barua .
Amused because of the interesting kind of argument and counter arguments being thrown around.
Sad (and this is more dominant feeling) that motives have been attributed.
Just because someone's ward studies in CBSE, his/her defence of CBSE should not be attacked on this count. This stratagem is chiefly used by the class which appears most frequently on TV debates.
Agreed, that there appears to be a point in Government's decision, but then quite valid points lie on the other side of argument also.
The faculty members are as much Indian as any other group in India , and also probably have the best knowledge about the admission system, quality of students coming into IIT, and it is they who had some significant contribution to making IIT what it is today. This makes (or should have made!)them an important stakeholder in the admission process (notwithstanding the fact that their wards are CBSEites).
And logic is on their side. Since the need for comparison never arose earlier at such a scale , the relevant data is hard to come by. So , the best thing should have been to dry run first , generate data and then reach on a decision. (However , I personally prefer an independent JEE with a test which ensures to test the innate ability of the students (throughout India)and the test should continually be validated by IIT itself to evolve into first rate admission test, but that's beside the point)
IIT prides itself to be hardcore analytical and if we bring our 'feelings' into debate , we shall not be doing justice to the very reputation of the institution, dignity and standard of which we all are trying to maintain and known worldwide.
Why is there any compromise in the stand that board marks should be used only as cut off and not be given weightage? First let the HRD has to prove with figures what advantage is there by doing this. Only then it makes sense to dive into time wasting calculations. What disadvantage is there in simply having Board marks as cut off only? Then it will not matter which board one is from.
@Prashant, I did collect some data 5-6 years ago, and it showed that most students perform well in boards as well. I don't have exact information, but that can be collected very easily. Prof. Barua was JAB Chairman a few years ago, and would have this information for that year. But believe me, if that information was supporting their argument, we would have seen the numbers by now.
@AK, The Ramasami Committee report does talk about some of the problems, which includes coaching culture, stress due to multiple exams, and lack of focus on school education by students. There is absolutely no data, facts, or analysis, as to how the new system will solve any of these problems. In fact, most of us believe that it will worsen the problems in the country.
IITs are still willing to give a try to these new ideas, despite strong reservations, if the changes are brought in slowly. After all when we don't understand the system well, we need to do experiments. What we are against is this massive change with only a feeling of a few big-wigs to support the change.
If 2014 is an election year, so are we allowed to make quick decisions with unknown impact before that?.
If there are no flaws in current format why we can't wait for a year or more, else point out the flaws rather than being revolutionary!
Prof. Sanghi, Best of luck for the Senate Meeting today, the future of IIT Kanpur seems to be in your hands now.
Please don't let these politicians (the Directors) tamper the IIT System for their own personal interests.
Here is the 'Quality Education Survey' mentioned in the comment thread above.
Quality Education Survey
It does back the argument that abilities across boards are different and the differences are statistically significant.
Significant differences in schools affiliated to different boards and in different cities: The findings from
board-wise comparison reveals that schools from Council for the Indian School Certificate Examinations
(CISCE) and Central Board of Secondary Education (CBSE) performed among the top 2 and the differences
with other boards were statistically significant. CISCE also performed significantly better than CBSE. The
comparison of scores reveals that students from Kolkata and Delhi performed signicantly better than Mumbai,
Chennai and Bangalore. Mumbai performed at par with overall average of these 5 metros. Bangalore
performed significantly lower than other metro cities. Kolkata had a large number of ICSE schools whil Delhi
had mostly CBSE schools which could be the reason for their higher performance.
Agreed these reflect acquired abilities and not necessarily innate ones, but if innate abilities are all that matter, perhaps just one flat IQ test would be the fairest thing to have, right.
I find prof. Singhi argument more convincing because it is supported by proper data,more and more data pouring in, which supports his argument,however counter argument offered loose its significance as it mere based on presumptions.
only 30 thousand icse class 12 science stream students gave board examination this year they will get severely hit by the percentile system despite of their better performance than state boards, in uttar pradesh board almost 26lakhs students gave examination this year.
Great job, Prof, I've been following your blog for months, but have posted only today.
I've started requesting parents of XII standard students to follow your blog.
Regards
Rama
@Dheeraj Sanghi, I personally feel that even the present IIT/JEE system is far from being perfect.
1. "I am saying that a person with x percentile rank (not percent marks) in CBSE is better academically prepared than a person with same
percentile rank in a typical state board."
I think the goal of JEE should be to select best brains with academic tenacity and interest for the program, instead of just academic preparation in JEE syllabus(P-C-M).
There are students who were exceptional in physics/math/cs but couldn't clear chemistry cut-off.
For CSE students why do we need chemistry cut-off? I did not feel too much academic preparation in physics/chemistry was needed for a CS B.Tech.
Similarly, I know a student really exceptional in chemistry/bio and comparatively average in math/physics. He was really passionate about biotech but missed it by few ranks because of his total. JEE doesnot take students aptitude for that particular program into consideration.
That is why I think department allocation should be done at the end of 1st year when students get a feel about the engineering subjects and programs.
2. I joined KGP CS from a state board, vernacular medium school. I found most of the students were from CBSE/ISC, majority were from cities
and had coaching at Hyd/Kota/Delhi/Chandigarh.
No doubt they were well-prepared for JEE, but were all of them the best bet for research/top technical jobs? NO!
There were arrogant students who were burnt out after JEE and still used to think they were superior than any non-IITians.
When I joined my PhD program in a top-tier US university I found many students from Jadavpur University/Anna University/BITS Pilani/NITs
were more bright and diligent than many IITians. These students were not able to clear JEE because of the syllabus-difference(with the state board),
medium problem(IIT question papers were in english and many of these students studied in vernacular medium) and weren't fortunate enough to get good
coaching or live in big cities. However they had raw talent and perseverance which finally helped them to do better. Whereas many IITians faded away after one exam.
3. Apart from top students in IIT, I have seen, in general, a correlation between boards results and GPA. I dont know whether doing good in boards has something to do with tenacity or not.
It is completely a personal observation, may be you could support this with your experience in IITK.
4."After I posted it, I have been told of better numbers. CBSE Science is less than 10 percent of all india science students. So, if we want to make sure that 10,000 CBSE students make it to the next round, then the total students selected would have to be more than 1L. And if you further consider that at least some of those CBSE students are interested in alternate careers like medicine, law, economics, or go abroad, etc., then the numbers will have to be even larger, may be around 2L."
This is not logical as you are assuming only CBSE science students go for medicine,econ etc. and not the state boards students. :)
@kingkhan, I am absolutely in agreement with you that the current system has serious flaws. In fact, if you look at my older blogs from 2-3 years ago, I have pointed out many flaws.
The interesting thing is that I have, on many occasions in the last five years, written to IIT Directors about proposed changes, and most often the feedback that I have received is, "your ideas are good, but JEE process is a political decision, and I don't think I can convince the Minister of your ideas."
So, we have been living with JEE being a political decision and a political process for a long time. But the current changes are such that we were not willing to let the political process decide admissions any more.
@Prof. Sanghi: Sir, I earnestly applaud for your efforts and wish you luck as well!
However, I am amazed to see the whole issue, which now looks like a comedy of sorts. There is a group which is pursuing the most insincere/vicious schemes in a very sincere manner and there is another group comprising of sincere people vehemently opposing this insincerity - the key point being that at least the Machiavellian attitude is what everyone is sure of, including the "Machiavellians", and this is what we need to precisely resist. The only weapon in this battle is however that of political deviousness. Logic, rationales and intellect have taken a back seat, and for the first time I believe we are going to witness a battle of its kind, the world has never witnessed before.
Post a Comment