“Unscientific” practices of Educational Boards: why tolerate?

Image Courtesy: Vishal Bhatt: www.facebook.com/wizard.vishal

Most of the educational boards in India conduct examinations at the end of 10^th and 12^th grades. The 10^th grade result plays a big role in narrowing down student’s choice of academic discipline for further studies (e.g., liberal-arts, commerce, or science-stream); whereas the 12^th grade result serves as a criterion (at times, a sole criterion) for the choice of career as well as admission to the higher education institution. Over the years, the board examination system has changed at administrative level with things happening in a more efficient way. However, things were poor, are poor, and are on a path towards poorness when it comes to scientific assessment. Here’s why:

The state boards have never published any of the psychometric properties (e.g., reliability, validity, item discrimination, & difficulty value) of their tests. Perhaps nobody has ever cared for asking for it. Today, through this article, I would like to raise this vital issue and draw your attention to a mass-deception exercise in the form of the “Board Exams” that decides careers of our youth.  

Ten big points for improving the Board Examination:

1.                  Improving Reliability and Validity of Test Items

All tests have these two basic characteristics:

·         Validity: Measure the concepts that we want to measure (i.e., A weighting-machine should measure weight, not any other property).  

·         Reliability: Scores should be consistent for the same competency level of student (i.e., A weighting-machine should show same weight every time we step on it, as long as our weight is constant).

The State Board Examination papers often have test items that are likely to have significantly poor reliability and validity. Moreover, some test items simply measure the memorization of the opinion (and not scientific facts) of text-book authors that has nothing to do with the subject content. Let’s have a look at the following test items, which are taken from Question Bank-2008 of Gujarat Secondary Education Board (Source- http://203.77.200.35/gseb/question-paper.htm).

Towards what Indians have developed the feeling of equality? (Std. 10- Social Science)

   A)     Huge living beings

   B)     Nature

   C)    All living beings

   D)    Animals

Which one is the most valuable asset to human being? (Std. 12- Psychology)

   A)     Money

   B)     Strength

   C)    Mind

   D)    Language

What could be the rationale for above test items? Perhaps, “how well do you remember personal (unscientific) opinions of our authors?” The test maker should check validity and reliability for tests and provide rationale for each test item.

2.      Diminishing Excessive Focus on Lower Cognitive Level

In this era of globalization, where our youth will compete with the best of the world, we got to encourage critical and creative thinking and problem solving skills. Research shows that the teaching methodology of teachers and the learning methods of students greatly depend on the type of final examination (type and quality of questions). Thus, if the majority of our students are choosing rote memorization and superficial learning strategies to crack the State Board examination, something must be wrong with the type of questions being asked. The board exams focus excessively on knowledge and understanding levels of cognition (as per Bloom’s Taxonomy). The students rarely get opportunities to exhibit their in-depth learning driven by mastery oriented goals. Our test items should measure higher levels of cognition as well (i.e., application, analysis, synthesis, evaluation).

Let’s look at the following question taken from March-2010 10^th grade social science test-item.

What is Environmental Degradation? Mention the measures to control Environmental Degradation. (5 marks)

This question is at knowledge-level and requires only good memorization skills to get a full score. How about questioning in the following manner-

Based on your knowledge of social science text book, prepare a detailed plan of action to cope up with the Environmental Degradation in your home town/village/city. (5 marks) 

The above question will demand insightful learning of environmental degradation; and will measure higher levels of cognition. Of course, this will also demand better teaching from teachers. Such test items have potential to break the cycle of poor teaching→ poor learning → poor testing; and encourage students to study for mastering the subject, rather than remembering the content. They will also encourage teachers to facilitate learning, and to critically analyze, evaluate, and discuss issues in the classroom. 

3.      Preparing Tests that Distinguish Students

In the present scenario, the test items are not designed so that they distinguish between a mediocre student, a poor student, and a brilliant student (i.e., discrimination parameter in technical terms). This requires creation of large item pool and thorough item analyses of each item.

4.      Making Difficulty Level Uniform

Many times test paper of one subject is more difficult than that of other subjects. It is a mystery if the difficulty level of a test is analyzed by the test-developers. If the difficulty level of tests of optional subjects is different, it will be unscientific to compare the total scores across exams of these subjects. For example- student “A” has scored 80 in Sanskrit-test, which has lower difficulty level, whereas student “B” has scored 80 in Hindi-test, which has high difficulty level. In such case, the score of 80 in Hindi has higher value than that of in Sanskrit. In current practice, these technicalities are completely ignored.

5.      Rationalizing Time Limits

Many times, some exam-papers are lengthy and some are not. And at times, for two students of same competency level, the difference in their result is determined by their writing speed. Now, should the measurement of writing speed be an objective of the state board examination? We ought to devise the length of exam rationally and uniformly. Do we want to give our students time to critically think, to evaluate, to judge, to analyze, and to apply their knowledge? Or we simply want them to vomit out all rote-learning of the previous night? Have the test developers considered the speeededness factor in analyses?

6.      Establishing Uniform Test Administration

Though tests are administered simultaneously in the entire state, the testing conditions are not uniform. The supervisors are not trained and do not have standardized code of conduct. Thus, it is likely that the students in different exam-halls interact differently with the supervisor. Some supervisor may be proactive and helpful, while others may be sluggish and arrogant. This can severely affect the morale of the test taker. The conditions in which students take examination ought to be standardized. Also, environmental factors (like- temperature, wind/rain), physical distance between students, quality of writing desks, noise level, and supervisor behavior should be controlled uniformly across the state. Thus, someone who takes exam in a private school in Ahmedabad has the same exam-experience as someone writing exam in a remote village of Dang.  In addition, we need to control systematic cheating that takes place in many exam-centers. No student should get “center-advantage” or “center-disadvantage”.

7.      Eliminating Subjective Evaluation and Improving Transparency

Many teachers/ parents/ students/ educationalists keep questioning the credibility of the board examination. Subjective evaluation is one of the major factors hurting the credibility of the test. This is related with the inter-rater reliability of test. The test score should be consistent even if different examiner evaluates the test. We got to study the inter rater reliability of these exams. At the very least, inter-rater reliability coefficient has to be greater than 0.7, given that the stakes are high for students.  We cannot allow mood/ personality/or personal views of examiners to determine the careers of our future citizens.

In addition, the examination system has to be more transparent. The students should have an opportunity to know why their score is cut, and what information is missing. They should also get an opportunity to see the answer-key. Of course, there may not be one right response for an essay type question but a model response should be shared with the students so that they can work on their deficiencies. This will also help teachers to have deeper understanding of what objectives they should consider while teaching.

8.      Accommodating Students with Special needs

Most of the educational boards do not provide information on how they accommodate children with special needs except for blind, or speech/hearing impaired. There is a wide range of children that require support. For example- learning-disabled, emotionally disturbed, autistic, schizophrenic, & intellectually challenged etc. When will we acknowledge that formally in educational system?

9.      Improving Predictive Value

If the result of an examination is to be considered for admission or employment purpose, then such examination should have proven predictive value. Though its results are used for admissions and employment, the board exam is aimed only to measure the present competency of student. It does not predict the future success of student in respective subjects. In simple terms, score of 100 in Physics does not predict that the student will succeed in Physics courses at the University level. Now, if we want the result of this examination to be considered for university admissions/ employment, we may include subject-wise aptitude test like SAT (Scholastic Aptitude Test) in the US. With help of experts, the board can generate tests of high predictive value. The result of such tests may provide important additional information to the universities/colleges/employers about the natural proclivity of the student towards the subject.

10.  Gradually move on to Computerized Adaptive Testing (CAT)

In this era of IT revolution (something that India is proud of), we can no longer carry on laborious  expensive and exhaustive ways of large scale assessment. The boards should devise concrete action plans to move on to CAT. Of course, this will take time (to self-educate, to educate educational administrators/teachers/students/parents and masses and to generate resources). But, there are great advantages of CAT as mentioned below:

·         Facilitating convenience of Students/ Parents

·         Standardize test-administration, higher credibility and scientific testing

·         Better control on systematic cheating

·         Greatly encourage computer literacy into masses

·         Development of testing industry in India

In technical terms, the present practices represent a crude and a bit distorted version of classical testing theory. Research in the field of educational measurement suggests that item response theory (IRT) is superior indeed. Many developed nations have moved on to more scientifically rigorous ways of assessing educational outcomes through IRT. It is a high time for Indian states to begin reforms in this direction as well; else they will be out-educated by developed nations with a great margin. There are endless reasons why India has continued these practices. I am simply not going to dig that up as that will be nothing more than a blame game. But, it is clear; I cannot tolerate the current status quo on this front, can you?