summary chapter 3 from book language assessment principles and classroom practices

CHAPTER 3 : Designing classroom language tests

TEST TYPES

The first task you will face in designing a test for your students is to determine the purpose for the test. Defining your purpose will help you choose the right kind of test, and it will also help you to focus on the specific objectives of the test. We will look first at two test types that you will probably not have many opportunities to create as a classroom teacher-language aptitude tests and language proficiency tests and three types that you will almost certainly need to create-placement tests, diagnostic tests, and achievement tests,

1.      Language Aptitude Tests

One t e type of test-although admittedly not a very common one-predicts a person's success prior to exposure to the second language. A language aptitude test is designed to measure capacity or general ability to learn a foreign language and ulti mate success in that undertaking Language aptitude tests are ostensibly designed to apply to the classroom learning of any language. Two standardized aptitude tests have been used in the United States: the Modern Language Aptitude Test (MLAD (Carroll & Sapon, 1958) and the Pasteur Language Aptitude Battery (PLAB) (Pimsleur, 1966). Both are English language tests and require students to perform a number of language-related tasks. The MLAT for example, consists of five different tasks.

2.      Proficiency tests

A proficiency test is not limited to any one course, curriculum, or single skill in the language; rather, it tests overall ability. Proficiency test have traditionally consisted of standardized multiple choice items in grammar, vocabulary, reading comprehension, and aural comprehension. A typical example of a standardized proficiency test is the Test of English as a Foreign Language (TOEFL") produced by the Educational Testing Service. The TOEFL is used by more than a thousand institutions of higher education in the United States as an indicator of a prospective student's ability to undertake academic work in an English-speaking milicu, The TOEFL consists of sections on listening comprehension, Structure (or grammatical accuracy).reading comprehension, and written expression.

3.      Placement tests

Certain proficiency tests can act in the role of placement tests, the purpose of which is to place a student into a particular level or section of a language curriculum or school. A placement test usually, but not always, includes a sampling of the material to be covered in the various courses in a curriculum; a student's performance on the test should indicate the point at which the student will find material neither too easy nor too difficult but appropriately challenging. Placement tests come in many varieties: assessing comprehension and production, responding through written and oral performance, open-ended and limited responses, selection (e.g., multiple-choice) and gap-filling formats.

4.      Diagnostic tests

A diagnostic test is designed to diagnose specified aspects of a language. A test in pronunciation, for example, might diagnose the phonological features of English that are difficult for learners and should therefore become part of a curriculum Usually. such tests offer a checklist of features for the administrator (often the teacher) to use in pinpointing difficulties. A writing diagnostic would elicit a writing sample from students that would allow the teacher to identify those rhetorical and linguistic features on which the course needed to focus special attention.

5.      Achievement tests

An achievement test is related directly to classroom lessons, units, or even a total curriculum. Achievement tests are (or should be) limited to particular material addressed in a curriculum within a particular time frame and are offered after a course has focused on the objectives in question- Achievement tests can also serve the diagnostic role of indicating what a student needs to continue to work on in the future, but the primary role of an achievement test is to determine whether course objectives have been met—and appropriate knowledge and skills acquired—by the end of a period of instruction. Achievement tests are often summative because they are administered at the end of a unit or term of study. They also play an important formative role. An effective achievement test will offer washback about the quality of a learner's performance in subsets of the unit or course. This washback contributes to the formative nature of such tests.

SOME PRACTICAL STEPS TO TEST CONSTRUCTION

1.      Assessing Clear, Unambiguous Objectives

In addition to knowing the purpose of the test you're creating, you need to know as specifically as possible what it is you want to test. Sometimes teachers give tests simply because it's Friday of the third week of the course, and after hasty glances at the chapter(s) covered during those three weeks, they dash off some test items so that students will have something to do during the class. This is no way to approach a test. Instead, begin by taking a careful look at everything that you think your students should "know" or be able to "do." based on the material that the students are responsible for. In other words, examine the objectives for the unit you are testing.

2.      Drawing Up Test Specifications

Test specifications for classroom use can be a simple and practical outline of your test. (a) a broad outline of the test, (b) what skills you will test, and (c) what the items will look like. Let's look at the first two in relation to the midterm unit assessment already referred to above. (a) Outline of the test and (b) skills to be included. Because of the constraints of your curriculum, your unit test must take no more than 30 minutes. This is an integrated curriculum. so you need to test all four skills. Since you have the luxury of teaching a small class (only 12 students!), you decide to include an oral production component in the preceding period (taking students one by one into a separate room while the rest of the class reviews the unit individually and completes workbook exercises). You can therefore test oral production objectives directly at that time. You determine that the 30-minute test will be divided equally in time among listening, reading. and writing. (c) Item types and tasks. The next and potentially more complex choices involve the item types and tasks to use in this test. It is surprising that there are a limited number of modes of eliciting responses (that is, prompting) and of responding on tests of any kind. Consider the options: the test prompt can be oral (student listens) or written (student reads), and the student can respond orally or in writing.

3.      Devising Test Tasks

Ideally, you would try out all your tests on students not in your class before actually administering the tests. But in our daily classroom teaching, the tryout phase is almost impossible. Alternatively, you could enlist the aid of a colleague to look over your test. And so you must do what you can to bring to your students an instrument that is, to the best of your ability, practical and reliable.

In the final revision of your test. imagine that you are a student taking the test. Go through each set of directions and al] items slowly and deliberately. Time yourself. (Often we underestimate the time students will need to complete a test.) If the test should be shortened or lengthened. make the necessary adjustments. Make sure your test is neat and uncluttered on the page, reflecting all the care and precision you have put into its construction. If there is an audio component, as there is in our hypothetical test, make sure that the script is clear, that your voice and any other voices are clear, and that the audio equipment is in working order before starting the test.

4.      Designing Multiple-Choice Test Items

Multiple-choice items, which may appear to be the simplest kind of item to construct, are extremely difficult to design correctly. cautions against a number of weaknesses of multiple-choice items:

•     The technique tests only recognition knowledge.

•     Guessing may have a considerable effect on test scores.

•     The technique severely restricts what can be tested,

•     It is very difficult to write successful items.

•     Washback may be harmful.

•     Cheating may be facilitated.

multiple-choice items offer overworked teachers the tempting possibility of an easy and consistent process of scoring and grading But is the preparation phase worth the effort? Sometimes it is, but you might spend even more time designing such items than you save in grading the test. Of course, if your objective is to design a large-scale standardized test for repeated administrations, then a multiple-choice format does indeed become viable. First, a primer on terminology.

1)      Multiple-choice items are all receptive, or selective, response items in that the test-taker chooses from a set of responses (commonly called a supply type of response) rather than creating a response. Other receptive item types include true-false questions and matching lists. (In the discussion here, the guidelines apply primarily to multiple-choice item types and not necessarily to other receptive types.)

2)      Every multiple-choice item has a stem, which presents a stimulus. and several (usually between three and five) options or alternatives to choose from.

3)      One of those options, the key, is the correct response, while the others serve as distractors

SCORING, GRADING, AND GIVING FEEDBACK

1.      Scoring

As you design a classroom test, you must consider how the test will be scored and graded. Your scoring plan reflects the relative weight that you place on each section and items in each section. The integrated-skills class that we have been using as an example focuses on listening and speaking skills with some attention to reading and writing. Three of your nine objectives target reading and writing skills.

Because oral production is a driving force in your overall objectives, you decide to place more weight on the speaking (oral interview) section than on the other three sections. Five minutes is actually a long time to spend in a one-on-one situation with a student, and some significant information can be extracted from such a session. You therefore designate 40 percent of the grade to the oral interview. You consider the listening and reading sections to be equally important, but each of them, especially in this multiple-choice format, is of less consequence than the oral interview. So you give each of them a 20 percent weight. That leaves 20 percent for the writing section, which seems about right to you given the time and focus on writing in this unit of the course.

2.      Grading

Your first thought might be that assigning grades to student performance on this test would be easy. just give an "A" for 90-100 percent, a "B" for 80-89 percent, and so on. Not so fast! Grading is such a thorny issue that all of Chapter 11 is devoted to the topie How you assign letter grades to this test is a product of

•       the country, culture. and context of this English classroom,

•       institutional expectations (most of them unwritten),

•       explicit and implicit definitions of grades that you have set forth.

•       the relationship you have established with this class, and

•    student expectations that have been engendered in previous tests and quizzes in this class.

3.      Giving  feedback

A section on scoring and grading would not be complete without some consideration of the forms in which you will offer feedback to your students, feedback that you want to become beneficial washback. In the example test that we have been referring to here—which is not unusual in the universe of possible formats for periodic.

Reference:
Brown, H. Douglas. 2004 . language assessment: princple and classroom practices. New York: Pearson Education.