Language Testing Assessment Summary

 

Informal and Formal Assessment

Informal Assessment = Classroom tasks designed to elicit performance without recording results and making fixed judgments about a student’s competence. = tidakmenggambarkan competence keseluruhan.

Informal assessment can take a number of forms, starting with incidental, unplanned comments and responses, along with coaching and other impromptu feedback to the student.

Formal Assessment = Exercise or procedures specifically designed to tap into a storehouse of skills and knowledge.

Formal assessments are exercises or procedures specifically designed to tap into a storehouse of skills and knowledge.  Is formal assessment the same as a test? We can say that all tests are formal assessments, but not all formal assessment is testing.

 

Formative and Summative Assessment

Formative assessment: evaluating students in the process of "forming" their competencies and skills with the goal of helping them to continue that growth process. The key to such formation is the delivery (by the teacher) and internalization (by the student) of appropriate feedback on· performance, with an eye toward the future continuation (or formation) of learning.

Summative assessment aims to measure, or summarize, what a student has grasped, and typically occurs at the end of a course or unit of instruction. A summation of what a student has learned implies looking back and taking stock of how well that student has accomplished objectives, but does not necessarily point the way to future progress. Final exams in a course and general proficiency exams' are examples of summative assessment.

 

 

Norm-Referenced and Criterion-Referenced Tests

In norm-referenced tests, each test-taker's score is interpreted in relation to a mean (average score), median (middle score), standard t-/ deviation (extent of variance in scores), and/or percentile rank. The purpose in such tests is to place test-takers along a mathematical continuum in rank order. Typical of norm-referenced tests are standardized tests like, the Scholastic Aptitude Test. (SAT) or the Test of English as a Foreign Language (TOEFL_ intended to be administered to large audiences, with results efficiently disseminated to test-takers. Such tests must have fixed, predetermined responses in a format that can be scored quickly at minimum expense. Money and efficiency are primary concerns in these tests.

Norm-referenced examples

 

Criterion-referenced tests, on the other hand, are designed to give test-takers feedback usually in, the form of grades, on specific course or lesson objectives. Classroom tests involving the students in only one class, and connected to a curriculum, are typical of criterion-referenced testing. Example:

Score	Criteria
5	Very fluent; Speak without intonation; No Grammatical error; Native-like
4
3

 

 

 

Discrete-Point and Integrative Testing

Discrete-point tests are constructed on the assumption that language can be broken down into its component parts and that those parts can be tested successfully. It was claimed that an overall language proficiency test, then, should sample all four skills and as many linguistic discrete points as possible.

Discrete-point testing = mentest/menilai students di 1 part tersendirisajadari 1 skill / seluruh skill (example: speaking: Pronunciation)

Integrative Testing = there are more than one skill to be required to do one test

Language should be tested integratively not discretely

 

Direct test vs Indirect Test

Indirect test

To know how much the students know about language

Contoh:

Mengukurkemampuansiswamelalui written monolog/dialog ygdikumpul

Direct test = students are test directly what we want to measure. Contoh:

In the term of validity direct test is more valid and reliable than indirect test

 

Objective and Subjective Test

Objective test

No judgment is required on the part of the scorer e.g. multiple choice items, True/False items, short answers, etc.

Subjective Test

Judgment is called for scoring, e.g. essay items, paragraph writing, delivering speech, acting out a dialog, etc.

 

Test Types:

Proficiency test

It is designed to measure people’s ability in a language regardless of any training they may have had in that language, e.g. TOEFL. IELTS, TOEIC.

Placement Test

It is designed to provide information that might help to place students at the stage of the teaching program most appropriate to their abilities.

Language Aptitude Test

It is designed to measure capacity or general ability to learn a foreign language and ultimate success in that undertaking language.

Diagnostic Tests

A diagnostic test is designed to diagnose specified aspects of a language or to identify learners’ strength and weakness.

Achievement Tests

It is designed to establish how successful individual students, group of students, or the cources themselves have been in achieving objectives.

 

CHAPTER 2: Principles of Language Testing

An effective test has:

Practically, Reliability, Validity, Authenticity, and Washback.

A.    Practically, a test:

·         Is not excessively expensive

·         Stays within appropriate time constrains

·         Is relatively easy to administer

·         Has a scoring/evaluation procedure that is specific and time-efficient

B.     Reliability

A test is reliable if it yields consistent results when repeated over time.

To measure students’ proficiency

It is consistent and dependable.

Factors that influencing reliability:

·         Student-related reliability

The most common learner-related issue in reliability is caused by temporary illness, fatigue, a "bad day," anxiety, and other physical andpsychological factors. Also included in this category are such factors as a test-taker's "test-wiseness" or strategies for efficient test taking.

2 factors: Psychological factors and physical factors.

·         Rater reliability

There are 2 rates:

1.      Inter-rater reliability occurs when two or more scorers yield it inconsistent scores of the same test, possibly for lack of attention to scoring criteria, inexperience, inattention, or even preconceived biases.

2.      Intra-rater reliability is a common occurrence for classroom teachers because of unclear scoring criteria, fatigue, bias toward particular "good" and "bad" students, or simple carelessness.

Those can be because of lack of attention to scoring criteria, inexperience, inattention, preconceived bias.

·         Test administration reliability

The manner in which the examiner presents, the instructions, the characteristics of the room, outside, noises, etc.

·         Test reliability

The sampling of the objective, the degree of ambiguity of items, restriction on freedom of response, the clarity of instructions, the quality of layout, the familiarity with the formal, the length of the test.

The nature of the test itself can cause measurement errors. If a test is too long, test-takers may become fatigued by the time they reach the later items and hastily respond incorrectly. Timed tests may discriminate against students who do not perform well on a test with a time limit.

C.     Validity

·         Kesesuaian antara alat ukur dengan apa yang diukur

·         Is the degree in which the test measures accurately in what it is intended to measure (Hughes; 2003)

·         The extent to which interference made from assessment results are appropriate, meaningful, and useful in terms of purpose of the assessment. (Gronlund, 1998)

·         The extent to which it measures what it is supposed to measure and nothing else. (Heaton, 1988)

·          A test must be valid because

·         Apa yang kita nilai sesuai dengan apa yang kita ajarkan yang sesuai dengan learning objective.

·         Untuk memberikan keadilan untuk siswa, bahwa siswa tersebut diukur sesuai dengan kemampuan mereka

Ø  Content-Related Evidence (Content Validity)

·         Alat ukur yang sesuai dengan materi yang diajarkan

·         A test is said to have content validity if its content constitutes a representative sample of the language skills, structures, etc. which it is meant to be concerned

·         The course objective ó the test items

·         Content validity: tertera di kurikulumdansudahdiajarkan, jika 2 halitusudahada, barudiadakan test

·         You can usually identify content-related evidence observationally if you can clearly define the achievement that you are measuring

·         Another way of understanding content validity is to consider the difference between direct and indirect testing

·         A test would have content validity only if it included proper sample of the relevant structures of the purpose of the test.

·         The importance of content validity:

1.      The greater a test’s content validity, the more likely it is to be an accurate measure of what it is supposed to measure

2.      Such a test is likely to have a harmful backwash effect

Ø  Criterion-Related Evidence (Criterion-Related Validity)

·         The degree to which results on the test agree with those provided by some independent and highly dependable assessment of the candidate’s ability.

·         Kesesuaianantarasebuahhasiltesdenganhasilteslainnyameskipuntesnyatidaksama

·         In the case of teacher-made classroom assessments, criterion-related evidence is best demonstrated through a comparison of results of an assessment with results of some other measure of the same criterion.

·         2 kinds of criterion-related validity:

1.      Concurrent Validity

Is established when the test and the criterion are administered at the same time. E.g. achievement testing. 2 test yang diberikan yang berjarakwaktu yang sebentar

2.      Predictive Validity

This concerns the degree to which a test can predict candidates’ future performance. E.g. Placement testing. Bias di lihatsaatbelajarmengajar

·         A test has concurrent validity if its results are supported by other concurrent performance beyond the assessment itself.

·         The predictive validity of an assessment becomes important in the case of placement tests, admissions assessment batteries, language aptitude tests, and the like.

·         The predictive validity= diprediksi di awaldanharus survive with the prediction.

·         The predictive validity= dikorelasikanantarahasil placement test dan achievement test.

Ø  Construct-Related Evidence

·         Kesesuaian tes dengan theoretical background mengenai unsur-unsur yang membangun skill yang ingin diuji

·         A test is said to have construct validity if it can be demonstrated that it measures just the ability which it is supposed to measure.

·         Construct = any underlying ability/trait that is hypothesized in a theory of language ability.

·         Example: oral proficiency includes pronunciation, fluency, grammatical accuracy, vocabulary use, and sociolinguistic appropriateness

·         Construct validation is a research activity, the means by which theories are put to the test and are confirmed, modified, or abandoned. It is through construct validation that language testing can be put on a sounder, more scientific footing.

Ø  Consequential Validity

Consequential validity encompasses all the consequences of a test, including such considerations as its accuracy in measuring intended criteria, its. impactonthe preparation of test-takers, its effect on the learner, and the (intended and unintended) social consequences of a test's interpretation and use.

Ø  Face Validity

·         Kesesuaian antara tampilan test dengan apa tujuan testnya.

·         Face validity refers to the degree to which a test looks right, and appears to measure the knowledge or abilities it claims to measure, based on the subjective judgment of the examinees who take it, the administrative personnel who decide on its use, and other psychometrically unsophisticated observers (Mousavi, 2002, p. 244).

·         Face validity will likely be high if learners encounter:

1.      A well-constructed, expected format with familiar tasks,

2.      A test that is clearly doable within the allotted time limit,

3.      Items that are clear and uncomplicated,

4.      Directions that are crystal clear,

5.      Tasks that relate to their course work (content validity), and

6.      A difficulty level that presents a reasonable challenge.

·         Face validity is not something that can be empirically tested by a teacher or even by a testing expert. It is purely a factor of the "eye of the beholder" - how the test-taker, or possibly the test giver, intuitively perceives the instrument.

·         Validity in scoring = if a test is have to validity, the way in which the responses are scored must be valid too.

·         A test is said to have face validity if it looks as if it measures what it is supposed to measure.

D.    Authenticity

·         Authenticity, a concept that is a little slippery to define, especially within the art and science of evaluating and designing tests.

·         Authenticity is the degree of correspondence of the characteristics of a given language test task to the features of a target language task (Bachman and Palmer 1996)

·         In a test, authenticity may be present in the following ways:

1.      The language in the test is as natural as possible.

2.      Items are contextualized rather than isolated.

3.      Topics are meaningful (relevant, interesting) for the learner.

4.      Some thematic organization to items is provided, such as through a story line or episode.

5.      Tasks represent, or closely approximate, real-world tasks.

 

E.     WashBack

Washback generally refers to the effects the tests have on instruction in terms of how students prepare for the test.

Reading

Interactive reading: menekankan pada makna, top down (general ke specific), minimum dua paragraf.

Cloze task

mengisi kekosongan dalam paragraf.

C-test procedure (rumpang di dalam kata)

Cloze elide procedure(menghilangkan kata yg tidak perlu)

Impromptu reading plus coprehension question: Jenis teks tidak diketahui, lebih kepada cara menjawab. ex: reading text in toefl test

Interactive reading short answer tasks

Interactive Reading Editing

Memahami seluruh teks.

Interactive reading scanning

Menscan dan menemukan informasi. Timing bisa jadi penilaian.

Interactive Reading – ordering task

Menyususn kalimat acak

Interactive reading – information transfer

Test taker diminta untuk membaca grafik dan diminta untuk mengintrepretasikannya. Make inferences overly.

Extensive reading

Top down extensive reading: memiliki background knowledge terlebih dahulu

Penilaian informal: response, comment.

Point penilaian harus point penilaian reading.

Skimming task

Proses membaca cepat untuk menentukan inti atau gagasan utama.

Penilaian: informal, formative (berkelanjutan).

Skimming: proses, Scanning: strategy.  Untuk mecari informasi tertentu..

Note taking: Mengorgasir semua materi dari awal sampai akhir, siswa membuat catatan yang dianggap penting.

 

Based on question and Answer session.

Agus=Dalam interactive reading itu bisa lebih spesifik dan bisa di nilai, beda dengan extensive reading yang mencakup banyak penilaian.(Annisa NK)

Rania=make inference that are not presented orally, maksudnya adalah informasi yang didapat sesuai dengan data yang ditampilkan (Marwah),

information transfer  itu adalah bagaimana siswa dapat mem     (uci)

Interactive reading: combine form – meaning focus (fokus makna) general to specific (top-down).

Cloze task: kemampuan mengisi gap, minimal 2 paragraf, fixed ratio deletion (jarak gap sama). Rational deletion (jarak antar gap bebas).

Two approaches: exact word, appropriate word.

C- test (kata2 yang rumpang)