Informal and Formal Assessment
Informal Assessment = Classroom
tasks designed to elicit performance without recording results and making fixed
judgments about a student’s competence. = tidakmenggambarkan competence
keseluruhan.
Informal assessment can take a
number of forms, starting with incidental, unplanned comments and responses,
along with coaching and other impromptu feedback to the student.
Formal Assessment = Exercise
or procedures specifically designed to tap into a storehouse of skills and
knowledge.
Formal assessments
are exercises or procedures specifically designed to tap into a storehouse of
skills and knowledge. Is formal
assessment the same as a test? We can say that all tests are formal
assessments, but not all formal assessment is testing.
Formative and
Summative Assessment
Formative assessment: evaluating
students in the process of "forming" their competencies and skills
with the goal of helping them to continue that growth process. The key to such
formation is the delivery (by the teacher) and internalization (by the student)
of appropriate feedback on· performance, with an eye toward the future
continuation (or formation) of learning.
Summative assessment aims to measure,
or summarize, what a student has grasped, and typically occurs at the end of a
course or unit of instruction. A summation of what a student has learned
implies looking back and taking stock of how well that student has accomplished
objectives, but does not necessarily point the way to future progress. Final
exams in a course and general proficiency exams' are examples of summative
assessment.
Norm-Referenced
and Criterion-Referenced Tests
In norm-referenced tests, each
test-taker's score is interpreted in relation to a mean (average score), median
(middle score), standard t-/ deviation (extent of variance in scores), and/or
percentile rank. The purpose in such tests is to place test-takers along a
mathematical continuum in rank order. Typical of norm-referenced tests are
standardized tests like, the Scholastic Aptitude Test. (SAT) or the Test of
English as a Foreign Language (TOEFL_ intended to be administered to large
audiences, with results efficiently disseminated to test-takers. Such tests
must have fixed, predetermined responses in a format that can be scored quickly
at minimum expense. Money and efficiency are primary concerns in these tests.
Norm-referenced
examples
Criterion-referenced tests, on the other hand, are designed to give test-takers
feedback usually in, the form of grades, on specific course or lesson
objectives. Classroom tests involving the students in only one class, and
connected to a curriculum, are typical of criterion-referenced testing.
Example:
|
Score |
Criteria |
|
5 |
Very fluent;
Speak without intonation; No Grammatical error; Native-like |
|
4 |
|
|
3 |
|
Discrete-Point and Integrative Testing
Discrete-point tests are constructed on
the assumption that language can be broken down into its component parts and
that those parts can be tested successfully. It was claimed that an overall
language proficiency test, then, should sample all four skills and as many
linguistic discrete points as possible.
Discrete-point
testing = mentest/menilai students di 1 part tersendirisajadari 1 skill /
seluruh skill (example: speaking: Pronunciation)
Integrative Testing = there are more
than one skill to be required to do one test
Language should be
tested integratively not discretely
Direct test vs
Indirect Test
Indirect test
To know how much
the students know about language
Contoh:
Mengukurkemampuansiswamelalui
written monolog/dialog ygdikumpul
Direct test =
students are test directly what we want to measure. Contoh:
In the term of
validity direct test is more valid and reliable than indirect test
Objective and Subjective Test
Objective test
No judgment is
required on the part of the scorer e.g. multiple choice items, True/False
items, short answers, etc.
Subjective Test
Judgment is called
for scoring, e.g. essay items, paragraph writing, delivering speech, acting out
a dialog, etc.
Test Types:
Proficiency test
It is designed to
measure people’s ability in a language regardless of any training they may have
had in that language, e.g. TOEFL. IELTS, TOEIC.
Placement Test
It is designed to
provide information that might help to place students at the stage of the
teaching program most appropriate to their abilities.
Language Aptitude Test
It is designed to
measure capacity or general ability to learn a foreign language and ultimate
success in that undertaking language.
Diagnostic Tests
A diagnostic test
is designed to diagnose specified aspects of a language or to identify
learners’ strength and weakness.
Achievement Tests
It is designed to
establish how successful individual students, group of students, or the cources
themselves have been in achieving objectives.
CHAPTER 2:
Principles of Language Testing
An effective test
has:
Practically,
Reliability, Validity, Authenticity, and Washback.
A. Practically,
a test:
·
Is not excessively expensive
·
Stays within appropriate time constrains
·
Is relatively easy to administer
·
Has a scoring/evaluation procedure that is
specific and time-efficient
B. Reliability
A
test is reliable if it yields consistent results when repeated over time.
To
measure students’ proficiency
It
is consistent and dependable.
Factors
that influencing reliability:
·
Student-related reliability
The
most common learner-related issue in reliability is caused by temporary
illness, fatigue, a "bad day," anxiety, and other physical andpsychological factors. Also included in this category are such
factors as a test-taker's "test-wiseness" or strategies for efficient
test taking.
2
factors: Psychological factors and physical factors.
·
Rater reliability
There
are 2 rates:
1. Inter-rater reliability occurs when two
or more scorers yield it inconsistent scores of the same test, possibly for
lack of attention to scoring criteria, inexperience, inattention, or even
preconceived biases.
2. Intra-rater reliability is a common
occurrence for classroom teachers because of unclear scoring criteria, fatigue,
bias toward particular "good" and "bad" students, or simple
carelessness.
Those
can be because of lack of attention to scoring criteria, inexperience,
inattention, preconceived bias.
·
Test administration reliability
The
manner in which the examiner presents, the instructions, the characteristics of
the room, outside, noises, etc.
·
Test reliability
The
sampling of the objective, the degree of ambiguity of items, restriction on
freedom of response, the clarity of instructions, the quality of layout, the
familiarity with the formal, the length of the test.
The
nature of the test itself can cause measurement errors. If a test is too long,
test-takers may become fatigued by the time they reach the later items and
hastily respond incorrectly. Timed tests may discriminate against students who
do not perform well on a test with a time limit.
C. Validity
·
Kesesuaian antara alat ukur dengan apa
yang diukur
·
Is the degree in which the test measures
accurately in what it is intended to measure (Hughes; 2003)
·
The extent to which interference made from
assessment results are appropriate, meaningful, and useful in terms of purpose
of the assessment. (Gronlund, 1998)
·
The extent to which it measures what it is
supposed to measure and nothing else. (Heaton, 1988)
·
A test
must be valid because
·
Apa yang kita nilai sesuai dengan apa
yang kita ajarkan yang sesuai dengan learning objective.
·
Untuk memberikan keadilan untuk siswa, bahwa siswa tersebut diukur sesuai dengan kemampuan mereka
Ø
Content-Related Evidence (Content Validity)
·
Alat ukur
yang sesuai dengan materi yang diajarkan
·
A test is said to have content validity if its
content constitutes a representative sample of the language skills, structures,
etc. which it is meant to be concerned
·
The course objective ó the test items
·
Content validity: tertera di
kurikulumdansudahdiajarkan, jika 2 halitusudahada, barudiadakan test
·
You can usually identify content-related
evidence observationally if you can clearly define the achievement that you are
measuring
·
Another way of
understanding content validity is to consider the difference between direct and
indirect testing
·
A test would have
content validity only if it included proper sample of the relevant structures
of the purpose of the test.
·
The importance of
content validity:
1.
The greater a
test’s content validity, the more likely it is to be an accurate measure of
what it is supposed to measure
2.
Such a test is
likely to have a harmful backwash effect
Ø
Criterion-Related Evidence (Criterion-Related
Validity)
·
The degree to which results on the test agree
with those provided by some independent and highly dependable assessment of the
candidate’s ability.
·
Kesesuaianantarasebuahhasiltesdenganhasilteslainnyameskipuntesnyatidaksama
·
In the case of teacher-made classroom
assessments, criterion-related evidence is best demonstrated through a
comparison of results of an assessment with results of some other measure of
the same criterion.
·
2 kinds of criterion-related validity:
1. Concurrent
Validity
Is
established when the test and the criterion are administered at the same time.
E.g. achievement testing. 2 test yang diberikan yang berjarakwaktu yang
sebentar
2. Predictive
Validity
This
concerns the degree to which a test can predict candidates’ future performance.
E.g. Placement testing. Bias di lihatsaatbelajarmengajar
·
A test has concurrent
validity if its results are supported by other concurrent performance
beyond the assessment itself.
·
The
predictive validity of an assessment becomes important in the case of
placement tests, admissions assessment batteries, language aptitude tests, and
the like.
·
The predictive validity= diprediksi di
awaldanharus survive with the prediction.
·
The predictive validity= dikorelasikanantarahasil
placement test dan achievement test.
Ø
Construct-Related Evidence
·
Kesesuaian tes dengan theoretical background mengenai unsur-unsur yang membangun skill yang ingin diuji
·
A test is said to have construct validity if it
can be demonstrated that it measures just the ability which it is supposed to
measure.
·
Construct = any underlying ability/trait that is
hypothesized in a theory of language ability.
·
Example: oral proficiency includes
pronunciation, fluency, grammatical accuracy, vocabulary use, and
sociolinguistic appropriateness
·
Construct validation is a research activity, the
means by which theories are put to the test and are confirmed, modified, or
abandoned. It is through construct validation that language testing can be put
on a sounder, more scientific footing.
Ø
Consequential Validity
Consequential
validity encompasses all the consequences of a test, including such
considerations as its accuracy in measuring intended criteria, its. impactonthe
preparation of test-takers, its effect on the learner, and the (intended and
unintended) social consequences of a test's interpretation and use.
Ø
Face Validity
·
Kesesuaian antara tampilan test dengan apa tujuan testnya.
·
Face validity refers to the degree to which a
test looks right, and appears to measure the knowledge or
abilities it claims to measure, based on the subjective judgment of the
examinees who take it, the administrative personnel who decide on its use, and
other psychometrically unsophisticated observers (Mousavi, 2002, p. 244).
·
Face validity will likely be high if learners
encounter:
1. A
well-constructed, expected format with familiar tasks,
2. A
test that is clearly doable within the allotted time limit,
3. Items
that are clear and uncomplicated,
4. Directions
that are crystal clear,
5. Tasks
that relate to their course work (content validity), and
6. A
difficulty level that presents a reasonable challenge.
·
Face validity is not something that can
be empirically tested by a teacher or even by a testing expert. It is purely a
factor of the "eye of the beholder" - how the test-taker, or possibly
the test giver, intuitively perceives the instrument.
·
Validity in scoring = if a test is have to
validity, the way in which the responses are scored must be valid too.
·
A test is said to have face validity if it looks
as if it measures what it is supposed to measure.
D. Authenticity
·
Authenticity, a concept that is a little
slippery to define, especially within the art and science of evaluating and
designing tests.
·
Authenticity is the degree of correspondence of
the characteristics of a given language test task to the features of a target
language task (Bachman and Palmer 1996)
·
In a test, authenticity may be present in the
following ways:
1.
The language in
the test is as natural as possible.
2.
Items are
contextualized rather than isolated.
3.
Topics are
meaningful (relevant, interesting) for the learner.
4.
Some thematic
organization to items is provided, such as through a story line or episode.
5.
Tasks represent,
or closely approximate, real-world tasks.
E. WashBack
Washback generally refers to the
effects the tests have on instruction in terms of how students prepare for the
test.
Reading
Interactive reading: menekankan pada makna, top down (general
ke specific), minimum dua paragraf.
Cloze task
mengisi kekosongan dalam paragraf.
C-test procedure (rumpang di dalam kata)
Cloze elide procedure(menghilangkan kata yg tidak perlu)
Impromptu reading plus coprehension
question: Jenis teks
tidak diketahui, lebih kepada cara menjawab. ex: reading text in toefl
test
Interactive reading short answer tasks
Interactive Reading Editing
Memahami seluruh teks.
Interactive reading scanning
Menscan dan menemukan informasi. Timing
bisa jadi penilaian.
Interactive Reading – ordering task
Menyususn kalimat acak
Interactive reading – information transfer
Test taker diminta untuk membaca grafik
dan diminta untuk mengintrepretasikannya. Make inferences overly.
Extensive reading
Top down extensive reading: memiliki background knowledge terlebih
dahulu
Penilaian informal: response, comment.
Point penilaian harus point penilaian
reading.
Skimming task
Proses membaca cepat untuk menentukan inti
atau gagasan utama.
Penilaian: informal, formative
(berkelanjutan).
Skimming: proses, Scanning: strategy. Untuk mecari informasi tertentu..
Note taking: Mengorgasir semua materi dari awal sampai akhir,
siswa membuat catatan yang dianggap penting.
Based on question and Answer session.
Agus=Dalam interactive reading itu bisa
lebih spesifik dan bisa di nilai, beda dengan extensive reading yang mencakup
banyak penilaian.(Annisa NK)
Rania=make inference that are not
presented orally, maksudnya adalah informasi yang didapat sesuai dengan data
yang ditampilkan (Marwah),
information transfer itu adalah bagaimana siswa dapat mem (uci)
Interactive reading: combine form –
meaning focus (fokus makna) general to specific (top-down).
Cloze task: kemampuan mengisi gap, minimal
2 paragraf, fixed ratio deletion (jarak gap sama). Rational deletion (jarak
antar gap bebas).
Two approaches: exact word, appropriate
word.
C- test (kata2 yang rumpang)
