作品展示十三
SET-10 Test Description
Validation Summary
SET－10 测试描述
有效性综述
有效性报告 － Ordinate SET-10
1. Introduction
1．前言
The SET-10® Spoken
English Test from Ordinate® Corporation evaluates the facility in oral English
of people whose native language is not English.
Ordinate 公司®的SET-10®英语口语测试可对非英语人士的英语口语熟练程度作出评估。
Academic institutions, international corporations and government agencies
throughout the world use the SET-10 to evaluate the ability of students, staff
or officers to understand spoken English and to express themselves clearly and
appropriately in English. The test is intended for use with adults and with
students
over 15 years of
age.
世界各地的学术机构、跨国企业和政府部门都在运用SET-10测试来评估学生、员工和官员的外语能力，即他们理解英语口语的能力以及运用英语来清晰、适当地表达自己的能力。本测试的评估对象是成年人和十五岁以上的学生。
2. Test
Description
2．测试描述
The SET-10 is a
ten-minute spoken English test for adult non-native speakers of English.
SET-10的测试对象为非英语国家的成年人，历时10分钟。
The test is delivered over the telephone and scored automatically by computer.
测试内容由电话线路传输，由计算机进行自动打分。
The SET-10 provides numeric scores and performance levels that describe the
test taker’s facility in spoken English – that is, the ability to understand
spoken English on everyday topics and to respond appropriately at a native-like
conversational pace in intelligible English.
SET-10测试提供的分数报告包括数字分数和能力等级，对参试者的英语口语熟练程度作出描述，也就是他们听懂日常话题英语句子的能力，以及能像本地英语人士那样用令人听得懂的英语给予适当的回答。
Test Format and
Administration
测试的模式和管理
The SET-10 test is
administered over the telephone by Ordinate’s testing system.
测试的管理由Ordinate 公司的测试系统通过电话线路来加以实施。
The system presents the test taker with a series of spoken prompts in English,
which the test taker should respond to appropriately in spoken English.
系统将提供给参试者一连串的英语口语提示，而参试者必须根据这些提示用英语口语来适当地回答。
The test consists of five sections, A through E. In the first four sections,
which are used to determine the test scores, the prompts and expected responses
are generally single utterances.
测试包括五个部分，从A部分到E部分。前四个部分决定出测试的分数，其中的提示和回答大多是一句简短的句子。
The last section contains longer questions that require more elaborate responses.
In a typical test, these responses are not scored. They are used on a regular
basis for validation and norming purposes.
第五部分含有较长的问题，需要更为详细的回答，而在通常情形下，并不对这些问题的回答进行打分，而是出于对测试有效性和测试人群进行评估的目的收集一些数据。
Test administration
is supported by a test paper. The test paper is a single sheet of paper with
material on both sides.
测试的管理过程包括一张试卷，它是一张纸，正反两面都有内容。
On the first side, the test paper displays general instructions and an
introduction to the test procedures (See Appendix A.). These materials are the
same for all test takers.
正面印有关于测试的总体指示以及对测试程序的介绍（见附录A），其内容对所有的考生都是一样的。
On the second side, the test paper displays the individual test form (See
Appendix B.).
试卷反面的内容则各不相同（见附录B）。
The individual test form is unique for each test taker. The individual test
form contains the Test Identification Number, displays verbatim the spoken
instructions and item examples, and presents the printed sentences that the
test taker must read aloud in Part A: Reading.
每个考生拿到的试卷都是不相同的，上面有测试识别号码，以及印有与语音提示和项目例句完全相同的内容，另外还印着在“A部分：朗读”中考生必须大声朗读出来的句子。
It is best practice
for the test administrator to give the test paper to the test taker at least
five minutes before the test begins. The test taker then has the opportunity to
read both sides of the test paper before the test begins and can ask questions.
管理者应采取的最佳做法是，在测试正式开始前至少提前五分钟就将试卷发到考生的手中，以便让他们事先阅读试卷正反面上的内容，以及向管理者提问。
After reading the
test paper, the test taker calls the Ordinate testing system, is instructed by
the system to enter the Test Identification Number, interacts in English with
the computer over the telephone, and hangs up when finished.
在看了试卷之后，考生就可以自己拨打电话，接通到Ordinate公司的测试系统。然后系统会指示考生输入测试识别号，通过电话与系统做英语的互动交流，以及在测试结束后挂断电话。
As the test taker interacts with the Ordinate testing system during test
administration, an examiner voice speaks all the instructions for the various
parts of the test.
当考生在与测试系统作互动式交流时，电话中会出现一个考官的声音，他会把测试各个部分的所有注意事项全都一一念出来。
Each test item
requires the test taker to understand a spoken utterance and speak in response
to it.
每个测试题都要求参试者听懂一句话，然后给出回答。
Test items are presented in various native-speaker voices that are distinct
from the examiner voice.
将测试题目念出来的声音是好多种英语本地人士的语音，这些声音与考官的声音截然不同。
When the test
administration is over, Ordinate’s testing system analyzes the test taker’s
responses and returns an Overall score and four diagnostic subscores, usually
within minutes after the call is completed.
测试结束之后，Ordinate公司的测试系统会对考生的回答进行分析，然后给出一个总分和四个评定性的小分。通常在测试完成后的几分钟之内分数就可获得。
Two of the subscores reflect the content of the test taker’s responses (what
was said) in relation to the task presented in the item prompt. The other two
subscores reflect the manner of the test taker’s responses (how they were said)
in terms of fluency and pronunciation quality.
有两个小分表示的是考生根据测试系统给出的语音提示所回答的内容（他实际上说了些什么），另两个小分表示的是考生“回答的方式”（他是如何说的），即有关流利程度和发音质量方面的。
Test administrators and score users can view and print out test results from a
password-protected section of Ordinate’s website.
测试管理者和分数使用者可以在Ordinate公司网站上的一个有口令保护的区域看到测试结果，并可将它们打印出来。
A typical sequence
of events in the SET-10 testing process is as follows:
典型的SET-10测试按以下的顺序进行：
0. (optional,
recommended) The test taker takes a practice SET-10 test.
1. The test taker
gets the SET-10 test paper and reviews it.
2. The
administrator answers any procedural or content questions.
3. The telephone
call is made to the Ordinate testing system.
4. The test taker
enters the Test Identification Number on the telephone keypad.
5. The test taker
speaks in response to the content of the SET-10.
6. The telephone
call is terminated, and the test taker hangs up.
7. The Ordinate
system analyzes the test taker’s spoken performance.
8. The Ordinate
system posts the test taker’s scores at www.ordinate.com.
9. The test taker
or score user retrieves the SET-10 scores from www.ordinate.com.
0 － （可选项，建议采纳）考生先参加一场模拟的SET-10测试。
1 － 考生获得一张试卷，然后仔细阅读。
2 － 管理者就考试程序及内容方面回答考生提出的问题。
3 － 拨打电话、接通到Ordinate公司的测试系统。
4 － 考生在电话键盘上输入各自的测试识别号。
5 － 考生根据SET-10测试的语音提示内容回答问题。
6 － 电话测试结束，考生挂断电话。
7 － Ordinate系统对考生的口语能力作出分析。
8 － Ordinate系统将考生的测试成绩公布在网站上：www.ordinate.com
9 － 由考生或分数使用者在www.ordinate.com 网站上获得SET-10测试分数。
Test Construct
测试结构
The SET-10 test
measures facility in spoken English – that is, the ability to understand spoken
English on everyday topics and to respond appropriately at a native-like
conversational pace in intelligible English.
SET-10测试将测评考生的英语口语熟练程度，即考生听懂日常话题英语句子的能力，以及像本地英语人士那样用令人听得懂的英语给出适当回答的能力。
Another way to express the construct facility in spoken English is “ease and
immediacy in understanding and producing appropriate conversational English.”
还有另一个方法来评估英语口语的熟练程度的构造，那就是“轻松地、即刻地理解英语口语，并给于适当回答”。
This definition relates to what occurs during the course of a spoken conversation.
该定义与口语交流过程中实际上发生的事件有关。
While keeping up with the conversational pace, a person has to track what is
being said, extract meaning as speech continues, and then, on occasion,
formulate and produce a relevant and intelligible response.
一个人在跟上谈话节奏的同时，他还要了解对方正在说些什么，也就是在谈话进行过程中辨别出内在意义，然后在必要时说出意义相关的句子，给出令人听得懂的回答。
These component processes of listening and speaking are schematized in Figure
1, adapted from Levelt (1989).
图1中列出的是听说过程的组成要素，它们是根据Levelt1989年的心理语言学模型改编而成的。
< 听 <
|
听句子 辨别出单词 识别短语结构 对叙述进行解码 理解上下文 领会对方要求（如有的话） |
|
清晰地回答 建造短语结构 选择句法内容 建造短语 选择话题 决定如何回答 |
＞ 说 ＞
图１：口语听说的会话处理成份
In the SET-10, the
Ordinate testing system presents a series of discrete prompts to the test taker
at a native conversational pace as recorded by several different native
speakers, producing a range of native accents and speaking styles.
Ordinate SET-10测试系统向受测试者发出一连串相互间独立的语音提示，该类提示是按多种不同的本地英语人士的声音录制的，语速是按本地英语人士的正常会话速度，带有一系列当地人的口音和谈话风格。
These integrated “listen-then-speak” items require real-time receptive and
productive processing of spoken language forms, and the items are designed to
be relatively independent of social nuance and high-cognitive functions.
这种“听了就说的”合成化考题要求考生对口语形式的语言具有实时的接收领会和加工合成能力。考题的设计在相当程度上排除了社会化的细枝末节以及高度认知化的功能。
The same facility
in spoken English that enables a person to participate in everyday native-paced
English conversation also enables that person to satisfactorily understand and
respond to the listening/speaking tasks in the SET-10.
如想在SET-10的听说测试项目中取得理想的成绩，即能够听懂语音提示和给出正确的回答，考生必须具有下列情形的相同能力，即他们能参与到按当地人的同等语速的英语会话中去，以便参与有关日常话题的讨论。
Explained another
way, the SET-10 test measures the test taker’s control of core language
processing components, such as lexical access and syntactic encoding.
从另一个角度来解释，SET－10测试主要检测考生是否掌握了语言加工过程中的核心能力，诸如掌握词汇的一词多义现象和句子的解码过程等。
For example, in normal everyday conversation, speakers go from building a
clause structure to phonetic encoding (the last two stages in the right-hand
column of Figure 1) in 40 milliseconds (Van Turennout, Hagoort, and Brown,
1998).
举例说，说话者在参与普通的日常交谈时，应在40毫秒内组织成短语结构，并执行语音编码过程（即图1中右栏的最下面两项），（详见Van Turennout、Hagoort和Brown 1998年的著作）。
Similarly, the other stages shown in Figure 1 have to be performed within the
small period of time available to a speaker involved in everyday communication.
同样的，对于参与日常话题交流的人们来说，他们也必须在相当短的时间内执行表1中的其它步骤。
The typical window in turn taking is about 500 milliseconds (Bull and Aylett,
1998).
在典型的口语交流过程中，其依次旋转的时间之窗为500毫秒（Bull 和 Aylett，1998年）。
If language users involved in real-time communication cannot perform the whole
series of mental activities presented Figure 1, both as listeners and as
speakers, they will not be able to participate actively in such communication.
如参与实时交流的语言使用者无法执行表1中心理活动的全过程，则无论是听话人还是说话人都无法积极地参与到该种交流中。
In this process,
automaticity is required in order for the speaker/listener to be able to pay
attention to what needs to be said/understood rather than to how the encoded
message is to be structured/analyzed.
在这个过程中，要求说话者和听话者都能自动地领会，以便将注意力放在需要说些什么、理解些什么上面，而不是将注意力集中在该种编码信息是何种构造的、应怎样分析等。
Automaticity entails the ability to access and retrieve lexical items, to build
phrases and clause structures, and to articulate these without conscious
attention to the linguistic code (Cutler, 2003; Jescheniak, Hahne, and
Schriefers, 2003; Levelt, 2001).
自动领会能力使得人们能在谈话过程中可以自由地存取词汇、构建短语和句型，以及在并未有意识地思考语言学规则的情形时将句子成分连接组合成令人听得懂的句子。（Cutler 2003；Jescheniak、Hahne及Schriefers
2003；Levelt 2001）
The SET-10 probes
the psycholinguistic elements of spoken language performance rather than the
social and rhetorical elements of communication.
SET-10测试所探究的是口语语言能力中的语言心理学成份，而不是交流中的社会学和修辞学方面的内涵。
Because during the test this probing is performed in real time, the SET-10
measures the degree of automaticity in language performance.
由于在测试过程中这种探索是实时进行的，因而SET-10测试所测定的是语言能力的自动领会程度。
A person has to understand and produce language at some level of accuracy and
fluency to participate in a spoken interchange.
在参与一场口语交流时，人们必须具备一定程度的准确性和流利程度，才能理解对方的谈话，并给出自己的回答。
Since performance standards can be established for accuracy and fluency based
on representative samples of language users, the SET-10 checks the level of
accuracy and fluency, and at the same time directly measures the rate and level
of language process control.
由于可以将语言使用者的代表性样本作为基础来建立起可测定出语言准确性和流利性的标准，因而SET-10测试能够评定出语言准确性和流利性的等级，并在同时直接测定出语言过程控制中的水平和等级。
图2：在口语交流中，信息编码和解码是一种实时的链式过程
To summarize, the
SET-10 measures basic encoding and decoding of oral language as performed in
integrated tasks in real time.
总之，SET-10测定人们在口语交流中基本的实时、集成化的编码和解码能力。
This performance predicts a more general spoken language facility, which is
essential in successful oral communication.
这种能力可以预测更为广泛的口语能力（天赋），而在成功的口语交流中这种能力（天赋）是至关重要的。
The reason for the predictive relation between spoken language facility and
oral communication skills is schematized in Figure 2.
图2中列出的是语言口语天赋与口语交流技能之间的预测性关系的理由。
This figure puts Figure 1 into a larger context, as one might find in a
socially situated dialog.
该图将图1置于更大的领域之中，即人们在一场社会性的对话中可能发觉的内容。
The language structures that are largely shared among the members of a speech
community are used to encode and decode various threads of meaning that are
communicated in spoken turns.
在一个语言社区中大量通用的语言结构可对多种意义的线索进行编码和解码，而该种意义线索是口语形式的。
These threads of meaning that are encoded and decoded include declarative
information, as well as social information and discourse markers.
这些被编码和解码的意义线索包括了陈述性的信息，以及社会化的信息和论述标识等。
World knowledge and knowledge of social relations and behavior are also used in
understanding the spoken turns and in formulating the content of spoken turns.
世界性的常识以及社会关系和社会行为的知识也可被用来理解口语，以及组织起口语内容。
However, these social-cognitive elements of communication are not represented
in this model and not directly measured in the SET-10.
然而，这些带有社会性认知成份的语言交流并不在该模型中出现，也不在SET-10测试中直接加以测量。
3. 3.
Content
Design, Material and Development
3．内容设计、素材及开发
Content
内容
The SET-10 test
measures both listening and speaking skills, emphasizing the test taker’s
facility (ease, fluency, immediacy) in responding aloud to common, everyday
spoken English.
SET-10测试主要测定人们的听力及说话技能，重在测定受测试者是否能针对日常内容的英语口语给与大声回答，并强调其熟练程度（即轻松自如、流利、即刻回答等）。
All SET-10 items
are designed so that both native speakers and proficient non-native speakers
find them very simple to understand and to respond to appropriately.
SET-10的所有试题都被设计成如下情形：对于本地英语人士和精通英语的非本地人士而言，它们都是很容易理解的，并可恰当地给出回答。
The items cover a broad range of skill levels and skill profiles.
试题覆盖了范围广泛的技能等级和技能领域。
Verification of
these test characteristics is described below in Section 6 on validation.
对这些试题特点所作的评估将在下面的第六章“有效性”中给出。
The vocabulary used in the test items and responses is restricted to the most
frequent words found in the Switchboard Corpus (Godfrey and Holliman, 1997), a
corpus of three million words used in spontaneous telephone conversations.
在测试题及回答中使用到的词汇被限定在Switchboard Corpus（交换台文献）(Godfrey and
Holliman, 1997)中的最常用词汇集中。该文献包括了300万个真实性电话交谈中出现的词汇。
In general, the language structures used in the test reflect those that are
common in everyday English. This includes extensive use of pronominal
expressions such as “she” or “their friend” and contracted forms such as
“won’t” and “I’m.”
总的来说，测试的语言结构反映了日常的通用英语，包括大量使用代名词，如“她”、“他们的朋友”，以及缩略形式如”won’t”和 “I’m”等
Each SET-10 item is
independent of other items and presents unpredictable spoken material in
English.
每个SET-10的试题都独立于其它试题，所给出的英语口语材料都是无法预测的。
Context-independent
material is used in the test items for three reasons.
采纳独立于上下文的测试材料有以下三个理由：
First,
context-independent items exercise and measure the most basic meanings of
words, phrases, and clauses on which context-dependent meanings are based
(Perry, 2001).
首先，独立于上下文的测试题可用于练习和测定单词、短语和句子的最基本意义，而基于上下文的意义是建筑在这些单词、短语和句子之上的(Perry, 2001)。
Second, when language usage is relatively context-independent, task performance
depends less on factors such as world knowledge and cognitive style and more on
the test taker’s facility with the language itself.
第二，如果语言用法在相当程度上是独立于上下文的，在回答时就可以较少地依赖实际事实，如世界性常识和认知性的风格等，而是更多地着重于受测试者的语言能力本身。
Thus, the test performance relates most closely to language abilities and not
to other test-taker characteristics that may be outside the target construct,
which is facility in spoken English.
这样，测试就更着眼于语言能力，而不是检测受测试者超出了目标构造的其他特点。所谓的目标构造就是英语口语的熟练程度。
Third, context-independent tasks maximize response density; that is, within a
given test administration time, the test taker has more time to demonstrate
performance in speaking the language. Less time is spent in developing a
background cognitive schema for the task.
第三，独立于上下文的测试题可以最大程度地集中回答密度，那就是在测试规定的时间内，考生有更多时间来展现其说外语的能力，而将较少的时间花在对测试题的认知背景之上。
Both the test items
presented and the expected responses are constrained to contain common English
vocabulary and constructions that can be consistently understood and/or
produced by at least 90% of a reference sample of educated native speakers of
English.
所给出的测试题以及期待得到的回答，其词汇被限定在日常英语词汇的范围内，而在测试人群样本中至少有90％的受过教育的英语本地人士能够一如既往地理解或回答它们。
In addition, these item types maximize reliability by providing multiple, fully
independent measures.
此外，由于具备完全独立的多重手段，这些试题类型的可靠性获得了最大化的提升。
P.7
They elicit responses that can be analyzed
automatically to produce measures that underlie facility with spoken English,
including phonological fluency, sentence comprehension, vocabulary, and
pronunciation of rhythmic and segmental units.
试题所得出的回答可被自动分析，产生出可检测英语口语能力的手段，包括对发音的流利性、句子理解、词汇、带节奏的发音，以及片语单元等的检测手段。
Test Structure
测试构造
The SET-10 test consists of 61 items
that are presented in five separate sections (Parts A through E).
SET-10测试包括61道题，出现在五个独立的部分中（从A部分到E部分）。
Each of the five parts presents the test taker with a different task type, as
shown in Table 1.
每一部分的试题类型各不相同，见表1。
|
测试部分 |
试题类型 |
试题数 |
|
A部分 |
大声朗读 |
8 |
|
B部分 |
复述句子 |
16 |
|
C部分 |
回答简短问题 |
24 |
|
D部分 |
造句 |
10 |
|
E部分 |
开放性提问 |
3 |
表1：SET－10的构造
In Part A, test
takers are instructed to read particular sentences from among a set of numbered
sentences printed on the test paper (See Appendix B.)
在A部分，系统将指示受测试者从印刷在试卷上带号码的句子中读出指定的句子（见附录B）。
In parts B through E, the item materials are presented by voice only, with no
direct support from the test paper.
从B