FLT: A CASE STUDY
The Human Resources Department of the City of Oakland approached International Contact, Inc. to develop a solution to their problem of confirming whether the employees at Public Contact Positions who request bilingual pay really speak another language.
The existing system of informal peer review provided no guarantee and could potentially create internal friction. The City wanted a testing solution that was convenient and inexpensive, as well as objective and fair.
Other generally accepted methods of language fluency weren’t viable. American Translator Association (ATA) certification is held infrequently, only certifies written translation and costs $600 per candidate. Berlitz and other language schools conduct telephone interviews, creating scheduling headaches for the HR Department and only reduces the cost to $125 per employee.
The City needs bilingual speakers able to take messages or provide direction to Oakland Citizens with limited English proficiency (LEPs) or monolingual. Native fluency in expression or written translation ability is not required for this purpose. Therefore, the test needed only to measure comprehension, spoken communication and “content translation”. Standardized tests that measure only verbal skills weren’t available.
Together with Human Resources, International Contact, Inc. designed a Computer Administered Test (CAT) that could measure the required skills on the employee’s recorded samples of responses to real life situations. Sample scenarios were taped and presented to self-declared bilingual volunteers in person. Any scenario that was too complicated or didn’t readily fulfill the parameters we were testing was weeded out. Every question was designed to reveal the employee’s oral language ability. The test was developed within the parameters of item response theory and increasing complexity.
A principal concern of HR was certainty that Union representatives would accept whatever action resulted from the test as complying with contractual obligations. FLT responded to this requirement by instituting rigidly objective metrics that would serve as specific anchor points, and developing a Protest Procedure that delineated the scoring process, thus ensuring that expert graders would be grading the same elements.
Once the content of the questions was settled, grading scales with anchors were developed to assist in standardization according to an objectivity scale developed with CAT system. Together with Human Resources we developed the parameters that were to be tested. A standardized scoring scale was established to ascertain that scores by different linguistic experts reflect the same parameters for each response.
Foreign Language Testing (FLT) programming was developed so the test could be administered via computer at the City’s site. Fluent speakers of the target language who have been trained in the standardized parameters grade the results. Thus, testing time is reduced to approximately 30 minutes, scheduling is at the convenience of Human Resources and the cost reduced to $50 per test. FLT provides the additional benefit of maintaining the sound files in the employee’s record, for any review required.
Qualifications of Reviewers
International Contact, Inc. imposes strict qualifications on every language expert that works with us. In addition to a resume, which must include evidence of native language fluency and scholarship, they possess certification by ATA or equivalent certification from their native country. Most importantly, they have a proven track record producing translations that have the stamp of approval of the target market over at least two years- some have as much as 25 years of experience!