A TEST SCORING SERVICE INFORMATION BULLETIN
Measurement
and Research Services
Memo No.16
November 12, 2004
Measurement and Research Services provides free scoring of multiple choice tests for all teaching faculty. Numerous scoring and reporting options are available such as item weighting, listings of student errors, individual score reports, and output of scores to WYLBUR, or IBM compatible 3.5 ” disk. This bulletin describes the services that are offered and the procedure for submitting a test to be scored.
Scannable answer sheets are available for departmental purchases through TAMU’s Purchasing and Stores Department. Students can purchase the answer sheets at many bookstores in town, including the MSC Bookstore. The answer sheets are gray, have space for up to 150 questions with five response options per question. The number of the form will be either NCS MP90051, NCS Pearson MM90051-2 or SCANTRON FORM NO. 0-101607-TAMU. Usable forms will be labeled, “TEXAS A&M UNIVERSITY Measurement and Research Services”.
When administering your test, please be sure that your students are using No. 2 pencils to fill out their answer sheets. Sheets completed with pens or hard pencils can not be read by scanning machines. Students should be instructed to enter their names and social security numbers on answer sheets and to fill in the corresponding bubbles. Students should be asked to not fold, bend or make stray marks on the answer sheets.
SUBMITTING TESTS FOR SCORING
When a test is submitted for scoring, please line up all the forms in the same direction. Make sure that you have finished your answer key, header sheet and the other forms (if needed). Please have the tests in an envelope or box that is clearly marked with instructor’s name and the course number.
Instructor Supplied Information
You will be asked to fill out a green header sheet (a Test Scoring Request form) which will ask your name, department, course number, course section, and phone number. You will also be asked to select the types of scoring, reporting and output options that you would like. Most of the options that are available are described in the next section. Examples of scoring and report options are available at the test scoring office.
The Answer Key
For each set of tests to be scored, an answer key needs to be included. The answer key is simply an answer sheet that contains the correct answers. The answer key should have the appropriate bubble filled in to indicate that it is an answer key. That is, the “A” should be darkened in the field labeled “Sheet Type.” The answer sheet must be on top of the tests to be scored. If it is not, your tests will not be scored correctly, and a warning message will appear on the first page of your test summary report. Also, it is helpful to write the word “KEY” on the answer sheet to aid in its identification.
If you want to delete an item from the test, leave the item on the key blank. If an item has two or more correct answers the item doesn’t need an answer entered on the key. The correct key values will be entered from the information supplied on the Multiple Response sheet.
Recording Scores on 3.5” Disks
If you are going to keep a record of test results and responses on a 3.5” disk, you must supply the disk. The disk should be double-sided and double density or high-density. The disk should be formatted for an IBM compatible PC before it is brought to us.
THE TEST ANALYSIS REPORT
Each time a test is scored, the following information about the test will be produced:
a) a listing of the correct responses,
b) a list of the scoring, reporting and output options requested.
c) a description and display of the distribution of student scores.
d) an analysis of the test items, and
e) a description and summary of the test’s characteristics.
The test-scoring program offers a multitude of scoring, reporting, and output options. In the following sections, each of the options is described. For a complete description of the output and a guide for interpreting the results, see Measurement and Research Services Memo No. 18, “ A Description of Texas A&M University’s Test Score Report.”
SCORING OPTIONS
Multiple Forms
In large classes it is a common practice to use more than one test form. A test form is a version of a test with a specific ordering of items. The test-scoring program can score up to four forms of a test at the same time. A letter designates each form: A for the first form, B for the second form, C for the third form, and D for the fourth. When more than one test form is used, the students must indicate on their answer sheet in the field titled “TEST FORM” which form they are taking.
To score tests with multiple forms, the correspondence between the items on the first form and the second, third, and fourth forms needs to be defined. The simplest method is to use one of the standard item reordering methods listed on the scoring request form. The seven standard item reordering methods are:
Reversal: The order of the items on the first form are reversed. For a five item test, the correspondence between Form A and the alternative form which used reversal is shown below.
|
|
Form A |
Form B |
|
|
|
|
|
|
1 |
5 |
|
Item |
2 |
4 |
|
Number |
3 |
3 |
|
|
4 |
2 |
|
|
5 |
1 |
Odd-Even Switch: Each even numbered item on Form A is switched with the odd numbered item that preceded it. If the test contains an odd number of items, the last item on the alternative form will be the same as the last item on Form A. The correspondence between Form A and alternative form on a five-item test which uses an odd-even switch is shown below.
|
|
Form A |
Form B |
|
|
|
|
|
|
1 |
2 |
|
Item |
2 |
1 |
|
Number |
3 |
4 |
|
|
4 |
3 |
|
|
5 |
5 |
Split-Half Switch: The last half of the items on Form A are placed at the beginning of the alternative form. When a test has an odd number of items, the last test item on Form A will also be the last item on the alternative form. The correspondence between Form A and the alternative form which uses split-half switching is shown below for a five-item test.
|
|
Form A |
Form B |
|
|
|
|
|
|
1 |
3 |
|
Item |
2 |
4 |
|
Number |
3 |
1 |
|
|
4 |
2 |
|
|
5 |
5 |
Odd-Even Switch followed by Reversal: The order of the items on Form A are first modified by odd–even switching and then the order is reversed. An example for a five- item test is shown below.
|
|
Form A |
Form B |
|
|
|
|
|
|
1 |
5 |
|
Item |
2 |
3 |
|
Number |
3 |
4 |
|
|
4 |
1 |
|
|
5 |
2 |
Split–Half Switch followed by Reversal: The order of the items on Form A are first modified by a split–half switch and then the order is reversed. An example for a seven-item test is shown below.
|
|
Form A |
Form B |
|
|
|
|
|
|
1 |
7 |
|
Item |
2 |
3 |
|
Number |
3 |
2 |
|
|
4 |
1 |
|
|
5 |
6 |
|
|
6 |
5 |
|
|
7 |
4 |
Split-Half Switch followed by Odd-Even Switch: The item order in Form A is first modified by a split-half switch, and then modified by an odd-even switch. An example of this method of reordering for a seven item test is shown below.
|
|
Form A |
Form B |
|
|
|
|
|
|
1 |
5 |
|
Item |
2 |
4 |
|
Number |
3 |
1 |
|
|
4 |
6 |
|
|
5 |
3 |
|
|
6 |
2 |
|
|
7 |
7 |
Split-Half Switch followed by Odd-Even Switch followed by Reversal: The item order in Form A is first modified by a split half switch. The order is then modified by an odd-even switch and then the item order is reversed. An example for a six-item test is shown below.
|
|
Form A |
Form B |
|
|
|
|
|
|
1 |
2 |
|
Item |
2 |
3 |
|
Number |
3 |
6 |
|
|
4 |
1 |
|
|
5 |
4 |
|
|
6 |
5 |
For tests of unusual length, we have a quick computer program that will generate the correct item order for any of the reordering methods cited above. If you have any questions at all regarding the built-in reordering methods, we would encourage you to check with us first before administering your test.
You need not use one of the built-in item ordering methods. If you would like to create your own alternative form, you will need to fill out a Form Sheet with your tests. These sheets are available at the test scoring office. To fill out the Form Sheet, you will need to enter the following information:
1) the form that the sheet describes (B, C, or D),
2) the item numbers described by the sheet, (Each sheet can hold information for forty items. If your test has more than forty items, but less than eighty-one, two sheets will be needed. Up to four sheets may be necessary.)
3) the correspondence between the item number on Form A and the same item’s number on the alternative form. This information must be entered for each item.
Item Weighting
Each item on the test may be assigned a scoring weight. Weights are usually assigned because the instructor wants the total score to sum to a specific value. Other instructors assign differential scoring weights because they believe some items to be more important than others. From a psychometric point of view, the practice of assigning individual item weights based on personal judgement is highly questionable, however for those who desire to do this, the option is available.
Item weights may be assigned in two different ways. They are:
1) Equal item weights may be entered by filling in the field labeled “Scaling Factor” on the header sheet. The weight contains a decimal point so that weights may be assigned that are carried to the thousandth place. This option may come in handy if you want your test points to sum to an unusual value. You should remember however, that once a total is computed for each person, the score will be rounded to the nearest whole integer.
2) Item weights may be assigned to individual test items through the use of a Weight Sheet. These sheets are available at the test scoring office. To use the Weight Sheet, individual weights are entered for each item on the test. All weights must be non - negative integers less than 1000. To simplify entering these values, only those weights which are different from that of the preceding item need to be entered on the sheet. When an item is assigned a weight of zero, the item will be included in the item analysis, but the total score will be unaffected.
Response
Weighting
A scoring weight may be assigned to each response option of an item. This allows partial credit to be given to responses which are not completely correct. For each item with multiple response weights, a section of the Multiple Response Sheet must be filled out. Weights must be non-negative values less than 100. These weights will be multiplied by the value of the scaling factor. Multiple Response Sheets are available at the test scoring office, where a bulletin with complete examples can also be obtained.
Bonus Points
If you want to add a constant number of points to every examinee’s score, you may enter the value on the Scoring Request Form in the field labeled “Bonus Points”. This constant may take any integer value between –999 and +999.
REPORTING OPTIONS
The test score results can be reports in several different formats and you may select the options as you desire, however we ask that you request no more than you actually need. The seven different reporting options are described in the following section.
Name Roster
The name roster is an alphabetized list of the examinees which contains the following information:
a) name (last name, first name, middle initial)
b) ID number
c) test score
d) percentile rank
e) standard score
f) percent correct
ID Roster
The ID roster lists student scores by the last five digits of the students’ SSN. The roster is listed in ascending order and contains the same information listed on the name roster with the exception of student name. When used in conjunction with the option to allow students to decide whether their score will be listed, this roster can be used for publicly posting student test scores.
Name/Error Roster
This roster is an alphabetical listing of the examinees which contains the following information:
a) name (last name, first name, middle initial),
b) ID number,
c) test score,
d) percent correct, and
e) a listing of each student’s incorrect responses
This roster is useful when changes must be made on the key. The listing of errors makes it easy to identify students who answered the item which is being regraded. If this option is selected, the simple Name Roster is not needed.
ID/Error Roster
The ID/Error Roster is a listing of student scores and errors which is sorted in ascending order by the last five digits of students’ SSN. It contains the same information as the Name/Error roster with the exception of the student’s names. If this option is selected, the simple ID Roster is not needed.
Individual Score Reports
This is a report that is designed to be given to each student in class. An individual score report contains the following information:
a) student name and ID number,
b) instructor name, course, and test date,
c) section number,
d) test form and test number,
e) test score and percent correct, and
f) a listing of the items answered incorrectly and the correct answers.
These reports are usually sorted by student name. If grade rosters are to be printed for each section, the individual score reports will be printed alphabetically by name within each section. You may request to have these reports sorted by students’ SSN.
Individual score reports can not always be printed with the rest of the test score report. The time required for printing these reports can be prohibitive. The operator has the responsibility to decide whether to print the reports in our office, or to send them to be printed at the CIS. Reports produced will be on cards and be
separated. NO INDIVIDUAL SCORE REPORTS WILL BE PRINTED
DURING THE FINAL EXAMINATION PERIOD.
Reports for Selected Students
The instructor has the option to allow the student to decide if his or her score will be listed on a publicly posted roster. This option should be chosen only after students have been informed to mark Option A on their answer sheet if they choose to have their scores posted. Only students who elect to have their scores posted will be listed on the ID and ID/error rosters. All names and IDs will be included on the name and name/error rosters and on individual score reports.
Report Rosters by Section
If answer sheets from more than one course section are graded together, rosters for each section may be requested. Name, ID, name/error rosters, ID/error rosters will all be reported by section. Sections will be identified by the values entered in the “Section” field of the students’ answer sheets. Summary statistics are reported at the end of each section’s roster.
OUTPUT OPTIONS
Information on student performance and test characteristics can be recorded on, 3.5” disk, or WYLBUR file. When an output file is requested, the scanning operator will ask you to name the files to be created. If a WYLBUR file is requested, you will have to supply the scanning operator with the WYLBUR user ID and password. A description of the variables and the file structure of each data file is detailed in Measurement and Research Services Memo 19. “Records Format for Data File Produced from Optically Scanned Sheets.”
Test Score File
This file contains an alphabetized listing of student score information and was designed to be used with gradebook programs. When grade rosters are listed by course section, the records in the test score file will be sorted alphabetically by name within each section. If this file is stored on floppy disk, the file name will have a TSD extension. For example TEST1.TSD. Measurement and Research Services can not run a gradebook program for you, but we can get you in contact with people who are currently using one. Also, a gradebook program for IBM compatible computers is available from the scanning operator.
Item Banking File
When requested, item analysis information for each test item will be placed in an output file. This data is useful when creating or updating a test item bank. When stored on disk, the file name will have an IBD extension.
Student Response File
This file contains all the information read from each student’s answer sheet, and should be requested if you are planning to rescore the test. The answer sheet information is stored in this file in order that the sheets were scanned. If the file is stored on disk, the file name will have a RSP extension. If you select the WYLBUR option, the file will be transferred to your WYLBUR files.
COLLECTING THE RESULTS
When dropping off a test please ask when your test will be ready to be picked up. You may call to be sure it is ready for you. Scanning and scoring’s direct phone number is 862-7379. Turn-around time is typically 24 hours. If you have a test which must be scored as soon as possible, please make an appointment one week in advance, so that arrangements can be made to get your test out quickly.
In order to pick up a test you will need an ID card. Student workers must present:
1. Permission to release test results to a student from the professor or T.A. on departmental
letterhead or note paper with the course(s), student’s name and professor’s signature. Photocopied professor’s signature is acceptable if an original signature and the title of a departmental representative appears.
2. Positive photo identification.
Students who are unable to provide a note and /or identification will not be allowed to pick up a test. Scanning staff will call the department if they have any questions about scanning couriers. Students will be asked to sign, date and write their ID number on the note. They also need to print their names, date, and pick up time on the sign in sheet.
Any sheets which cannot be read by the scanning machine will be clipped with a note explaining the reason. Our staff will not alter or change any sheet, even if it is rejected by the scanning machine.
Back to the top
Back to SCANNING
Back to MARS