Abstract

Reading comprehension is a key path to knowledge acquisition and is closely related to academic achievement. Therefore, in order to devise an appropriate intervention plan, the test taker's actual reading ability must be accurately assessed. The present study proposed a new scoring method that reflects the examinee's response time for each item in order to increase the reliability of reading comprehension tests. The new score was obtained from the raw score multiplied by the response time-weight. The response time-weight is the ratio of an individual examinee's response time to the mean response time of all the examinees. It was hypothesized that time-weighted scores would demonstrate higher alternate forms reliability than the raw scores. In Experiment 1, 73 university students completed two alternate tests with paragraph jigsaw puzzle items that required arranging paragraphs in the correct order. The correlation between the two time-weighted scores was significantly higher than that between the two raw scores. In Experiment 2, 77 participants took two tests where they read the passage and then solved multiple-answer items without the passage. The result showed the same pattern as in Experiment 1. The results of the two experiments suggest that the time-weighted scoring method can measure the examinees’ reading comprehension ability more consistently.