Abstract

Difference score analysis and analysis of covariance (ANCOVA) are commonly used to evaluate treatment effects in pretest-posttest control group designs. However, the two methods do not always yield equivalent results and may produce biased estimates under certain conditions. This study conducted a simulation to compare their performance across various conditions. Factors such as group assignment method, score reliability, score stability, treatment effect size, and sample size were systematically manipulated. The two methods were compared in terms of estimation bias, Type I error rate, and statistical power. Under random assignment, both methods produced unbiased estimates, with ANCOVA showing slightly greater power. Under non-random assignment conditions, the results varied depending on the assignment mechanism. ANCOVA performed better under pretest-score-based assignment, whereas difference score analysis was more accurate under nonequivalent group designs. When assignment was based on true pretest scores probabilistically, both methods produced biased estimates. Additionally, score reliability and stability influenced the results within each assignment condition. These findings highlight the importance of considering group assignment and measurement characteristics when selecting an appropriate analysis method.