바로가기메뉴

본문 바로가기 주메뉴 바로가기
 
 

logo

메뉴

상황판단검사에서 시나리오 효과를 통제한 탐색적 요인분석

Exploratory Factor Analysis on Situational Judgment Data after Controling for Scenario Effect

초록

이 글에서는 상황판단의 방식으로 제작된 검사 자료에서 탐색적 요인분석을 실시하는 방식을 제시하였다. 상황판단 방식은 심리학에서 80여년의 역사를 가진 검사제작 방식이고, 산업 및 조직 장면에서 지난 20여 년 동안에 선발용 검사로 정착되었지만 아직은 구성개념을 추정하는데 많은 어려움을 극복하지 못한 상태이다. 그것은 상황판단검사에서 문항점수에 문항에 대한 반응 뿐 아니라 시나리오 효과가 함께 포함되어 있고, 내용적인 구조를 탐색하는 단계에서 시나리오 효과를 제거하고서 요인분석을 하는 것에 매우 큰 어려움이 있었기 때문이다. 본 연구에서는 시나리오 효과를 제거하기 위하여 시나리오 내 문항들에서 측정오차 간 상관의 추정을 허용하는 탐색적 요인분석을 하였다. 즉, 시나리오 내 측정오차 간 상관을 추정하는 것과 나머지 부분에 대한 탐색적 요인분석을 하는 것을 동시에 수행함으로써, 분석 과정이 어느 한 쪽으로 편향되지 않는 방식인 탐색적 구조방정식 모형을 사용하였다. 그 결과로 시나리오 효과가 통제된 상태에서 요인이 탐색되어 앞으로 상황판단검사의 구성개념 타당화에 새로운 가능성을 제시하였다.

keywords
상황판단검사, 탐색적 요인분석, 방법효과, 시나리오 효과, 탐색적 구조방정식 모형, situational judgment test, exploratory factor analysis, method effect, scenario effect, exploratory structural equation modeling

Abstract

We demonstrated how to perform exploratory factor analysis on situational judgment data of creative personality. Situational judgment test(SJT) has a long history of 80 years and has settled down in industrial/organizational settings for the last 20 years. However, we have not overcome the problem of estimating construct because item scores in SJT contain method effect generated by the scenarios as well as response scores to the substantive questions. We applied Asparouhov and Muthen’s(2009) logic of explorary structural equation modeling(ESEM) to open an approach to estimating common factor structures after controling for the method effect generated by scenarios. Using ESEM, it is possible to specify measurement error correlations in the frame of structural equation modeling and to use exploratory approach to factor analysis on the remaining part of data. As a result we could estimate a four-factor structure on data of 40 items with eight scenarios measuring creative personality of college students.

keywords
상황판단검사, 탐색적 요인분석, 방법효과, 시나리오 효과, 탐색적 구조방정식 모형, situational judgment test, exploratory factor analysis, method effect, scenario effect, exploratory structural equation modeling

참고문헌

1.

강민우, 이순묵, 윤창영. (2005). 지시문과 채점방식에 따른 상황판단검사의 타당도 비교. 한국심리학회지: 산업 및 조직, 18(3), 547-565.

2.

김효선 (2013). 상황판단형 창의성격검사 제작 및 타당화. 성균관대학교 석사학위 청구논문.

3.

이상철, 이순묵, 조영일. (2003). 지필형 상황판단검사에 대한 비평적 고찰. 한국심리학회지: 산업 및 조직, 16(3), 129-154.

4.

이순묵 (2004). 상황역량 측정에서 상황에 대한 두 관점: 측정오차인가, 해석되어야 할 환경요인인가? 한국심리학회지: 산업 및 조직, 17(2), 243-264.

5.

이순묵 (1995). 요인분석 I. 서울: 학지사

6.

이순묵, 윤창영, 김영록. (2005). MTMM 자료분석에 대한 비평적 고찰. 한국심리학회지: 산업 및 조직, 18(3), 455-480.

7.

AERA, APA, & NCME (1999). Standards for Educational and Psychological Testing. Washington, DC: Authors.

8.

Asparouhov, T. & Muthen, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438.

9.

Becker, T. E. & Cote, J. A. (1994). Additive and multiplicative method effects in applied psychological research: An empirical assessment of three models. Journal of Management, 20, 625-641.

10.

Bovaird, J. A. & Koziol, N. A. (2012). Measurement models for ordered-categorical indicators. In Hoyle, R. H. (Ed.) Handbook of Structural Equation Modeling. NY: Guilford Press.

11.

Browne, M. W. (1979). The maximum-likelihood solution in inter-battery factor analysis. British Journal of Mathematical and Statistical Psychology, 32, 75-86.

12.

Browne, M. W. (1980). Factor analysis of multiple batteries by maximum likelihood. British Journal of Mathematical and Statistical Psychology, 33, 184-199.

13.

Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111-150.

14.

Browne, M. W. & Tateneni, K. (1997). Noniterative Estimation for the Multiple Battery Factor Analysis Model. Behaviometrika, 24(1), 3-18.

15.

Campbell, D. T. & Fiske, D. W. (1959). Convergent and Discriminant Validation by Multitrait-Multimethod Matrix. Psychological Bulletin, 56, 81-105.

16.

Chan, D. & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.

17.

Conway, J. M. (1996). Analysis and design of multitrait-multirater performance appraisal studies. Journal of Management, 22, 139-162.

18.

Conway, J. M. (1998). Estimation and uses of the proportion of method variance for multitraitmultimethod data. Organizational Research Methods, 1, 209-222.

19.

Gorsuch, R. L. (1983). Factor Analysis (2nd Ed.). Hillsdale, NJ: LEA.

20.

Hunter, J. E. (1983). A causal analysis of cognitive ability, job knowledge, job performance. Journal of Vocational Behavior, 29, 340-362.

21.

James, L. R., Demaree, R. G., Mulaik, S. A., & Ladd, R. T. (1992). Validity generalization in the context of situational models. Journal of Applied Psychology, 77, 3-14.

22.

Landy, F. J. & Shankster, L. J. (1994). Personnel selection and placement. Annual Review of Psychology, 45, 261-296.

23.

Lee, G., Brennan, R. L., & Frisbie, D. A. (2000). Incorporating the testlet concept in test score analyses. Educational Measurement: Issues and Practice, 19, 9-15.

24.

Lievens, F. & Conway, J. M (2001). Dimension and Exercise Variance in Assessment Center Scores: A large-scale evaluation of multitrait multimethod studies. Journal of Applied Psychology, 86, 1202-1222.

25.

Lowry, P. E. (1997). The assessment center process: New directions.Journal of Social Behavior and Personality, 12, 53-62.

26.

MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111, 490-504.

27.

Marsh, H. W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and few solutions. Applied Psychological Measurement, 13, 335-361.

28.

Marsh, H. W. & Bailey, M. (1991). Confirmatory factor analyses of multitrait-multimethod data: comparison of the behavior of alternative models. Applied Psychological Measurement, 15, 47-70.

29.

McDaniel, M. A., Hartman, N. S., & Grubb III, W. L. (2003). Situational Judgment Tests, Knowledge, Behavioral Tendency. Paper presented at the 18th Annual Conference of the Society for Industrial and Organizational Psychology. Orlando, April, 2003.

30.

McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., & Campion, M. A. (2001). Use of Situational Judgment Tests to Predict Job Performance: A Clarification of the Literature. Journal of Applied Psychology, 86, 730-740.

31.

McDaniel, M. A., Yost, A. P., Ludwick, M. H., Hense, R. C., & Hartman, N. S. (2004). Incremental validity of a situational judgment test. Paper presented at the 19th Annual Conference of the Society for Industrial and Organizational Psychology. Chicago. April, 2004.

32.

Montanelli, R. G. Jr. & Humphreys, L. G. (1976). Latent Roots of Random Data Correlation Matrices with Squared Multiple Correlation on the Diagonal: A Monte Carlo Study. Psychometrika, 41, 341-347.

33.

Moss, F. A. (1926). Do you know how to get along with people? Why some people get ahead in the world while others do not. Scientific American, 135, 26-27.

34.

Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An Alternative Selection Procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647.

35.

Mulaik, S. A. (2010). Foundations of Factor Analysis. New York, NJ: Chapman & Hall/CRC.

36.

Mullins, M. E. & Schmitt, N. (1998). Situational judgment testing: Will the real constructs please present themselves? Paper presented at the 13th Annual Conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.

37.

Robertson, I., Gratton, L., & Sharpley, D. (1989). The psychometric properties of managerial assessment centres: Dimensions into exercises won’t go. Journal of Occupational Psychology, 60, 187-195.

38.

Smith, K. C. & McDaniel, M. A. (1998). Criterion and construct validity evidence for a situational judgment measure. Paper presented at the 13th Annual Convention of the Society for Industrial and Organizational Psychology, Dallas, TX.

39.

Sternberg, R. J., Conway, B. E., Ketron, J. L., & Bernstein, M. (1981). People's Conceptions of Intelligence. Journal of Personality and SocialPsychology, 41, 37-55

40.

Wagner, R. K. & Sternberg, R. J. (1985). Practical intelligence in real-world pursuits: The role of tacit knowledge. Journal of Personality and Social Psychology, 49(2), 436-458.

41.

Wagner, R. K. & Sternberg, R. J. (1993). TKIM: The commonsense manager, user manual. New York: Hartcourt Brace Jovanovich, Inc.

42.

Weekley, J. A. & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25-49.

43.

Weekley, J. A. & Jones, C. (1999). Further Studies of Situational Tests. Personnel Psychology, 52, 679-700.

logo