Multitrait-Multimethod Technique for Construct Validity: A Case Study for Instruments of Critical Thinking Ability and Creative Thinking Ability in Programming Course

Every psychological measuring instrument has a certain feature or stimulus introduced specifically to represent the trait it intends to measure. A measurement instrument, whether in the form of a test, rating scale, or other devices, almost certainly elicits systematic variance in responses due to the two groups of features. If an irrelevant method variance contributes to the obtained score, then the score is considered invalid. Therefore, it is necessary to test the construct validity of the instrument. This study applied the multitrait-multimethod test to test the validity of the constructs with case studies on creative thinking ability test instruments and critical thinking skills tests in a programming course. Initially, the instrument's content validity was tested qualitatively and quantitatively by applying the Lawshe formula. The multitrait-multimethod matrix showed that the convergent validity was met because the reliability values on the diagonals were all high. Furthermore, the multitrait-multimethod matrix also showed that the discriminant validity was also fulfilled because the monotrait-heteromethod cell values showed a high correlation. In contrast, the heterotrait-monomethod cell values showed a low correlation value. The findings above indicated that the creative thinking ability test and critical thinking ability test, which were developed in the form of a description test and a multiple-choice test, have met the construct validity, respectively.