Pilot study on the digitalization of the national qualification exam for Korean engineers

Abstract

The Human Resources Development Service of Korea developed a digital exam for five representative engineering categories and conducted a pilot study comparing the findings with the paper-and-pencil exam results from the last three years. This study aimed to compare the test efficiency between digital and paper-and-pencil examinations. A digital examination was conducted with 93 participants through two rounds of test-status measurements. We collected the following data: early check-out rate, pass rate, and the average total response time per grade. The composition ratio of each question type in the digital exam was determined per the “Exam Operation Regulation” and selected from a databank spanning five years by experienced instructors. The early check-out rate of the digital exam at the Engineer level was 90% (percentage points), approximately 15%p higher than that for the paper-and-pencil exam, but the pass rate was low (9.8%p); the response time per question for the digital exam was approximately 0.2 min shorter. The results for both exams were similar in the Industrial Engineer category. The early check-out rate did not positively affect the exam pass rate, but the time taken depended on the number and characteristics of the subjects. It is necessary to improve the simplification and convenience of digital examinations, optimize test environments, and establish operation and management systems in line with the change in the Korean Technical Qualification Framework. We also suggest a gradual shift to digital examination by developing questions and tools using technologies to accurately measure individual abilities.