Deteksi Tindakan Kecurangan Pada Ujian Daring Menggunakan Multimodal Data Berbasis Deep Learning

Detection of Cheating Behavior in Online Examinations Using Multimodal Data Based on Deep Learning

Kecurangan pada ujian daring menjadi tantangan karena keterbatasan pengawasan secara langsung. Penelitian ini bertujuan mengimplementasikan dan mengevaluasi pendekatan multimodal berbasis deep learning untuk mendeteksi tindakan kecurangan pada ujian daring menggunakan data video wearcam dan audio dari Online Exam Proctoring (OEP) Dataset. Pada modalitas video digunakan ResNet50, ResNet50V2, dan InceptionV3 sebagai feature extractor yang dipadukan dengan LSTM, BiLSTM, dan GRU. Pada modalitas audio digunakan ekstraksi fitur Mel-Frequency Cepstral Coefficients (MFCC) dan Wavelet yang kemudian diklasifikasikan menggunakan arsitektur yang sama. Hasil prediksi kedua modalitas digabungkan menggunakan metode ensemble learning berbasis voting yaitu weighted soft voting. Hasil eksperimen menunjukkan bahwa kombinasi weighted soft voting dengan bobot video sebesar 0.4 dan audio sebesar 0.6 menghasilkan performa terbaik pada klasifikasi biner dengan accuracy sebesar 0.97 dan rata-rata F1-score sebesar 0.97. Pada klasifikasi multikelas enam kelas, metode ensemble learning yang dilakukan memperoleh accuracy sebesar 0.96 dan rata-rata F1-score sebesar 0.91. Hasil penelitian menunjukkan bahwa integrasi data visual dan audio mampu meningkatkan kemampuan deteksi dan klasifikasi tindakan kecurangan pada ujian daring dibandingkan penggunaan modalitas tunggal.

Online exam cheating has become a significant challenge due to the limitations of direct supervision. This study aims to implement and evaluate a multimodal deep learning approach for detecting cheating behaviors in online examinations using wearcam video and audio data from the Online Exam Proctoring (OEP) Dataset. For the video modality, ResNet50, ResNet50V2, and InceptionV3 were employed as feature extractors and combined with LSTM, BiLSTM, and GRU architectures. For the audio modality, Mel-Frequency Cepstral Coefficients (MFCC) and Wavelet features were extracted and classified using the same architectures. The prediction results from both modalities were then integrated using a voting-based ensemble learning method, namely weighted soft voting. The experimental results indicate that the weighted soft voting method, assigning weights of 0.4 to the video modality and 0.6 to the audio modality, achieved the highest performance in binary classification, with an accuracy of 0.97 and an average F1-score of 0.97. For six-class multiclass classification, the proposed ensemble learning approach achieved an accuracy of 0.96 and an average F1-score of 0.91. These findings demonstrate that the integration of visual and audio data enhances the detection and classification of cheating behaviors in online examinations compared to single-modality approaches.