The increase in medical imaging and, consequently, the growing workload of radiologists requires workflow optimization. The interaction between radiologists and artificial intelligence is a promising area because it can improve radiologist productivity and reduce diagnostic errors. In this study, we describe an eye-tracking experiment with four practicing radiologists who cumulatively examined 1,000 unique chest X-ray images that included samples with multiple pathologies. We demonstrate the feasibility of a self-supervised approach for gazebased data for problems with a small amount of labeled data. We achieve an accuracy of 64.7% for binary classification of radiologist diagnostic error with 218 samples per label. The proposed approach can be used in the system for assessing the need for a second opinion to improve the quality of diagnostics.