Detection of lung diseases from chest X-rays has been of great interest from the research community during the last decade. Despite the existence of large annotated public databases, computer-aided diagnostic solutions still fail on challenging rare abnormality cases. In this study, we investigated the paradigm of combining the analysis of chest X-rays and physician gaze patterns during the analysis of these X-rays to improve the computerized diagnostic accuracy. Tobii Eye Tracker 4C has been mounted to a physician workstation and his eye movements were recorded during the analysis of 400 chest X-rays in two days of work. The X-rays have been sampled from CheXpert, RSNA, and SIIM-ACR public databases labeled with 14 different pathology types. The task was formulated as a binary classification problem. A ResNet34-based neural network has been trained to map the input chest X-ray with the output physician gaze map and binary pathology label. The proposed network improved the diagnostic accuracy to 0.714 of the area under receiving operator curve (AUC) from 0.681 AUC obtained for the same ResNet34 trained to generate binary pathology labels alone. The proposed study has demonstrated the potential benefits of using gaze information in computerized diagnostic solutions.