The increasingly complex modern railways require advanced management systems to automate costly and time consuming maintenance processes. Such systems incorporate detailed maps of the railway assets such as signs, signals, control boxes etc. In addition, railway maps are the foundation of railway simulators used for training locomotive operators. Currently, development and update of these maps is carried out by trained personnel through manual inspection of multi-sensor data. Recent advances in machine learning and computer vision have paved the way for designing deep learning detection models able to achieve very high accuracies in challenging tasks. The performance of these methods has attracted the interest of the industry providing reliable automatic solutions to complex and large scale problems such as railway mapping. This dissertation proposes a pipeline that automates two main tasks of mapping development:(i) Image based object detection and (ii) 3D object localisation. For the detection task, we apply state-of-the-art deep learning detection algorithms in panoramic images. Then, we combine the image based detections with known 3D camera positions to estimate the 3D positions of the objects.