“That train has already left the station!” : improving the fidelity of a freight locomotive simulator at post-deployment

Fidelity evaluations are an important part of simulation design and critical for identifying any gaps in correspondence that can creep into a simulated environment. When conducted by experts, they can enhance the correlation between the simulation and the real environment, and provide some constructive insights attuned to the intended goals of the simulator. However, fidelity evaluations may become overwhelmed by the technical focus in the commissioning and acceptance processes, and too often, there is a tendency to overlook them or undermine their integrity, in favour of remaining within the scope of the original specification. This paper presents a case study of a fidelity evaluation that was conducted systematically during the post-deployment stages of a generic freight locomotive simulator. The evaluation was undertaken in a collaborative framework that converged the respective domain, task, simulation and human factors-based expertise in a team of evaluators, and considered the physical, functional and task-based components of fidelity. Further, the evaluation examined the extent of the simulator’s tractability in terms of the target users (researchers) and interrogated the scope for scenario development. The findings revealed several opportunities for improving the simulator’s fidelity, particularly for training, but also identified a number of striking deficiencies in its underlying architecture. This paper discusses the outcomes of the audit in relation to expectations of simulator fidelity in the developer and user, and the tension encountered when adjusting these post-deployment. Lastly, it provides some clarification between what to improve and what can be improved, when the proverbial train has, for all intents and purposes, left the station.