Defects such as dents on the rail surface might induce severe rolling contact fatigue (RCF) and then threaten the running safety of railway vehicles. This study aims to explore the initiation and propagation of RCF cracks around dents with different sizes through observing both surfaces and cross-sections of the defective rail. The experiments are conducted on a wheel/rail twin-disc machine. Results indicate that the microstructure evolution process around the dent can be divided into three main stages, namely crack initiation, crack propagation and dent removal. Dents in different sizes act very similarly in each stage, including the position, the crack growth path and the final appearance after the dent has been fully worn off. However, dents in different sizes cause different lifetimes of crack initiation and different lasting times of dents at each stage. For the lager dent, cracks initiate later but propagate for a longer time period, so the dent size plays an important role in the RCF evolution process.