Your railroad can showcase track weathering as a result of poorer care on the spurs and sidings. You may find more rust on these sidings which usually is light gray or grayish brown, whereas the mainline will reflect a good condition with dark brown or black in color.
When it comes to track weathering you might find very little ballast on the siding but you’re more likely to find cylinders. The mainline on the other hand will have regular purchased ballast from specific quarries. It is best to use granite or limestone as ballast on the railroad wherever possible. If you could consider these details as you create your prototype you can end up with a very realistic and well crafted railroad.