When to replace rims due to scoring ?

How do you know when a rim needs to be replaced due to scoring ? It is difficult for me to tell how
thick the rim walls are and how concerned I should be about the scoring.

Thanks

Rich