I think you are right, it's definitely something that comes with age.
I used to get bullied at school and for a long time it really affected me. It stopped around fourth year but I hadn't forgotten about it and thinking back made me feel bad. Now, looking back, even when I imagine the worst comments and the worst situations I found myself in, I can't evoke any sort of feeling from the memory whatsoever.
You get to a stage where you realise that everyone is different, no-one is perfect and everyone has their faults. You also come to appreciate that there are people in life that are in much worse situations than yourself, no matter how bad you may feel it got or gets at times. In turn I think you come to appreciate yourself and what you have that little bit more.