Yes you are right. I gained back some of my weight but I am actually smaller now than when I was this weight before. The difference being my activity levels!Do you work out and exercise? I know you can't "spot" reduce fat, but I believe a person that has worked out throughout weightloss tends to take a lower dress size than those who haven't.
I could be talking complete rubbish here, so feel free to correct me if I'm wrong.