Whenever I remove a game from Game Center I'd very much so like the Achievements to be wiped too. Why? Because I'm a perfectionist. But does that happen? No.
Lol.....just used it as an example. It was on my old iPod, and a second account. So nothing was harmed in the making of this screenshot I wouldn't take it off my main account though, I have nearly 20 achievements from it!
It also takes the achievement count off your account total. And yeah they are stored on their servers still, if you don't want the game enough to delete it, don't redownload it then! This is so peoples data and progress is still saved when you delete the app and then bring it back later, you still have the achievements. Not something to complain about.