I think it has to be something that has (or had) a lot of radio play. Something that people will know, and like.
It also depends on the genre of music that your band plays. If your band plays stuff like Pantera, Metallica and such, then goes into a cover of "99 Luftballons", that's not good. FWIW, I've seen a band do that. It didn't go over well, at least with me.
Then there's those covers that people may not know unless they had the album. If they kick a$$, then you can still do them, because not all great songs get airplay. My band does a couple of those.
If you do cover songs, it's either:
A) Do them exactly like the recording, or
B) Do them with your own subtle twist, yet keep the feel of the original song. My old band used to start out with the intro to Metallica's "Sad But True", then break into Seether's "Gasoline", which kinda bummed some people out, but pleased others.
It's all in trial and error, and where you are playing at. Some bars dislike covers, others welcome them. I'm getting ready to play my first gig with my new band, at a bar where covers aren't really played (not that they're not welcome), yet we do mostly covers right now. It's going to be interesting.