Finding a decent MP3 player is actually a lot harder than it seems. Sure there is a lot of choice but partly that's the problem, there's hundreds of different players and some are "integrated media centers" by which they mean they can control your TV, display a low-resolution lightshow when listening to music, lookup the cover art for the album you're listening to but more often than not crash.
I needed a player that was simple, allowed MP3's to be mixed with Ogg files, was preferably open source and didn't get too hung up on playlists. It also had to be pretty stable. Zinf has been a favourite since wayback when but it can be a pain to switch between music based on the file labels. It is also not being developed in Windows any more. After that came wxMusik which is pretty good but is very unstable (it crashes if you shutdown Windows without exiting the app). The File Tag editing is particularly good I feel.
I have finally settled on MusikCube, very stable, strong codec support and Tag editing, minimal interface (influenced by iTunes but not, for example, obsessed with the concept of playlists). It also claims to be able to play CDs (unlike most file players). So far I'm very impressed.