Most early PS1 games used redbook audio, and thus you could stick them in a CD player and listen to the music. Some later ones also used redbook audio as well, but there aren't very many made after 97'/98' that did so (at lest from what I've encountered).
Also, pretty much all CD-based consoles from the Sega CD/Mega CD-era and before used redbook audio, so if you have games from those consoles you could stick them in a CD player listen to the music as well. The majority of Sega Saturn games also were playable (music wise) in a CD player, although there were the occasional ones that were not (Mega Man 8 and Myst, being the two specific examples I've found).