Well, there are some Visual Novels with literally no gameplay, not even route choices, just linear text and pictures, and they still get identified as "games" that you "play".
Though that verb, "play" is also used for lot's of non-interactive things: to play a record, to play a certain number on the guitar, etc. It's sort of a catch-all term.