I've always wanted to hallucinate...
I have a friend who suffers from the same thing, except it's always when he wakes up for the morning anyway. He just can't move for a few seconds, or minutes sometimes.
Anyway, I think hearing would be scarier. First, your imagination would fill in the blanks. If I was seeing things, I think I'd pretty quickly figure out I was hallucinating.