I think pretty much anyone would get annoyed, that doesn't mean it's 'ableism'.
For example, I was in line to get food one day, and the line was decently long, so anyone in line had plenty of time to look at the menu and decide what they wanted to get (and the menu was written in enormous letters, so you could easily see if from a long distance away). The lady who was in front of me finally gets up to the counter to order, and THEN decides it's time for her to actually look at the menu and decide what she wants (and before you ask, no, she didn't have any questions about the menu to actual ask the employees).
Now, was I annoyed? Of course. But for all I know, maybe this lady is just really dumb, maybe she has actual brain damage, maybe she has some sort of disorder which makes her incapable of understanding why an action like this is annoying to other people. But it doesn't really matter, because it's still annoying me and all the other patrons behind me as well. It's not ableism: it's that people don't like waiting in lines for any longer than they have to.