This is true up to a point.
If you think otherwise, how long have you been a gamer? Do you remember what the first game you played was like?
For that matter, if you're old enough, you probably remember stuff like the NES controller. 4 buttons, and a directional control... And often you didn't even use all the buttons that much.
Contrast this to modern games with effectively 12 buttons and 4 analog axes of control and you can see the complexity has gone through the roof. (Especially with some games needing to resort to context sensitive actions, or multi-button combos to even map everything down to just 12 buttons)
If you cannot see how it may be difficult for new gamers to understand, then you've probably been blinded by your own level of experience, and cannot actually truly see things from the perspective of a new player.
That's not to say dumbing down games will help any, but it does show you need some less complex games to ease the barrier to entry.
If you start with something simple, and get the hang of that, you can probably cope with something more demanding later on.
But if you are expected to jump in the deep end right from the start, you may look at it and end up going "This is too hard, I give up" And go do something else instead...