Well, there'd be no apocalypse, or anything. I mean, the world got along fine before they came along.
This alternate world WOULD be, however, severely lagging behind our world in terms of technology of many varieties, as well as in artistic and economic development.
I once wrote an (admittedly condensed) essay about the societal contributions of the interactive media.
That said, it seems a rather impossible situation; as media has evolved artistically and technologically its naturally grown to encompass more and more of our cognitive processes. First we saw situations explained, then we listened to them explained, then we saw them played out, then we saw AND heard them played out, then we played them out ourselves. Interaction was always a logical conclusion to media development, and I doubt it could be avoided barring some massive technology-altering world event.