Haven't played it yet, but according to a friend who's already beaten it, the rats are indeed the most dangerous thing in that game.
However, he's also mentioned something that applies to the other argument: the "It's never just a rat!" side. That being that there were quite a few instances in which he'd do something deserving of a "WTF?! GUARDS! ATTACK!" only to have people just carry on as though they thought it was indeed just a rat. One case in particular had him possessing a rat, exploring a bit, then poofing back to normal in front of a large crowd, and no one seemed to care at all. :3