Where did Inquisition go wrong?
Lots of places.
For one, the use of Frostbyte as the engine. EAs push from memory, and whilst it did great things for the graphical look of the game, it supposedly caused problems with things like the tactical camera - hence why its basically useless other than as a 'pause' function.
The second issue comes in from their goals with the game. It is still very much intended as an action RPG rather than a tactical RPG. There isn't a lot of coherency to most battles, and most of the time they consist of just spamming all your abilities whenever you're able to, and occasionally micromanaging your mage away from the enemy. The enemy don't really use tactics [Not that they really did in Origins either], and you don't have to use tactics. This was more of their goal than their failing, I believe. The aim was to very much keep DA2s "Push a button something awesome happens" alive, and pander to a broader audience, however they also then tried to fit in some tactical RPG elements so they could at least try and impress their original fanbase. Since those elements were minimal and had the least effort of anything put into them, however, and thus it kind of fell flat.
Then there's the requisition quests, which is a case of good intentions but flawed execution. The idea behind them would have been to allow players to gain Power quickly if they were short on it. However, it takes longer to perform a requisition quest than it does to perform any other side quest to get Power most of the time, especially in cases like the Obsidian requirements for the Dwarven Puzzle box - where you couldn't get the second requisition quest until halfway through the game. It just introduced pointless grind. There were far better ways to execute this, some performed within the same game [Selling power for money]. Other examples include the shards, which were likely to encourage exploration - but were placed in such a way that rather than exploring you had platforming adventure's and the "Long, intended road vs quick glitch road" problem instead. Its just a poor execution of ok ideas.
Probably the biggest problem with DA:I however, was that Bioware cut away from what they are specialised at and do well at. Bioware always make focused, story driven games. Or at least used to. With Inquisition, they moved to a Sandbox, and in doing so they tried to figure out how to fill up that sandbox with content. How? The same way MMOs do of course - lots of meaningless quests, as you have to fill the whole area with stuff, and you don't have enough time to make it all good content, so better just throw lots of really basic content with an even amount of effort put into each of it. Even worse, they made it largely compulsory in order to gain power needed to move on with the main quests. Bioware generally write characters that are trope driven, but they write them well. In Inquisition, they tried to move away from the generic trope driven characters they're used to, and write new characters... which are still trope driven but different tropes. I.E: Dorian, who is very classic 'homosexual family problems' trope. Handled Ok, but still somewhat clumsily, and going for inclusivity... whilst using a stereotyped trope... isn't exactly the best method. Then you've got Sera, who tried to mimic, as others have put it, the 'Random is funny' Youtube personality trope. They then tried to give her some depth, however it ended up making it feel schizophrenic instead. Sometimes she could be considerate and caring, other times she would actively try to insult people just for the fun of it. Bioware left their area of expertise a bit too much in Inquisition, and put minimal effort into the areas they were good at, and that left the overall product a bit... worse for wear.
There's also issues with choices and such inherent to Bioware, that are the outcome of the design decision to only make stuff that will be seen by everyone. A number of devs feel that making content that half your playerbase won't see is a waste of time [I.E: The ending of any videogame ever]. This is the reasoning behind why you don't get vastly different outcomes for choices in Bioware games. Siding with the Templars, siding with the Mages. A couple of missions, and you're done. 1 main mission, and a few war map missions more specifically. Outside of that, everything is the exact same. Why? Because if you made two vastly different routes, like TW2 did for siding with Roche or Iorveth, then most of your players will only end up seeing one route, and its therefore a waste of your development time to make both. I can respect the reasoning, but in practice I don't feel it holds up.
Dragon Age Inquisition was a fairly good game, but it did have a lot of faults. Really we can only speculate as to why that happened, but I think I've given some reasonable reasons as to why. By and large, most of it can be summed up by Bioware trying to appeal to too many audiences at once, resulting in a mis-match of different parts that don't form a fully cohesive whole, and that feels unfocused, as opposed to their more focused games made to appeal to a specific audience - like DA:O and ME1.