Innovation is risky - some things succeed but most fail, and investors hate failure. So you can understand why big firms with big investors value safety. Safety means low innovation: cautious advances based on what's been observed to already work.
But then what do these big firms have, if they don't innovate? What they have is the ability to splash insane quantities of money on something epic. Most of these games have scope a step above and beyond the competition that near-limitless resources can provide, and it often works: even games with substantial flaws like Starfield flourish. That is really their selling point, their USP. Without it, they're just making middle of the road, formulaic games indistinguishable from all the others out there.
And bless, those dev teams might have been hot stuff when the series started 20 years back, but they just aren't geared to innovating any more, so they probably couldn't even if they wanted to.
In part I'd say.
The other part is the push for quick returns from investors and big returns and the risk of live service games is seen as lower than the potential returns.
Gaming apparently is losing investors because the bubble has burst and the kind of shady investors who were throwing money at it mostly because of pandemic era growth and moved on to the next get rich quick ideas as they failed to combine gaming and NFTs / Crypto. It's gambling, A.I., Porn and Crypto now are the big areas they want to invest in.
Big companies are struggling with this loss still chasing the big investors.
Add in the current climate in many market fields of execs bonuses being "Raise profits by x amount get a bonus" and so execs are incentivised to boost profit they hammer it for a year, get the big pay out then leave without people realising they did the equivalent of heating heating a building up by setting fire to the bottom few floors rather than anything sustainable.
Sustainability is being lost.
The example I like to go back to is Disney, they made admittedly most of their money from their theme park but a decent chuck was from selling their films again and again. They're impressive in the fact they refined a lot of the corporate tactics used in other areas today. They regularly re-released their films to buy, but not in huge numbers enough it would look like shops were selling out which made people buy for fear they'd struggle to find the film for their kids and it let them sell the film and bring them back round again to remind people the films and IP existed which let them sell other merch too. They were built on their merch and IP and being able to keep selling the stuff in part.
Companies forget this and in the age of infinite availability companies could be making a horde of titles that just keep selling or they market as classics of yester year and get people keeping buying them but companies don't want that because it takes years and time to build up that library and it's not massive potential returns quickly.