In all honesty, IMO, its because they've got access to far more powerful engines that don't limit them as much, and this leads to a focus on the superfluous rather than the core of the game.
You've pulled up some very different game series for your example; Star Citizen: SQ42 and Mass Effect alone are very different games, let alone comparing to Starfield or No Man's Sky. If we're looking at this as a phenomena across all space games, rather than just a specific sub-genre, the cause is going to be common to all space games. That cause, IMO, is scope creep - and its a problem that does affect non-space games, however there's just more of those so we don't notice as much.
Older games were far more limited - by hardware, technology, development time & budget, and so on. This is across all genres. Newer games are able to achieve significantly more, with larger development teams, longer development cycles in some cases, significantly better hardware, better rendering, procedural generation, and game development technology, and so on. All genres have tried to make use of this as best they can, however space games are in a unique case of how extreme they can take the lack of limits. In your classic RPG, going open world might mean you're limited to a single country, or continent. In space, the scope can creep to have the country, continent, planet, star system, and a good chunk of the galaxy too, all to a similar level of detail. The issues with a lot of games are far more pronounced with space games.
This increase in scope has led to a number of trends among modern gaming that contribute to a number of generally lower quality releases, especially in space games. Open world and non-linear games have taken over as they are where this new technology can really be shown off, however not all games, or developers, are equipped to properly handle these trends even on a smaller scale, let alone a larger one. It is very easy to have a very large world that feels empty, lifeless, directionless, boring, or many other similar adjectives. The problem is with both creating that much unique content so that you can't instantly latch onto the procedural generation patterns, while also structuring that much content into a compelling experience, rather than just throwing it onto the world.
This problem is also exaccerbated by the homogenisation of genres occuring in the modern era; every game must have RPG systems, open world exploration, non-linear segments, base building, crafting, and so on. Space, again, offers an open canvas for all of these to be thrown onto, and developing 20 different systems is going to mean they are each far less polished, far less engaging, and the whole experience is far less focused, than developing 3-5 systems.
Different games take different approaches to this problem, and often the more limited approaches IMO turn out better. As an example, Mass Effect 1 vs Mass Effect Andromeda. Both games had procedurally generated planets.
Mass Effect One didn't have the resources to really flesh them out, and thus put only a scattering of side content across some very basic, ok-sized procedural areas. The gameplay did not focus on these sections, and usually directed the player to engage with them in a structured way, if at all, before returning to the core structure of the game. You were not intended to spend a ton of time roaming around these planets between missions, but you could if and when you wanted to.
Mass Effect Andromeda did have the resources to flesh out the procedural zones. And so they did, consuming a lot of development resources in the process. This is where the focus on creating content went, and thus where the game focused the player to spend their time. Pure linear sections were much rarer, and instead you often had objectives scattered across the procedural zone, and had to trek through it to continue with a linear plotline. This put the focus on the procedural zone, and in doing so reduced the ability of the player to control when they engaged with different types of content; you were always engaged with the slower exploratory content, and locking into a more focused narrative section of content could take far longer. Further, trying to produce so much semi-handcrafted content clearly stretched the team thin. This resulted in a game that did not play to the narrative strengths of Bioware, and instead diluted them. This is without touching on any franchise-specific issues with the game, like how ME3's endings forced a large distance between 4 and the other trilogy to avoid narrative conflicts.
Starfield seems to have similar problems, but from the other side. People complain on the one hand about loading screens, and the game not being truly open world. Clearly, the expectation is for such games to have this huge glut of continuous map size. On the other hand, there are also a lot of complaints about repetitive content already that would only get worse with more game world for the developers to try and fill. Despite their previous experience, Bethesda were not equipped to create this scale of open world, and in increasing the scale they lost some of the freedom, and a lot of the attention to detail and density, that people have liked about previous Bethesda titles.
No Mans Sky was in a similar situation at launch; it created too much stuff, but the game itself wasn't engaging to be in. For some it is now fixed, however that really depends on what you want out of the genre.
Star Citizen takes another approach that clearly also has its problems; they claim to be fully filling out this whole world with all the content necessary and having it all done properly in an engaging manner. Its been in development for over a decade and is still nowhere even close to done. Honestly, it probably never will be done.
The great space games of the past had something in common IMO; a focused experience. Some focused on dogfights in a linear, mission-based structure. Some focused on trading and basic exploration. Some were narrative RPGs. Some were focused on being open sandboxes in space - usually ignoring the planets to make this possible. Each game, however, was technologically limited from expanding the scope too far, and focused on the core appeal of their game. Modern games, especially space games, often try to do it all; procedural planets, procedural star systems, thousands of them, crafting, often base building, a core story supposed to be compelling, freedom to do whatever you want - and so on. If developers focused on a specific experience, rather than trying to combine everything, we'd have better games in general IMO.
That said, some great space games have come out in the 2010s. Outer Wilds for one is great. But a lot of the big, do everything space games have been disappointing - which IMO only supports the idea that the problem is trying to do everything. Narrow the scope, and you can get a much better experience.
