Sgt. Sykes said:
I wonder, where did you get that figure, that 5-50 mil years should be enough to colonize the galaxy?
It comes from Fermi [http://abyss.uoregon.edu/~js/cosmo/lectures/lec28.html] himself, see the linked lecture. Based on reasonable limits like that the Speed of Light cannot be beaten you should be able to explore with a Von Neumann Probe every single star in our solar system in about 4 million years. From there it's just selecting the best places to go first. At a minimum our galaxy should be cluttered with Von Neumann probes from other intelligent species like ours.
However, we don't see these in our own system. You'd think that the earth would actually be flagged as interesting if these probes existed. This removes the argument that Space is just TOO BIG. Anyone who uses the TOO BIG argument is failing to grasp what the paradox presents.
The basis of the fermi paradox is that one of our base assumptions is completely wrong which presents one of our assumptions is actually a great filter. The assumption which is most likely wrong is that something is Ordinary about the condition of life.
This could be that ...
1) Life might be extremely rare to the point were it boarders unique.
2) Multicellular life might be extremely rare bordering on unique.
3) Intelligent life like ours might be extremely rare bordering on unique.
4) Interstellar travel may have a barrier yet undiscovered making travel impossible.
If we ever find life in our solar system that evolved separately away from the Earth like on Titan and Europa then we can conclude that 1 and 2 are false. If 1 and 2 are false it's easy to see that 3 should also be false, but that would still be an assumption.
Some people are desperate to prove 1 - 3 are false because one of the valid hypothesis solutions to the Fermi Paradox is the God Hypothesis. But even that falls to the simple logic of why make a universe so big and only put one planet with life in it.
4 is the most likely the be the actual issue, but if it's not that then we fall into the Zoo hypothesis which falls pray to other failings.
An off shoot of the Zoo Hypothesis is the Petri Dish experiment. Ether done by God, or by Alien Life. Alien's, or God would be interested in the Earth as an experiment to observe the evolution of life in general. Evidence for this is in the Nemesis Hypothesis. We have a scheduled Mass Extinction event every 26 Million Years. It's part of what Mass Effect was based on but they shrunk the time scale down to thousands of years. Initially it was thought to be a Star but a few years ago it was proven that the periodic nature of the extinctions exceed known limits that would be on a near by star. Even if the star existed it should have had its orbit in the galaxy altered slightly over the last 700 million years.
Yet others claim the extinction events are a miss read of the data, but it's clearly still debatable ether way.