Many cosmological arguments (not all) argue that everything that begins to exist must have a cause -- this is basically the principle of causality. But this chain of causality cannot be extended infinitely into the past for two reasons: 1) an infinite amount cannot exist in reality; therefore there must be a first cause that by definition is not the effect of a previous cause itself. 2) We have empirical evidence that the universe itself began to exist (Big Bang cosmology) and is therefore finite; therefore there must be something that exists independently of the universe that brought it into existence. In both cases we end up with something that sounds an awful lot like God.
The objection of some atheists is to ask, "Well then who created God? If everything requires a cause, then God would require a cause too right?"
The response to this should be obvious. Cosmological arguments do not claim that everything that exists requires a cause because there's simply nothing about sheer existence that would require a cause. What philosophers have claimed is that everything that begins to exist requires a cause. It's the "beginning" part that brings causality into play, not the "existing" part. So when we say that "Being does not arise from non-being," the focus is not on the "being" but on the "arising"; it's the latter that necessitates a cause, not the former.
The objection may then be put the other way around: "If God doesn't require a cause, why does the universe? Why couldn't the universe be this first cause?" Again, the response should be obvious: because it began to exist. That's the argument. Of course, you could claim that the argument fails or present an argument of your own that the universe didn't really begin to exist. But to simply say, "Well if God doesn't require a cause, why does the universe?" just ignores the argument that has been presented. It certainly doesn't answer it.
Thus, this objection is a complete straw man. It's a misstatement of the claims being made, a misstatement made in order to raise a bogus objection to certain cosmological arguments. The fact that otherwise brilliant people (such as Bertrand Russell) think this is a good objection only demonstrates that they didn't even hear the argument in the first place.
The reason "Who then created God?" is not a good objection is because the cosmological argument already addresses that issue. The whole point of these arguments is that there must be a cause that is not an effect of a previous cause itself. To ask why this first cause is this way is to ignore the argument that has just been made that this first cause is this way. Of course, showing that something is the case is not the same thing as showing why it is the case. (I would argue that one can answer the "why" question, but that's another issue.) But the atheist is claiming -- at least with this objection -- that unless the argument proves why something is the case, it doesn't prove that it's the case. This is obviously false.
So, for example, I could say that the ratio of a circle's circumference to its diameter equals pi (in Euclidean space). I could then prove this mathematically. The atheist objection would be "Why should this ratio equal pi?" The answer would be, "It does. Here's the proof again." The atheist would then object "Your mathematical proof doesn't explain why this ratio equals pi." And again, the answer would be, "It does equal pi. Here's the proof again." "But why should it be this way?" "It is this way. Here's the proof again." Etc. It reminds me of a Calvin and Hobbes cartoon where Calvin balks at his math homework. You put two numbers together and they magically become some third number. No one can say how or why it happens, you just have to accept it on faith. "As a math atheist, I should be excused from this."
So when an atheist asks why God should be excused from having to have a cause, the answer is simply to repeat the argument, which (allegedly) demonstrates that there must be a first cause that does not have a cause itself. Perhaps the argument fails to demonstrate this, but the objection that God would then require a cause doesn't even address it.
