Some people believe that there is a strong connection between atheism and Anarchism, others believe there is no such connection. Certainly it is true that Anarchist thinkers have been strongly unsympathetic to organized religion and tend to be atheists, but the reverse has not been true. Why is that? And what is the connection?
Anarchism is a position about society and undesirable structures within it (that is to say, the belief that hierarchies are unjustified). Atheism, on the other hand, is an ontological position (about the existence of gods). But if you ask atheists why they see the need to label themselves and discuss atheism, they will almost invariably answer you that Christian dogma infiltrating itself in the schools and politics is the main issue. Therefore atheism does reduce itself to a social issue to some extent.
I think the issue that links them both, therefore, is the concept of authority. Atheists reject both concepts of authority in Christianity: the authority of God over man, and the authority of organized religion against their believers and society as a whole. Anarchists merely reject other forms of authority which any individual atheist may or may not reject.
And in a sense, this does show Anarchism and atheism as similar, insofar as most arguments an atheist uses against Christian authority can also be used against other forms of authority. I have already explained one case of this, the impossibility of transferring exterior obligation. Atheists may, for instance, argue that Christian principles are unethical and clash with modern sensibilities; Anarchists often argue the same thing about laws. Atheists argue that a supposedly good god is incompatible with the existence of evil in the world; Anarchists argue that a supposedly good State/economy/etc is incompatible with all the evils that it creates in the world. Atheists argue that a specific religious belief system should not be imposed on the whole of society; Anarchists argue the same thing about political belief systems. This similarity arises from the fact that the arguments are based on what is specific to authority: the disconnected sense of ethics, the factionalism, the attempt to control society, and so on.
There are also resemblances in the way both approach epistemology. Atheists are very concerned with teaching others how to approach supernatural claims and how to distinguish true from false beliefs. They seek to uphold a rational, evidence-based, skeptic attitude towards these claims. Anarchists have a very similar concern towards economic or political claims, and seek to uphold a rational, evidence-based, skeptic attitude towards them. Both are extremely suspicious of arbitrariness, factionalism, presuppositionalism (attempts to make a belief system true by default), Special Pleading, and the attempt to put teleology as some kind of default which underlies natural processes.
The very latter point is equally important, insofar as both atheism and Anarchism believe in what I would call emergentism, the worldview which states that properties of units at one level of discourse are the result of the interactions between units at some lower level of discourse (e.g. that the properties of societies are the result of the interactions between individuals, that the properties of organisms are the result of the interactions between organs or cells, and so on). Atheists and Anarchists believe that, whatever the question, positing intervention from some higher power is not a proper answer, and only leaves other things unexplained. They believe that, if an explanation is to be found, it must be found in reality, not in dogmas.
As I said, all of these are derived from the nature of authority. The ontological claims, on the other hand, are wildly different. Atheists argue against the concept of God on the basis of it not existing, while Anarchists do not argue that the State and capitalism themselves do not exist. The morality creator and enforcer, in this case, actually does exist. What is fictional in archism are the geo-political entities and concepts which are enforced by the capital-democratic system. In atheism, the concepts which are enforced by God (that is to say, the priesthood and the True Believers, by proxy) are very much real.
In both cases, though, the deep level of unreality is burned under layers of rhetoric and mental or physical violence. In the case of religion, mental violence from childhood onwards is generally sufficient to maintain belief, because religion mainly concerns itself with mental obedience (to God, to a dogma, to a set of principles). In the case of politics, physical violence is necessary, since politics concerns itself with physical issues (the actions and movements of people), although it is also concerned with mental obedience (propaganda, marketing, rhetoric, marginalizing unacceptable positions). All of these of course go back to the power wielded by authority and the hierarchies within which it exists.
If atheism and Anarchism are both mainly concerned with authority, then atheism must be a subset of Anarchism, but not vice-versa. Atheism deals specifically with the two forms of authority relevant to religion, while Anarchism deals with all forms of authority.