"Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances."
Implicit in these freedoms is freedom of thought and belief, in that free exercise of speech or religion requires freedom of thought and belief. The law has long recognized that freedom of speech has limits (yelling fire in a crowded theater when there isn't a fire; fighting words; obscenity; defamation -- for which truth of statement is a complete defense; sedition; etc.). The law has also recognized that freedom of religion has limits (polygamy being an early test case, and it's worth noting these were all adults involved in the test case, peyote being a more recent example, in both cases, religions which used/required these activities were unable to overcome a neutral law prohibiting these activities -- this is Lockean).
The freedom to assemble in the first amendment has been reinterpreted a bunch, being at times held subordinate to the right to petition, but in general is not now. But freedom to assemble can be limited by public safety concerns.
So we have quite strong protection of our right to think what we want, slightly less strong protection of our right to say what we want, slightly less strong protection of our right to worship as we want, and considerably less strong protection of our right to join together for any of the above.
Sam Harris et al have every right to say outlandish things (as people with the freedom to do so have done for thousands of years, at least going back to Socrates and Plato and probably much earlier) about how we should limit the freedom of others to think, believe and speak. We'd damn well better have the sense to recognize that whether it's the Governing Body of Jehovah's Witnesses kicking out a member for what they think (even when they don't talk about it to anyone else), or a New Atheist proposing we go after people for holding beliefs that might potentially lead to Bad Actions, these people are proposing that we throw away rights we've fought wars and been activists to defend. Baby. Bathwater. Really fucking stupid.
And totally unnecessary. There are plenty of laws to go after Bad Actors -- for their Bad Actions, not their foolish and dangerous beliefs. And we've got all the freedom we need to speak truth to their foolish and dangerous beliefs. We create a structure in which we start saying you-can't-think-or-say-that-because-of-w