In JAMES MADISON AND THE MAKING OF AMERICA, I give considerable space to Madison’s campaign against Patrick Henry’s 1785 proposal for a general assessment. Among other things, Madison wrote “Memorial and Remonstrance: Against Religious Assessments,” in which he purported to show that close ties between government and religion have always harmed both government and religion. If by “to exclude religion” you mean “to prevent government from being religious,” I’m afraid you’re mistaken.
The paperback version of the book has a new introductory essay, “James Madison and the American Ideal of Religious Liberty,” taking the story up to the end of Madison’s life. He became more libertarian in this area all the way to the end.