In the Report of 1800 when talking about how the ultimate arbiter of the Constitution is the people themselves, Madison says that,
“However true therefore it may be that the Judicial Department, is, in all questions submitted to it by the forms of the constitution, to decide in the last resort, this resort must necessarily be deemed the last in relation to the authorities of the other departments of the government; not in relation to the rights of the parties to the constitutional compact, from which the judicial as well as the other departments hold their delegated trusts.”
This seems in direct contradiction to what he had written earlier in Federalist 49.
“As the people are the only legitimate fountain of power, and it is from them that the constitutional charter, under which the several branches of government hold their power, is derived, it seems strictly consonant to the republican theory, to recur to the same original authority, not only whenever it may be necessary to enlarge, diminish, or new-model the powers of the government, but also whenever any one of the departments may commit encroachments on the chartered authorities of the others. The several departments being perfectly co-ordinate by the terms of their common commission, none of them, it is evident, can pretend to an exclusive or superior right of settling the boundaries between their respective powers;”
Did he relinquish departmentalism in favor of judicial supremacy, at least at the federal level?
I understand him to have meant that the courts would have the final say in a particular legal dispute, but not when it came to, say, the text of the Constitution–which the Congress and states could alter.