Both in the Virginia and in the New York Ratification Convention, Federalists and Antifederalists alike thought that the federal courts would exercise the power of judicial review. The only alternative would be for them to enforce laws they considered unconstitutional, which would seem to make it nonsensical for them to swear to uphold the Constitution.
Napolitano, a friend of Scalia, is echoing Scalia’s ridiculous “textualist” argument that the judges “made … up” the power of judicial review. Being a “textualist” means ignoring the ratification conventions, which is ridiculous for a self-styled “originalist” to do.
Judicial review doesn’t necessarily make the judges “ultimate arbiters” of anything. The other branches, the voters, et al., also have to decide what the Constitution means when they exercise their various powers. Jefferson’s point was that judges only decide the cases before them, which is the alternative to judicial supremacy.