You are correct. The Southern States seceded through popularly elected conventions which voted in greater majorities than those which ratified the Constitution. And because sovereignty cannot be divided, the States, or the people thereof, have the ability to leave a compact.
