Looking back over this, I see that I posted the wrong link (though the one I posted is worth reading). The Walter Block article I meant to post is here:
Dr. Woods, that article by Dr. Block appears to contradict what you recall him saying in that debate. Block states that the North invading the South to rid it of slavery would violate the non-invasion (except in self-defense) provision of limited government libertarianism:
“If secession is always and everywhere justified, what, then, is the proper libertarian response to the existence of suttee, slavery, clitorectomy, etc., in other countries (e.g., in seceding territories)?
Under limited government libertarianism, the government of the north would take no steps to rid the sovereign Confederacy of its slavery (or India of its suttee). The purpose of the state in this philosophy is to protect its own citizens. Period. And, on the (historically accurate) assumption that the Confederacy showed no indication of invading the north, but merely wanted to be left alone to its own devices, that would be the end of the matter as far as the northern government was concerned.
However, even under these assumptions individual abolitionists would be perfectly free, and, indeed, justified, in going in to the Confederacy, guns in hand, with the intention of ridding the south of this evil institution of slavery. But if things went poorly for them, they could not then scurry back to the north, tails between their legs, hiding behind their mama’s skirts, because that would necessarily bring in the northern government into the fray. It would violate the non-invasion (except in self-defense) provision of limited government libertarianism, or minarchism.”