Well we do know a decent amount. Like you said, we have the debates, we have the Federalist Papers written by the people involved in writing the Constitution to explain what they had written, we have correspondence between the writers of the Constitution and between the writers and concerned citizens and on it goes.
There are multiple problems with using precedence to determine meaning. Every precedence will be stretched in order to allow for something new. Every time precedence is cited it will divert farther and farther from original intent.
Furthermore, if we look at “societal consensus” to define the Constitution, then what exactly is the point of having a constitution? How do we determine what consensus is? After all I have never heard of a vote on the meaning of the constitution. It seems that the people that are always calling for something like “societal consensus” or that the constitution is “living” are also the ones who want to speak for the rest of society as to what the new meaning should be.
The constitution was designed with the ability to change through the amendment process. If the public believes that government should have a new power, then the public can grant that new power through an amendment.
But besides all that, the constitution was meant to restrict government and to grant certain specific powers. Thus, when in doubt whether the government has power _______ the answer is more than likely that it does not.