Madison said in the House that he could not point to any provision of the Constitution that empowered Congress to give people charity. I’m not aware that Hamilton ever in any way disagreed with this.