His main point was that credit expansion would finance industrialization. Yeoman farmers didn’t need cheap credit. Empirically, it seems he was correct. The credit expansion in the north went to infrastructure and manufacturing. In the south it financed capital projects on large plantations. Jefferson was more troubled by the former since it fostered urbanization, which he thought was a life of dependence, at the expense of rural life, which he thought was a life of independence.
In reporting to President Washington, Jefferson made constitutional objections to the Bank of the United States, not economic objections:
Here’s a short piece on Jefferson’s views more broadly: