It’s more than just the rejection of mathematical formalism, but that’s part of what distinguishes Austrian economics from neoclassical. As Mises pointed out, mathematical functions cannot be used in economics theorizing because there are no constants in quantitative relationships generated by human action.
Take a look at Human Action, pp. 347-354; 706-711.
More generally, Austrians accept human beings as they are without the reductionist assumptions typical of neoclassical economics. For example, Austrians accept genuine uncertainty as an integral part of economic theory.
While Austrians reject mathematical formalism (and hence, optimization), they don’t reject the logic of economizing. Because of this, Austrians and neoclassicals can produce similar conclusions. For example, both would argue that production in the market economy is regulated by profit.
Thomas Taylor’s monograph provides some highlights of the distinctiveness of Austrian economics: