Foreigners can either hold the dollars overseas or spend them back in the U.S. If spent in the U.S., foreigners can either buy goods and services or make financial investments. As you say, the balance of payments accounts record the consequences of theses actions and are designed to always balance.
Of all the dollar currency in circulation, approximately two-thirds is held by foreigners overseas. They do this for buying goods which are bought and sold in dollars, as you suggest, and as a safe-haven asset.
http://www.federalreserve.gov/pubs/ifdp/2012/1058/ifdp1058.pdf
A given money, like the dollar, moves from one place to another based on differences in its market value, i.e., its purchasing power. If the purchasing power of the dollar were lower in New York City than in Denver, people would shift their demands to Denver until no difference remained. If the purchasing power of the dollar were lower in New York City than in Paris, people would shift their demands to Paris. To do so, they would sell dollars to buy Euros pushing the exchange rate of the dollar down until its purchasing power was the same in both places. The balance of payments accounts simply record the results of people’s actions which are determined by their preferences.
.
