No one can understanding the meaning of human action by external observations. Empirical evidence is about activity, not meaning. For example, while I’m in Wal-Mart I see someone pick up a tube of toothpaste, put it in a cart, hand it to a clerk, who puts it in a bag, and so on. I interpret this activity as human action because I presume the someone I observe is a human person like me. So, I infer that this person has an end he is trying to achieve with this activity and that he is employing means to do so on the basis of assessments he has made in his mind about the value he perceives in different courses of action.
We don’t observe human action in others, we infer that the activity we observe others engaged in is human action and we interpret this activity according to the meaning of human action which we acquire, not by observing others, but by reflecting on our own human actions.