This situation is akin to the welfare of today. Should one individual be able to steal if they are worse off than another individual? The answer is clearly no, otherwise property rights no longer exist.
What if the island only had enough bananas to sustain one person? Would the person arguing that the theft is OK still find it to be acceptable?
Assuming there is enough food to feed both men what recourse then would Man A have against the encroachment of Man B? Man A can not force Man B to labor for the same reason Man B can not force Man A into giving him a banana against Man A’s will. The only legitimate course is mutually agreed trade.
Now the question of can “Man A force Man B off the island?” comes up. Well, can you force someone out of your car when you are driving at highway speeds?
The original question and solution is rather simple once you set up some basic definitions of property rights.
Just having another person come into existence does not split the available property amongst all known individuals. Homesteading requires the combining of nature and labor. With the available technology today, one person could not have the capital to homestead an entire planet. Although it is an interesting thought that we may one day be able to homestead an entire planet for ourselves.