Along with the increase of evidence being generated for the purposes of informing policy, there has been a dramatic rise in the number of studies relying on social media as a platform for collecting data on a variety of topics in the social sciences. One such area is global health, which is an area of policy heavily affected by the blurring of national borders and rapid globalisation. Due to this, more studies involve diluted samples of participants around the world who have access to social media sites where surveys are promoted. To test what implications this may have on using social media to elicit data intended to inform global health policy, we trialled such a study using a variety of sites to generate survey data. The topic was on decision-making regarding medical travel, given that patients are increasingly crossing borders in order to receive care. The pilot involved a large amount of data from over 800 participants in 40 countries. However, while the sum result of all findings were useful on a high level, the nuances in working with this data is of huge relevance to policymakers' approach to utilising such evidence as even a sophisticated, highly adapted approach to modelling raised concerns about appropriate use of the data. Examples of this were the significantly uneven distribution of representation of nationalities that was not simply resolved using propensity weights, resulting in the minimising of highly relevant factors of minority groups and choices - that is, small but very important factors and outcomes may be missed due to large majority factors. This was particularly the case for medical choices between age groups, for individuals from regions highly affected by conflict, and also by under-represented countries in the data whose patterns did not fit those of highly represented countries, which also presented a proxy for national economic and health service standards.