Oh, where am I going with THIS? I haven't been from "around here" in a little over 12 years. The funny thing about living in so many different states is that I have only rarely been asked that question.
Normally, the question is worded this way: "Where are you from originally?" That means something different. The asker is interested in the subjects roots. Its a question directed at a person with the point of finding out about them.
But when someone states "You're not from around here, are you?" They aren't interested in the least about your roots. They are stating that your roots are different from theirs. Not just your roots, but your values are based on different experiences than those "from around here".
Someone I work with (with whom I haven't always had the smoothest relationship) asked me this question once. We were in the midst of a philosophical discussion about maintenance. When he asked this question, his point was that I wouldn't and couldn't understand because I do not possess the shared experiences of those who are from here. That's too harsh. He wasn't saying that I couldn't understand. He was saying that I don't and it took some explaining and a historical discussion of why certain sawmills failed ("I thought EVERYONE knew that it closed in 1983 because of ...") and closed for me to understand his point.
What's the point? A lot goes into good communication. Besides words, voice inflection, and body language, you need to consider assumptions based on past events. Misunderstanding happen that easily.