Obligatory societal generalizations
(Yes, there really is a train of thought connecting this post with the preceding one. 10 points if you can find it.)
I've always been of two minds about the "I'm not a math person" type of person. I always feel that, at least in today's society, those words give you a free pass for ignorance in areas where you really should have at least minimum proficience (that is to say, I don't really look kindly on "I'm not a math person" when tip-calculation time comes around, though with everyone's profusion of electronic gizmos the average person probably can calculate the tip on 5 different machines without straining a neuron, but I digress). And I do feel that math is specially singled out for this kind of treatment -- if I were to say, "Oh, I'm not a history person" when someone mentioned the Gettysburg Address (to pick a random example), I expect that that would draw at least a quizzical look, while "I'm not a math person" is pretty much accepted without comment, if not outright agreement and sympathy.
On the other hand, I recognize that there really are people who aren't "math people". To pick a not-at-all random example, I've tried to teach various mathematical concepts to my mother over the years, with not at all that much success. Things that I find simple, she just doesn't "get" on the same level, even after putting quite a bit of effort into it (and, to be honest, much more effort than I had to put in to understand the same concepts). It's not like she's not an intelligent woman -- she is, but just not when it comes to things mathematical.
I suppose the dividing line comes from effort. I'll bet a significant fraction of the people who claim that "they just don't get math" could understand enough to get by if they were willing to put some work into it, and, conversely, I'm willing to excuse those who have tried and still don't get it. But, of course, with the need for mathematical skills arguably at an all-time low, except in the specialized fields that depend on it, people don't see the benefit of learning it.