Recently, the creator of Dilbert, Scott Adams, became the subject of a furore where he encouraged people to not associate with black people on the basis of a poll run by Rasmussen Reports.
Importantly, Mr. Adams said that this was the “first” poll that had caused him to change his behaviour.
The irony is, what was reported about the poll was entirely different to the responses given in it. The poll surveyed a thousand people across the United States of America. The pop-up in Mr. Adam’s podcast said “Rasmussen Reports BLACK AMERICANS ONLY: “It’s okay to be white” 53% agree, 26% disagree, 21% not sure.”
The pop-up was materially misleading. Rasmussen’s own data is below:
From this data you can see that, for a start, Black Americans were not the only people surveyed. In fact, Black Americans only made up 13% of the thousand people surveyed, proportionate to the American population overall. Further, the actual question that was asked was not whether it was “OK to be white”. The actual question asked was:
Do you agree or disagree with this statement: “It’s OK to be white”.
Now as someone trained in statistics, I can tell you that this question is terribly phrased because it gives rise to ambiguity. The ambiguity is that there is a considerable difference between being asked whether it is “OK” to go around saying “It’s OK to be white” versus whether people are whatever “OK” means if they have white skin.
Where does the phrase “It’s OK to be white” come from?
Like so many divisive things, it originated from a 4Chan troll and was designed to generate dissent and anxiety. Later, it was adopted by neo-Nazi groups as a response to “Black Lives Matter”.
So if I were asked that question, I’d answer ‘No’ because I would not be happy to be associated with neo-Nazis. I will not say “It’s ok to be white”, not because I don’t like white people or are prejudiced against people due to their ancestry or the level of melanin in their skin, but because I am conscious of the trolling origin of the phrase.
When statisticians craft surveys, it is done for one of two reasons:
- either we are trying to acquire data to understand the world around us; or
- we are trying to use the survey to “manufacture consent” as Noam Chomsky would say, that is, using the questions in the surveys to attempt to elicit a response to drive a narrative.
This particular Rasmussen Survey was poorly designed and because it bothered to ask the question in such ambiguous terms, it appears to have been intended to manufacture a narrative to the effect that Black Americans do not like White Americans. Ironically, it failed obtain the data to support that narrative.
How to interpret the Rasmussen statistics
You can see from the pop-up in Mr. Adam’s podcast combined with the lazy reporting of the survey that people were of the view that a thousand Black Americans were asked whether “it’s OK to be white”, but this is not the situation.
Looking at Table 1 (above), it is evident that only 13% were black of the thousand people surveyed.
Moreover, of those 13%, or 130 people, only 26% of them were of the view that it was NOT OK to say “it’s OK to be white” because 18% on the 5-point Likert scale opted for “strongly disagree” and 8% opted for “somewhat disagree” (totalling 26%).
So how many people does that amount to?
It turns-out around 34 people out of those 130 felt that the phrase was unacceptable.
It is statistically invalid to extrapolate from a mere 34 people to suggest that, across America, 26% of all black people think it is “not OK to be white” because even the sample size of 130 people is too low for the size of the population to draw that qualitative conclusion.
Another problem with the survey was that many people reading it incorrectly conflated the “not sure” response with suggesting that it was “not OK to be white” to reach “we’re not sure that it is OK to be white”.
In statistics, a “not sure” or “don’t know” response suggests that the respondent:
- doesn’t have an opinion on the matter either way;
- doesn’t understand the question; or
- finds the question ambiguous.
There’s no reason to conflate a “not sure” with the negative any more than there is to conflate it with the positive. The respondents answering “not sure” can’t be conflated with other groups. Interestingly, the “not sure” answer tracks well across all races, so it might suggest that the ambiguity of the question made them unsure about answering that question positively or negatively.
Survey Biases
So why would I not consider the survey result to suggest that black people think it is not OK to be white? Rasmussen did not say how it conducted the survey, whether by phone or online, so I will cover both options.
I have a problem with online and phone surveys generally because of how they get people to answer the questions. Firstly, in order to get someone to answer political surveys such as this, you need people to decide that they want to respond.
If the survey is online, then you are getting people who feel strongly enough about the survey to go and log in and participate.
If they are cold-called by telephone, then you need them to answer the phone and then listen to the spiel, hand over their demographics and then decide to answer the survey.
If they are from a set of people who have pre-agreed to answer surveys, then it is usually because they have been promised some form of benefit in response for answering, such as to go into a draw for a cruise. We don’t know what kind of survey it was because Rasmussen has not revealed that information to the public.
In statistics, we have a concept called “response bias”. The only people who tend to respond to surveys have strong feelings about the questions being asked one way or another.
If it were people cold-calling in a telephone survey, we aren’t told how many people refused to participate.
If it were a “log in here and tell us what you think” survey, then there’s always a question as to who is likely to respond and whether they are “genuine” respondents.
If it were a pre-agreed pool of people who were responding, then this might explain the relatively large number of respondents who answered “not sure”.
Genuine responses
How do readers know that the people who are responding that they are black (or white, or other) are from the background that they say they are? We don’t have access to the demographic questions or the means by which Rasmussen says that they were aware of the racial identity of the respondents.
Clearly a number of other questions were asked to ascertain demographics, but how, in fact, does Rasmussen know that these demographics are accurate?
Again, we have no data on that.
When newspapers do polls online, a real problem is that occasionally someone will decide that they wish to “stack” the survey and will write a script and respond multiple times. We don’t have any data as to how Rasmussen tries to deal with such issues for online surveys.
One other interesting factor is that we don’t know the educational attainment of the people responding. Generally, we test for educational level because we’d expect that there could be differences of attitude depending on the level of education that a respondent has attained.
It’s an area of the survey demographics that hasn’t been asked but which we would usually test to see whether it was an important variable because the received wisdom is that the higher the level of educational attainment, the lower the level of prejudice.
Whether this is true in any given set of people varies but, because it has been a variable in other cases, it is usually included as a variable to see whether there’s a “cause” other than blind prejudice.
So should the survey have persuaded any person that black people are of the opinion that it is not ok to be white?
If you are of the view that the opinion of 34 people across all of the US can represent the 47 million black people across the US, then you might be persuaded. Whereas 130 out of the thousand people represents the 13% of the US population that is black, it is a very small, self-selected population from which to draw ANY valid conclusion.
This is why, if a student had presented this survey to me as an assignment,
I would have given it a fail grade.
As an attempt to set white people against black people by “rage farming”, a new “word” apparently added to the dictionary in the USA, I fear that it has worked all too well.
One of the best ways to make a set of people hate each other is to say “those people over there are out to hurt you” or “those people disapprove of you or your lifestyle”.
Despite the fact that over 74% weren’t happy to disagree with the statement “it’s ok to be white”, the theme of the news reports arising out of it was that black people don’t think it’s OK to be white.
You should not trust news reports and podcasts that blindly accept statistics.
Sometimes, the purpose of those statistics is to manipulate you. This is why you should check the statistics behind any survey that has “inflammatory” conclusions or questions.
