Bad Survey Questions

EXAMPLES OF BAD QUESTIONS & SUGGESTIONS OF HOW TO FIX THEM! Double Barreled question: Please indicate how much you agree or disagree with each of the following statements about the childcare program. Incorrect: I feel welcomed by staff and other youth at the center Correction: I feel welcomed by staff at the center I feel welcomed by other youth at the center Biased/leading question Incorrect: Community organizing is hard. Do leadership trainings help you feel prepared for community organizing? Much more prepared Slightly more prepared Somewhat more prepared Not more preparedCorrect: The leadership trainings prepare me for community organizing. Strongly agree Double Negative Incorrect Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened? (22%) Very possible Impossible Correct Do you doubt that the Holocaust actually happened or not? (9%) Very possible Impossible Possible Very impossible Possible Very impossible Agree Disagree Strongly disagree EVALUATING SURVEYS The following are additional examples of bad survey questions. As a group, apply our class discussion, evaluating each question and its response choices to identify one error in each question (ie. ouble barreled). Explain the problem that your group identifies in each question and propose an alternative that solves the problem. 1. More people have attended the movie, Gone with the Wind, than any other motion picture produced this century. Have you seen this movie? Yes No 2. In your opinion, how would you rate the speed and accuracy of your work? Excellent Good Fair Poor 3. Did you first hear about the bombing: ______ from a friend or relative ______ from your spouse ______ from a newspaper ______ at work ______ from the television or radio or other electronic media 4.How do you feel about the following statement? We should not reduce military spending. Strongly agree Agree Disagree Strongly disagree 5. People grow up in all different types of fam ilies. What type of family did you grow up in? ______Mom as single parent ______Dad as a single parent ______Both Mom and Dad 6. Where do you get most or all of your information about current events in the nation and the world? __radio __Newspapers __Magazines __internet 7. Higher than single-family density is acceptable in order to make housing affordable. Strongly Undecided Disagree Strongly Disagree Agree Agree 1 2 3 4 5CORRECTING SURVEYS The following are some suggestions of how to fix the problems of the survey questions found on the prior page. 1. Problem: Biased/Leading Solution: Have you seen the movie Gone with the Wind? Yes No 2. Problem: Double Barreled Solution: In your opinion, how would you rate the speed of your work? Excellent Good Fair Poor In your opinion, how would you rate the accuracy of your work? Excellent Good Fair Poor 3. Problem: Categories are not Mutually exclusive Solution: Did you first hear about the bombing: ____ from a friend ____ from a relative ___ _ from a media source 4.Problem: Double Negative Solution: How do you feel about the following statement? We should reduce military spending. Strongly agree Agree Disagree Strongly Disagree 5. Problem: Categories are not Exhaustive Solution: People grow up in all types of families. What type of family did you grow up in? ____two biological parents ____one biological parent & a step parent ____ mom as a single parent ____ dad as a single parent ____ with a relative other than my biological parents ____ in foster care ____ with adoptive parent(s) 6.Problem: Double Barreled Solution: Which media source do you rely on most to get information about current events involving the nation? ___radio ___ Newspapers __Magazines ___ Internet Which media source do you rely on most to get information about current events involving the world? ___radio ___ Newspapers __Magazines ___ Internet 7. Problem: Ambiguous Solution: In order to make housing affordable, is it acceptable to build housing whose b uildings hold more than one family? Strongly agree Agree Undecided Disagree Strongly Disagree