Chapter 7

Chapter 7: Survey and Interview Approaches A. Why conduct survey research?               i. In academia a. Basic researchers use surveys to advance scientific knowledge about human behavior b. Applied researchers use surveys to address practical issues               ii. May or may not be in academia                            a. Surveys, such as option pools based on representative samples, also give people a voice               iii. Survey research allows investigators to:                            a. Describe the characteristics of a population                            b. Compare the characteristics of different populations or different demographic groups within a population                                          1. “What % of college students drink between men and women…ethnic groups?”                            c. Describe population time trends                                          1. “Drinking in college students rends in the last 10 years”                            d. Describe relations among psychological variables                                          1. “Peoples personal believes either rooted in faith or luck”                            e. Test, hypothesis, theories, models B. Strengths and Limitations of Survey Research               i. Strengths:                            a. Ability to collect data from a large group of people very quickly                            b. Ability to collect info about self-reported demographics, preferences, attitude, lifestyles, habits etc.                            c. Issues of anonymity               i. Limitations:                            a. Some sampling models (see chp. 6 slides for sampling options) can generate misleading results                            b. Social desirability bias, inaccurate recall, misunderstanding questions (see chp. 7 for long list) C. Things to avoid when conducting survey research               i. Asking questions for the sake of curiosity                            a. Problem: Ethical implications and possible difficulty achieving IRB approval                            b. What are the possible sensitive topics?               ii. Administering questionnaires in the same order to every participant                            a. Problem: One questionnaire might impact how a participant answers questions on another questionnaire                            b. Solution: Counterbalance order of the questionnaire. D. Strengths of using an existing questionnaire               i. A lot of time will be saves               ii. Some/all psychometric properties have been established, so you will have an opportunity to uses a reliable and valid measure of the construct of interest               iii. You will be able to compare your findings to the work of others who used the same measure E. Why would you need to create a questionnaire               i.  Your construct of interest has been studied rarely or never               ii. There are currently no reliable or valid measures available               iii. The reliability or validity of existing measures are questionable

F. Things to do/avoid when creating questions               i. Create simple questions               ii. Make sure reading level is appropriate               iii. Write questions in a positive or neutral manner rather than negative               iv. Things to avoid:                            a. Leading questions                                          1. Problem: These are imbalanced and could suggest that one response is preferred over another                                          2. Example: “Do you support the universities’ unfair proposal to raise tuition by $500/year in order to modernize campus computer facilities?”                                          3. Balanced question: “Do you support or oppose the universities proposal to raise tuition by $500/year in order to modernize the campus computer facilities?”                            b. Loaded questions                                          1. Problem: Emotionally charged words are used and could suggest that one response is preferred over another                                          2. Example: “Would you favor or oppose a law banning restaurants form serving food that contains dangerous, heart damaging trans-fats”                            c. Double-barreled questions                                          1.Problem: Two issues are asked about within one question, which will force individuals to combine potentially different opinions into one judgement                                          2. Example: “Do you agree or disagree that alcohol use and marijuana use are a problem among today’s college students?”                              d. Double negatives                                          1. Problem: These questions cause confusion which increases the possibility of collecting incorrect information                                          2. Example: “Do you agree that once people reach a certain age of adulthood, it is not possible for adults to change undesirable personality traits?” F. Things to do/avoid when creating questions               i. Create simple questions               ii. Make sure reading level is appropriate               iii. Write questions in a positive or neutral manner rather than negative               iv. Things to avoid:                            a. Leading questions                                          1. Problem: These are imbalanced and could suggest that one response is preferred over another                                          2. Example: “Do you support the universities’ unfair proposal to raise tuition by $500/year in order to modernize campus computer facilities?”                                          3. Balanced question: “Do you support or oppose the universities proposal to raise tuition by $500/year in order to modernize the campus computer facilities?”                            b. Loaded questions                                          1. Problem: Emotionally charged words are used and could suggest that one response is preferred over another                                          2. Example: “Would you favor or oppose a law banning restaurants form serving food that contains dangerous, heart damaging trans-fats”                            c. Double-barreled questions                                          1.Problem: Two issues are asked about within one question, which will force individuals to combine potentially different opinions into one judgement                                          2. Example: “Do you agree or disagree that alcohol use and marijuana use are a problem among today’s college students?”                              d. Double negatives                                          1. Problem: These questions cause confusion which increases the possibility of collecting incorrect information                                          2. Example: “Do you agree that once people reach a certain age of adulthood, it is not possible for adults to change undesirable personality traits?”

G. Open-ending v. close-ended questions               i. Open-ended responses can be helpful in the preliminary stages of a research project               ii. It can be time consuming to covert open-ended responses into useable data                            a. The development of a coding system is necessary                            b. High inter-reliability is needed, which means different raters using the same coding system agree on how they classify or otherwise score participants responses               iii. Although close-ended questions can make analysis much easier, participants might not fee their answer is among the choices available.                            a. You could use “other” or “don’t know” as options, but these are not a perfect solution either since analyzing data will be more difficult H. Order of the questions               i. Group questions and questionnaires bases on topic               ii. Place open0ended questions before close ended questions               iii. Place general items before more specific items               iv. Place sensitive questions after those that are less sensitive I. Pilot Testing               i. Individuals from the population of interest complete the measure               ii. Determine confusing or unclear questions                            a. Sometimes interviews are needed to improve the questionnaire               iii. Provide initial psychometric data                            a. Focus on reliability 1st                                          1. Always internal consistency and sometimes test-retest                            b. Then on validity                                          1. Construct validity is very important                                          2. Remember, this type of validity is about whether or not the questionnaire measures the construct you intended to measure               iv. Factor analysis                            a. This statistical approach will allow you to determine whether you are measuring one more factors                            b. If you have multiple factors then the questionnaire will have subscales               v. Psychometric properties                            a. Reliability                                          1. the extent to which a measure is consistent and free from random error                            b. Validity                                          1. The extent to which a measure is an accurate reflection of the construct               vi. Survey Administration Options                            a. Face-to-face (also known as in-person)                                          1. Questions can be asked via interviews or respondents can complete them on computer/paper                                          2.Pros                                                        -This type usually achieves a higher response rate than other methods                                                        -Opportunity to establish rapport, which will encourage individuals to be honest, particularly when it comes to sensitive questions                                                        -rapport also increases the chances a participant will complete a long survey                                                        -The questionnaire can be checked over for completeness and you can answer any questions they have                                                        -Interviewers have access to participants’ verbal and facial cues which may indicate uncertainty about items, inattention or fatigue                                          3. Cons                                                        -Travel time/cost for either the interviewer or participant                                                        -If a participant is being interviewed, questions with complicated items or containing many response options might tax the participants’ memory capabilities                                                        -Interviewer effects:                                                                      -Aspects of the interviewer’s presence can distort participant’s true response                                                                      -One example is interviewer bias                                                                      -Another example is interviewer characteristics                              b. Telephone                                          1.Pros                                                       -There is an opportunity to establish some rapport                                                        -Confusing items can be explained                                                        -Random digit dialing can be used                                                                      -This means computers randomly determine phone numbers that will be dialed                                                                      -allows the opportunity to reach people with listed or unlisted phone numbers                                                        -A computer-assisted telephone interview system can be used                                                                      -This will enhance the speed of gathering and storing the information                                          2. Cons                                                        -Establishing rapport over the phone is more difficult than in person                                                        -Fatigue or boredom are more likely to occur sooner than in person                                                                      -A phone interview should be no more than 30 min long                                                        -Interviewer effects can still be a problem                                                                      -Verbal cues                                                                      -Assumptions about the interviewer’s gender, ethnicity, age                                                        -Cell-phone-only users complicate data collection                            c. Mail                                          1.Pros                                                        -Less costly (time as well) than face-to-face and telephone interviews                                                        -Participants have a long time to complete the questionnaire compared to the interview options                                                        -Some individuals might feel more comfortable answering sensitive questions compared to the interview options                                                                      -Issues of anonymity                                          2. Cons                                                        -Lower response rate compared to face-to-face and telephone interviews                                                        -No opportunity to have questions the participants has answered, clear up confusion, make sure all the questions have been answered and/or ensure the questionnaires have been answered in the correct order                            d. Electronic (websites, email)                                                           1. Pros                                                        -Costs are the lowest with the method                                                        -Answers do not need to be entered into a database for analysis                                                        -Online social networks allow researchers the chance to access special samples of people                                                        -Customization- It is very easy fir participants to go to relevant questionnaires and different sets of questions can be presented based on initial responses                                                                      -Less participant burden                                            2. Cons                                                        -Convenience sampling is typically used which means the data may or may not be representative of all web users within that population                                                        -People have the opportunity to click through without answering the questions causing inaccuracy                              e. Mixed mode of any of the above is also an option               vii. Thinking critically about survey results                            a. who conducted the survey , and for what purpose                            b. How was the sample obtained                            c. What method was used to administer the survey                            d. Are the survey questions appropriate                            e. Are the results interpreted properly and are the conclusions reasonable