G. Open-ending v. close-ended questions

i. Open-ended responses can be helpful in the preliminary stages of a research project

ii. It can be time consuming to covert open-ended responses into useable data

a. The development of a coding system is necessary

b. High inter-reliability is needed, which means different raters using the same coding system agree on how they classify or otherwise score participants responses

iii. Although close-ended questions can make analysis much easier, participants might not fee their answer is among the choices available.

a. You could use “other” or “don’t know” as options, but these are not a perfect solution either since analyzing data will be more difficult

H. Order of the questions

i. Group questions and questionnaires bases on topic

ii. Place open0ended questions before close ended questions

iii. Place general items before more specific items

iv. Place sensitive questions after those that are less sensitive

I. Pilot Testing

i. Individuals from the population of interest complete the measure

ii. Determine confusing or unclear questions

a. Sometimes interviews are needed to improve the questionnaire

iii. Provide initial psychometric data

a. Focus on reliability 1st

1. Always internal consistency and sometimes test-retest

b. Then on validity

1. Construct validity is very important

2. Remember, this type of validity is about whether or not the questionnaire measures the construct you intended to measure

iv. Factor analysis

a. This statistical approach will allow you to determine whether you are measuring one more factors

b. If you have multiple factors then the questionnaire will have subscales

v. Psychometric properties

a. Reliability

1. the extent to which a measure is consistent and free from random error

b. Validity

1. The extent to which a measure is an accurate reflection of the construct

vi. Survey Administration Options

a. Face-to-face (also known as in-person)

1. Questions can be asked via interviews or respondents can complete them on computer/paper

2.Pros

-This type usually achieves a higher response rate than other methods

-Opportunity to establish rapport, which will encourage individuals to be honest, particularly when it comes to sensitive questions

-rapport also increases the chances a participant will complete a long survey

-The questionnaire can be checked over for completeness and you can answer any questions they have

-Interviewers have access to participants’ verbal and facial cues which may indicate uncertainty about items, inattention or fatigue

3. Cons

-Travel time/cost for either the interviewer or participant

-If a participant is being interviewed, questions with complicated items or containing many response options might tax the participants’ memory capabilities

-Interviewer effects:

-Aspects of the interviewer’s presence can distort participant’s true response

-One example is interviewer bias

-Another example is interviewer characteristics

b. Telephone

1.Pros

-There is an opportunity to establish some rapport

-Confusing items can be explained

-Random digit dialing can be used

-This means computers randomly determine phone numbers that will be dialed

-allows the opportunity to reach people with listed or unlisted phone numbers

-A computer-assisted telephone interview system can be used

-This will enhance the speed of gathering and storing the information

2. Cons

-Establishing rapport over the phone is more difficult than in person

-Fatigue or boredom are more likely to occur sooner than in person

-A phone interview should be no more than 30 min long

-Interviewer effects can still be a problem

-Verbal cues

-Assumptions about the interviewer’s gender, ethnicity, age

-Cell-phone-only users complicate data collection

c. Mail

1.Pros

-Less costly (time as well) than face-to-face and telephone interviews

-Participants have a long time to complete the questionnaire compared to the interview options

-Some individuals might feel more comfortable answering sensitive questions compared to the interview options

-Issues of anonymity

2. Cons

-Lower response rate compared to face-to-face and telephone interviews

-No opportunity to have questions the participants has answered, clear up confusion, make sure all the questions have been answered and/or ensure the questionnaires have been answered in the correct order

d. Electronic (websites, email)

1. Pros

-Costs are the lowest with the method

-Answers do not need to be entered into a database for analysis

-Online social networks allow researchers the chance to access special samples of people

-Customization- It is very easy fir participants to go to relevant questionnaires and different sets of questions can be presented based on initial responses

-Less participant burden

2. Cons

-Convenience sampling is typically used which means the data may or may not be representative of all web users within that population

-People have the opportunity to click through without answering the questions causing inaccuracy

e. Mixed mode of any of the above is also an option

vii. Thinking critically about survey results

a. who conducted the survey , and for what purpose

b. How was the sample obtained

c. What method was used to administer the survey

d. Are the survey questions appropriate

e. Are the results interpreted properly and are the conclusions reasonable

	Created by Shelby Klein over 6 years ago

Survey and Interview Approaches