Chapter 4: Sampling

Note: THESE ARE FROM THE BOOK  Advertising Research Theory and Practice Second Edition Joel J. Davis School of Journalism & Media Studies, San Diego State University Prentice Hall Boston Columbus Indianapolis New York San Francisco Upper Saddle River Amsterdam Cape Town Dubai London Madrid Milan Munich Paris Montreal Toronto Delhi Mexico City Sao Paulo Sydney Hong Kong Seoul Singapore Taipei Tokyo *I take no claim in founding the information on these slides. I just took the most relevant information and posted them on a quick and easy to read slide set.

Primary research collects original, typically proprietary information to meet an advertiser’s or marketer’s informational needs. All primary research entails some form of sampling where a researcher selects people or objects from a population of interest for further study. Good research requires good sampling. When you sample, in these cases and in advertising research, you select and examine members of a larger population in order to learn something new and, in most cases, to draw conclusions about the larger population of which the sampled items are members.    

convenience sampling - Convenience sampling is just what the name implies: study participants are selected because they are convenient and accessible. Interviewing friends, associates, or individuals walking down the street or through the mall are forms of this type of sampling. Convenience sampling, as might be expected, is uncomplicated, quick, and low cost. random sampling - Random sampling is most associated with quantitative research. A random sample is one where the researcher ensures that each member of the population of interest has an equal probability of being selected.  

Sample or Census The first step in the random sampling process determines whether to use a sample or a census.  A sample, rather than a census, is used in the vast majority of research situations. A well-selected sample can provide information comparable to that of a full census.  

There are some situations, however, when a census is preferable. A census is preferable to a sample when the population of interest is small and identifiable, or sampling might eliminate important cases from the study, or credibility requires the consideration of all members of the target population.

Define Target Population The next step in the sampling process requires that you define the target population by explicitly specifying the characteristics of the group of individuals or things in which you are interested. This is a criticals step and therefore required for all types of samples.  

Consider each of the key components of the proposed target definition: Retail price.  Leading brands. Stores. Pain relievers. Across the United States. These are too ambiguous. We need to explcitly define our target

Target Definition and a Human Population. Populations of individuals are typically defined in some combination of demographic, geographic, and behavioral criteria. The demographic component of the target population definition specifies relevant age, gender, income, or other related characteristics of the population of interest. The geographic component specifies the geographic area(s) in which the target population resides.  The behavioral component specifies relevant category- or product-related behaviors. 

Population Target Definitions and Research Findings. The quality and validity of generalizations drawn from a research study are greatly influenced by its target definition. After all, because different target definitions exclude and include different individuals, the data collected from different groups of individuals is also likely to vary. Two studies designed to study the same thing, but with different target definitions, are likely to have quite different results.

Estimate Number of Contacts. Sample size requirements identified in the prior step reflect the number of completed interviews required for a desired confidence interval and confidence level. A research fact of life, however, is that not all individuals contacted will agree to participate in the research and not all who agree will actually complete the survey or other data gathering instrument. As a result, the number of people contacted is always greater than the final desired sample size. The number of required contacts is determined by the following formula: where DSS is the desired final sample size, ATP represents an estimate of the percentage of the target population who will agree to participate, and CS represents the percentage of those agreeing who will provide complete survey responses: DSS - desired sample size ATP - agreement to participate CS - Completed Surveys Sample size is calculated by the formula: Required Contacts = DSS divided by (ATP * CS)

Select Sampling Method Once the target population is defined, the next step determines which of two types of sampling methods will be used to identify items or individuals for study inclusion.  The choice of a sampling method is influenced by several factors: the type of generalization required, the researcher’s need to minimize sampling error, study timing, and cost. The relative advantages and disadvantages of probability and nonprobability samples mirror each other in these areas. Probability samples let a researcher estimate sampling error, calculate reliability, statistically determine the sample size required for a specified degree of confidence, and most important, confidently generalize the findings to the sample universe. In a probability sample, each individual in the target population has an equal chance of participating in the research. Nonprobability samples are quick and inexpensive to obtain. Research conducted among nonprobability samples is easy to design and carry out. However, a researcher using a nonprobabilty sample cannot calculate sampling error or reliability and has very limited confidence in generalizing the findings the sample universe.

Sample Frame The sample frame provides the detail on where members of the target population will come from by specifying the method used to identify the households, individuals, or other elements specified in the target population definition.  read page 87 to define the following: perfect registration over-registration under-registration Over- and under-registration, if left unaccounted for, can cause significant bias.

Types of Probability Sampling Once you know the characteristics of the target population and how the population will be identified, you next need to determine the specific probability sampling procedure by which individuals are selected for study inclusion. The three most common forms of probability sampling used in advertising research are simple random, systematic random, and stratified random samples. Simple Random Samples. Simple random samples are frequently used in advertising research. Here, each member of the population has an equal chance of being selected for inclusion in the research. By randomly selecting individuals from the sample universe, we can accurately estimate the behaviors of the entire target population. Systematic Random Samples. A variation of a simple random sample is a systematic random sample. Systematic random samples typically provide data identical to simple random samples with the added advantage of simplicity—no table of random numbers or coin toss is needed, and sample size can be firmly specified. Count the number of elements on the list. Determine the desired sample size. Compute a skip interval. Select a random place on the list to start. Select each element at the appropriate skip interval. Simple and Systematic Samples: Online Selection. The prior discussion of simple and systematic samples assumed that a sample frame was available and that random sampling could be used to select individuals from the identified source, either through explicit selection or through techniques such as random digit dialing. There are times, however, that the sample frame reflects a set of behaviors and, as a result, not all individuals in the sample frame are known. Stratified Random Samples In the prior examples, simple and systematic random sampling techniques worked well. They were efficient and provided reliable generalizations about the total population. However, these forms of sampling worked well only because the universe was relatively homogeneous with respect to what was being measured. Simple and systematic random samples provide fewer reliable generalizations about the total population when significant differences among population subgroups are suspected.  Stratified random sampling is a better choice versus simple and systematic random sampling whenever you have a situation where you think there is a large variation in what you are studying due to specific respondent characteristics. Stratified sampling is accomplished a four-step process and described below for the social media usage universe: First, one or more classification criteria that define the strata are identified. These classification criteria should define independent strata that do not overlap. Second, each element in the sample frame is assigned to one and only one stratum. Third, the total sample size is determined. Fourth, independent random samples (using either simple or systematic sampling methods) are selected from each stratum in a way that results in the total sample size being achieved. The fourth step, sampling from each stratum in order to achieve the desired final sample size, presents two options regarding the number of elements selected from each stratum. Either proportionate or disproportionate sampling may be used. Proportionate stratified sampling selects individuals in proportion to their stratum’s size within the total target population. Disproportionate stratified sampling selects a predetermined number of elements from each stratum despite the relative size of those strata.

Sample Size in Random Samples Confidence in the generalizations drawn from a random sample is directly affected by sample size. Generally, larger samples permit greater confidence in population estimates and generalizations. A confidence interval is an estimate, plus or minus, of the value of the population estimate; it states the range in which we believe the true population estimate lies. Sample size is determined in light of confidence intervals and confidence levels. Greater precision in either or both levels requires larger sample sizes.  

Sample Selection Bias in Probability Samples Sample bias occurs when members of the population of interest are selected in violation of the basic principle of random sampling, that is, where each observation has an equal chance of being selected for inclusion in the sample. Sample selection bias prevents the conduct of sound research and can lead to inappropriate conclusions about a sampled population. The sample planning process should therefore include an explicit discussion of how sample bias might be introduced into the study and how the sampling techniques used in the research served to eliminate identified potential sources of bias.

Bias and Online Panels. Researchers are increasingly turning to online panels as their source of respondents. When using panels, a researcher identifies target population characteristics and the desired sample size, and then the appropriate number of individuals with the specified characteristics is randomly selected from the panel for participation in the research. The underlying assumption of panel use is that panel characteristics mirror that of the broader U. S. population. Invitation-only panels are preferred over “opt-in” panels because this recruitment technique helps to reduce “self-selection” bias. Panels should aggressively and continuously identify “professional respondents” and immediately expel these individuals from the panel. All panel members’ demographic and other defining information should be verified. Panel demographic composition should be verified, and the panel itself should mirror the general U.S. adult population. Researchers should not have to resort to weighting results to compensate for the lack of panel representativeness. Panel response rates should be monitored and be made accessible to all researchers contemplating use of the panel. Consistently unresponsive panel members should be eliminated from the panel. The quality of information provided by panel members should be continuously monitored. Panel members who consistently provide poor information (e.g., by providing the same choice for all questions or completing surveys in too short a time frame) should be eliminated from the panel.

Nonprobability Sampling and Quantitative Research The previous section discussed three types of probability samples: simple random, systematic random, and stratified random. Each of these forms of sampling obtains probability samples because all elements in the defined universe have an equal chance of being selected. However, in spite of its advantages, not all advertising research uses probability sampling.   

Judgment Sampling Judgment sampling selects individuals from the target population based on an expert’s judgment of who might be the best to interview. The expert may be the researcher, others at the agency, the client, or a specialist with particular expertise. A storekeeper, for example, may decide to sample what he considers the “typical” customers of his business.

Quota Sampling Quota sampling is an extended form of judgment sampling. It attempts to ensure that demographic or other characteristics of interest are represented in the sample in the same proportion as they are in the target population. Quota samples are obtained through the following five steps: Determine the defining characteristics of the key subgroups. Determine the percent of the total population represented by each defining characteristic. Determine the percent of the total population represented by each quota cell. Translate the percent into a sample size. Sample the population.

Snowball Sampling Snowball sampling uses current study participants to help recruit future participants from among their friends and acquaintances. Thus, the sample group grows like a rolling snowball. Snowball sampling is typically used for very small, hard-to-reach, or highly specialized populations of individuals; populations where access is facilitated through personal introductions.

Sample Size in Nonprobability Samples The nature of nonprobability samples precludes the use of statistical techniques to determine confidence intervals and associated sample size. As a result, sample sizes in nonprobability research typically reflect some form of judgment. Some forms of judgment, however, are better than others. Unaided judgment is the most arbitrary approach to nonprobability sample size determination. What is the budget? A second approach reflects budget considerations. Frame of reference is a more reasonable approach to nonprobability sample size determination, where sample decisions follow the practices of others. Analytical requirements is probably the best method for determining nonprobability sample size. It is recommended that the total number of individuals or observations in major study subgroups total at least 100 while there be a minimum of 20 to 50 individuals in minor analytical groups.

Sample Selection and Qualitative Research Purposive sampling is the most common form of qualitative sampling. In this approach, a researcher starts with a specific purpose or information need in mind, and the sample is then selected to include only those people who in the judgment of the researcher will be able to provide information relevant to satisfying the information need.  extreme or deviant cases where individuals who are “outliers” are selected. Here, a researcher looks at what rarely happens in order to better understand what usually happens. typical cases where “average” or “typical” individuals are selected. Interviewing of individuals in this group is typically most productive after insights from interviews with the prior group are completed.  highly intense or passionate individuals who may not be extreme in their behaviors but are highly involved in the area being explored.  confirming or disconfirming cases where the attitudes or behaviors of individuals selected either support or negate the researcher’s pre-existing perspective. 

Applying Chapter Concepts This chapter introduced the procedures underlying both random and nonrandom sampling and provided guidance on the appropriate use of each sampling technique. While the procedures discussed remain constant across different research situations, the details of each sampling plan are selected to specifically respond to the research’s goals and informational needs. Summary The sampling process involves the selection and examination of the elements of a population for drawing conclusions about the larger population of which these elements are members. A good sample is efficient and provides reliable generalizations about the larger population. All sampling begins with a definition of the target population, the group of elements about which you wish to make inferences and draw generalizations. A well-defined target population unambiguously describes the group of interest and clearly differentiates those things or individuals who are of interest from those who are not. A determination of the sampling method occurs next. Given the informational needs motivating the research, and the time and financial considerations, either a probability or nonprobability sampling technique will be selected. A probability sample is when each individual, household, or item comprising the universe from which the sample is drawn has an equal chance, or probability, of being selected for inclusion in the research. The selection of sample elements is done purely by chance. A nonprobability sample is when the elements are not selected strictly by chance from the universe of all individuals, but are rather selected in some less random, often more purposeful way. Probability Sampling Probability sampling techniques require an additional three planning steps. First, a sample frame must be determined. A sample frame specifies the method you will use to identify the households, individuals, or other elements specified in the target population definition. You can take one of two approaches to specifying the sample frame. You can either construct or obtain a list to represent the target population, or when a list is incomplete or unavailable you can specify a procedure such as random digit dialing for identifying and contacting target individuals. Once a sample frame is selected, it is compared to the target population. A perfect sample frame is identical to the target population; that is, the sample frame contains every population element once and only once, and only population elements are contained in the sampling frame. Typically, however, sample frames are either too broad (over-registration) or too narrow (under-registration). In these latter instances, modifications in the sampling plan can be made to take into account the sample frame’s characteristics. Second, a specific probability sampling technique is selected. The most common forms of probability sampling are simple random samples, systematic random samples, and stratified random samples. Simple and systematic random sampling work well when the target population displays little variability among demographic, geographic, or behavioral subgroups. When wide variability is thought to exist, stratified random sampling (using either proportionate or disproportionate sample selection) is recommended. Third, statistical techniques are used to determine the most appropriate balance between required sample size and confidence intervals, that is, the range of measurement error. Nonprobability Sampling The most common forms of nonprobability sampling are convenience sampling, judgment sampling, quota sampling, and purposive sampling.