Section 1.5: Sources of Errors in Sampling

1. 1.1 Introduction to the Practice of Statistics
2. 1.2 Observational Studies versus Designed Experiments
3. 1.3 Random Sampling
4. 1.4 Bias in Sampling
5. 1.5 The Design of Experiments

In general, there are two types of errors that can result during sampling.

Nonsampling errors are errors that result from the survey process.

Examples of nonsampling errors might be nonresponses of individuals selected to be in the survey, inaccurate responses, poorly worded questions, poor interviewing technique, etc.

Sampling error is the error that results from using a sample to estimate information regarding a population.

There's really nothing we can do about this second type. Unless we sample every single individual in the sample, there will be some error in our results. Much later in the course, we'll talk about how we can actually get an estimate for how close we are to the true population information we're trying to get at.

Since we can't control the sampling error, we'll focus in this section on the different types of nonsampling errors. There aren't a lot of graphical ways to represent this material, so I'll just summarize each source of error here.

The Frame

Surveys of voters or even of ECC students require a complete list of all the individuals. If an individual isn't on the list, any sample taken won't be representative. A common example of this is surveys over the phone - think of the types of people who either don't have land lines, have caller ID, or maybe change phones so often that they're not on the list. Any survey done via the telephone is clearly suspect. Unfortunately, it's often the only practical option for pollsters.

Nonresponse

Any survey will always have a portion of those sampled who simply don't respond. At ECC, we do an annual employee satisfaction survey. The people in the Institutional Research office are ecstatic with a 40% response rate.

Check out this link from the SuperSurvey Knowledge Base with a more detailed description of some reasons.

Interviewer Error

Have you seen the movie, Kinsey? It's a movie based on the life of Dr. Alfred Kinsey, who formed the Kinsey Institute, which published the Kinsey Reports about the sexual behaviors of men and women. During research. Kinsey and his colleagues performed countless in-person interviews. Imagine what a difference the quality of the interviewer would make in a context like that!

Misrepresented Answers

A classic example here is a survey I've done in my developmental classes about how often students study. Because of the nature of the variable in question, it has to be self-reported, but many students misjudge or even lie about how much they're really studying.

Data Checks

There's nothing like finishing your research about how many children the typical family has and finding that outlier - 45! Chances are, it was most likely an incorrectly entered 4 or 5, but it may be too late at that point to find out. It's crurical to check your data for accuracy at every stage of your analysis.

Next we'll focus on specifics regarding the design of questionnaires.

 

Types of Questions

In general, there are two types of survey questions - open and closed.

Each design has its own limitations - the open question makes compiling the data difficult, while the closed question limits the responses. A good compromise is to first give a "presurvey" with open questions, and then use the most common responses from that survey to form the actual survey with closed questions.

Wording and Ordering of Questions

Here are a few good links to read for more information on question wording:

• Abortion, the Court and the Public, from the Pew Research Center
• Why Question Order Changes Poll Results, from CBS News
• Question Wording, information from the American Association for Public Opinion Research
• Conducting a Survey In Your Community, the department of Human and Community Development at the University of Urbana Champaign. (Note: The first page asks for a login, but this is optional.)
• What is a Survey?, a pamphlet (downloadable or viewable online) originally created by the American Statistical Association

 

<< previous section | next section >>

1 2 3 4 5 6 7 8 9 10 11 12 13

This work is licensed under a Creative Commons License.