Tipps & Tricks

Selecting question types

We usually talk about three main types of questions, where in most surveys it would be sensible to use a combination of these:

1. Closed questions In a closed question the respondent give their answer by selecting from a list of per-defined options (multiple choice). This makes it quick and easy to answer the question and makes the analysis work simpler. At the same time you risk having an incomplete list of options in terms of every possible answer to the question. It is, therefore, imperative to work thoroughly with the response options so that you can ensure the list is as exhaustive as possible.
2. Open questions In an open question with a free text answer field, the respondents formulate the answer themselves without a multiple choice being predefined. The advantage of using this type of question is that the respondent thinks and answers more freely than when choosing from multiple choice options, and this could bring in answers that are both surprising and useful. The disadvantage of using open-ended questions is that interpreting and processing the answers later on is more labor-intensive. When using open questions it is important to be clear about what you want an answer to. Write the question out in full instead of just “comment”, “input”, etc. Otherwise you might get answers to something else entirely than originally intended.
3. Combination of closed and open questions By using a combination of closed and open questions you can increase the likelihood of getting in every possible answer. This method is commonly used when you want to use multiple choice questions, but are not sure if all the response options are covered.

Forming the Questions

A poorly formed survey can make drawing valid data and conclusions difficult. Below we look at the elements that are important to bear in mind when forming the questions.

Language use in the questionnaire

In all questionnaires the semantics and choice of words are often more important than even the statistics. This means that the superstructure of a survey is, in principle, the language used, even though the statics form the basis for reporting of results. It is imperative that the questions included in a survey have certain characteristics:

• They should be relevant in the sense that they have an obvious connection to a clearly defined problem.
• They should be simple and expressed in a conversational tone. They should be adapted for the weakest link in the chain.
• They should be unambiguous, meaning there should be no risk of them being misunderstood or confused with other ideas, concepts or problems .
• They should not come across as hurtful, offensive, annoying or inflammatory.
• They should not contain slang, abbreviations, specialist/pompous terminology, etc. Ensure, to as great an extent as possible, that all the respondents will understand all the questions, and that they are understood in precisely the same manner!

One possible answer vs. several possible answers

A common mistake is using questions where more than one answer is possible, when there should only be one answer possible, and vice versa. It is, therefore, important to a have clear opinion about this in each question where this might be a relevant issue.

Non-leading questions

Make them as neutral as possible. Asking questions in a way that leads the respondent to certain answers must be avoided.
“Ecological damage is the most important challenge that Norway is facing. How much do you agree or disagree that the parliament should grant more funds for environmental purposes?”
In the example above, it is taken as a given that the most important challenge Norway is facing is damage to the environment. Therefore, it can be concluded that this question is not neutral but leading.

Mutually exclusive response options

An often encountered source of error is that the response options are not mutually exclusive. It’s easy to make a mistake and, for example, set up age categories as follows:

How old are you?
Below 20 years
20-30 years
30-40 years
40-50 years
50 years or older
In this case the respondent is asked for only one answer, yet if the respondent is 30 years old, for example, there are two options that would apply. Below is an example of how the response options could rather have been set up:

How old are you?
Below 20 years
20-29 years
30-39 years
40-49 years
50 years or older

Collective exhaustive response options

It is important to include response options that cover every thinkable (and, maybe even unthinkable?) answer a respondent could give to a question. If the number of response options is inadequate, this could be a possible source error when you are interpreting the results.

What is most important to you when choosing a petrol station?
Availability
Chain
Price

In this case a number of response options is missing, such as selection, opening hours, others. If the respondent does not find an adequate list of response options, they will not be able to give the right answer and you will not have gotten an answer to your question.

Correlation between questions and response options

There should be a connection between phrasing and word choice of the question and the response options, so as to prevent the respondent from misunderstanding and to avoid the risk of the results becoming invalid. It also gives the impression that the survey is not properly prepared and is therefore not of great importance to the sender.

How satisfied or dissatisfied are you with Company AS?
1 (Very bad)
2
3
4
5
6 (Very good)

Don’t take anything for granted

The people answering your survey must be allowed to not know, or not have, more or less, fixed perceptions or attitudes to something that does not affect them in their daily lives. This is why it is important to assure ourselves that what we mean by a word, idea/concept or constructs is also understood in the same way by those who answer, and we must be aware that words and ideas can mean different things to different individuals and different groups.

Ask about one thing at a time

Do not ask about more than one issue in the same question. Otherwise, you will have problems determining what the given answers actually apply to afterwards.
Example: How satisfied or dissatisfied are you with the service and information you receive from Company AS?
The example asks about both information and service. The question should rather be about only one of the items.

Back