A comprehensive guide to designing effective surveys; For those who believe in data collection!

You might think that designing a standard survey takes hours or requires reading a lot of articles and books. But you should know that is not the case! Creating meaningful effective surveys and streamlined forms comes down to a few basic steps and smart tips.

In order to design an online questionnaire and survey, all you need to do is to follow the right path and be aware of the effect of question type, answer choices, writing style, tone, literature, etc.

In this guide, we will walk you through the steps of creating an online effective survey and questionnaire and give you all the dos and don’ts when designing an effective survey.

Reading this post takes only eighteen minutes. And while applying the recommended steps, you just need to pay some attention while you do so.

The first step: Determining the perspective and purpose of the survey

One of the important reasons why research and survey designers get confused is the failure to determine the purpose of the effective survey at the beginning of the work. In fact, you cannot ask accurate and appropriate questions until you are clear about your task and your expectations of the results. So in the first step, answer some important questions:

• What is my main purpose in designing an effective survey?
• What is my purpose in asking each question?
• What will I measure with an online survey?
• How do the final answers help me make a better decision?
• Who is the audience of the survey or the statistical population of the research and survey?
• Can I get the information I need from the target population with an online effective survey?
• What information do I need about my target population to analyze the results?

Answering the above questions will help you:

a) specify your needs, b) design only questions that you need answers to for further actions or decisions, c) define your target population, d) make sure whether creating an effective survey in the first place is a good solution for you or not!

The second step: Determining the target or statistical population

Whether you plan to conduct a survey or market research for the company or collect data for a thesis or personal study, you should define your target population before designing an effective survey.

The target population or the statistical population is defined as a specific group of the population that is categorized under one class or group due to one or more common characteristics.

Groups such as women working in private companies, human resource managers active in startups, customers of energy drinks, or incoming students of Oxford University in 2023 are all examples of the target groups and statistical population of your research and survey.

By determining your target audience, you can not only design more precise questions but also choose better methods for distributing your effective survey.

For example, when a sportswear brand wants to evaluate its brand penetration or brand awareness, it determines its target audience (men and women who are athletes and sports lovers), who can be spotted on sports news pages. Therefore, by publishing the survey on related media, the desired data can be easily obtained.

The third step: designing questionnaires and surveys

Now that your task regarding the purpose of the effective survey, the required data and the target community is clear, it is time to design your questionnaire and survey.

Note: Since designing paper surveys is not our topic in this post, we will only explain how to design an online survey along with its standards. Of course, with the development of the internet and considering the costs of other methods, fewer people will consider using paper surveys.

Designing effective survey questions – open-ended or close-ended questions

Generally, when designing surveys, we are faced with two types of questions, open-ended and close-ended.

Open-ended questions are questions that do not present the respondent with a list of options and he/she can express his opinion without restrictions.

These types of questions are used in qualitative research and are suitable for discovering the unknown. For example, if you don’t know what your audience needs or you can’t really understand the reason why they don’t buy from you, an open-ended question can clear up your confusion.

Questions like «What features do you think product X should have?» and «Why did you stop buying our product?» are examples of open-ended questions.

Unlike open-ended questions, other types of questions, called close-ended questions, are common in many measurements and quantitative research.

In close-ended questions, the audience is faced with a list of options. Analyzing the results of these questions is easier and in many cases, gives you meaningful results as long as you define your list of responses without bias.

Multiple-choice, Likert-scale, or rating/ranking questions are examples of close-ended question types.

To learn more about the types of survey questions, read the post on choosing the type of question in the survey:

A guide to choosing the type of question in the online survey

The effective survey structure

Each survey consists of three sections: introduction, main body, and conclusion.

The introduction of the survey

The introduction of the survey is a place to explain the reasons for the research, show instructions for the answers, and motivate the respondents to participate. In this introduction:

• Explain to the audience the topic and importance of your research, evaluation, questionnaire or survey.
• Write down the number of questions and the answering time.
• In case of asking personal questions, be sure to inform the respondent of the confidentiality of their answers. Or, by conducting an anonymous survey, prevent receiving invalid information.
• Include instructions for filling out the survey in plain language.

In the Porsline software, you can create a customized welcome page for each survey and provide the necessary information to the audience upon opening the survey.

The main body of the survey

In the main body of the survey, ask the screening questions first, then ask the main questions and bring the demographic questions at the end.

Screening questions are questions that help you prevent the completion of the survey by people other than the statistical population of the study at the very beginning. For example, if your survey is to be completed by people over the age of 20, by adding an age screening question, you can easily separate the responses from people under that specific age.

Demographic questions are questions that are chosen to know the characteristics of the target population such as gender, employment, study habits, income, etc. These questions make it possible to analyze and classify the results.

The main questions of the effective survey are the questions you write to achieve the goal of your research. To design such questions, use the ready-made survey templates prepared by Porsline or prepare the questions you want by selecting among various question types.

The survey conclusion

After completing the questions, you need an appropriate conclusion to your survey. In this section:

• Thank the participant in the effective survey and research.
• If your survey contains the correct responses, put them at the disposal of the respondents. You can share each respondent’s answer link with them using the «Show Answer Page» or «Show Quiz Key»
• Redirect the participant to the site’s home page or other preferred pages.
• Ask the participant to send the survey to his/her friends and other acquaintances.

Types of questions and their applications

The types of questions you will deal with depending on your goal are the following:

Short Text questions

This question type is useful when the answer you want is one or two words.

Common uses:

Receive audience identity information and quiz questions with single-word answers.

Common examples:

Please write your first and last name.

What is the name of your organization?

What is your job position?

What is the most vital organ of the human body?

Note: For questions such as receiving phone number, mobile number, zip code, national code, account number, and card number, use short text questions with short answers.

Long Text questions

When the answer to your question is more than just two words and you need to know the reason for something, use the long answer text question.

Common uses:

Receiving demographic information, audience persona research, descriptive tests of schools and universities, or even recruitment tests of organizations

Common examples:

What sites do you usually visit to get daily news?

How would you describe a typical day at work?

In your opinion, what features can make our product more useful for you?

What are the causes of earthquakes?

Multiple Choice Questions (and Picture Choice)

Multiple Choice questions are one of the best question types for surveys that get you detailed data.

Common uses:

Product survey questions, close-ended demographic questions such as age, gender, income, etc. along with marketing research questions used in competitor evaluation surveys or brand influence evaluation.

Common examples:

Which of the following options is the main reason for your purchase?

1. Product quality
2. Design and appearance
3. Price
4. Others

How much is your monthly income?

1. Less than five thousand dollars
2. Between five and ten thousand dollars
3. Between ten and fifteen thousand dollars
4. Above fifteen thousand dollars

Rating scale questions or Likert scale questions (Psychometric scale)

When you want to know the audience’s opinion or feeling about a certain topic or measure their agreement or disagreement with a statement, the Likert question type is suitable for your survey.

The Likert scale gives the audience a numerical range to measure satisfaction, agreement, disagreement, probability, or frequency.

Likert scale and its applications in organizational questionnaires and surveys

Common uses:

Customer satisfaction, studying customer habits, self-evaluations, performance evaluations of employees or managers, personality and psychological tests, and customer loyalty surveys.

Common examples:

Are you satisfied with our company’s after-sale service?

1. I am not satisfied at all
2. I am somewhat dissatisfied
3. I have no opinion
4. I am somewhat satisfied
5. I am completely satisfied

My manager adheres well to company processes and procedures:

1. I completely disagree
2. I somewhat disagree
3. I have no opinion
4. I somewhat agree
5. I completely agree

Rating questions

Measure the level of audience satisfaction with rating/ranking questions. This question type allows the audience to choose answers freely.

Common uses:

Employee satisfaction survey, service quality survey, customer experience survey, organizational loyalty survey.

Common examples:

How would you rate the quality of our services?

♥♥♥♥♥

Ranking questions

They are used to know the priority of topics in the customer’s mind or the order of importance of product features.

Common uses:

Demographic questions and audience recognition, market and marketing research, product testing, and product features.

A common example:

Which features are more important to you when buying a refrigerator?

1. The size and weight
2. Cooling efficiency
3. The spaciousness of the shelves
4. Beauty and appearance of the design
5. Warranty and after-sale service
6. Additional facilities such as ice maker and drain, etc.

Email/Link/Website/Numerical Questions

Questions that have a clear and unchangeable response pattern are very useful in designing smooth and practical forms. For example, let’s say you need to follow up and get the respondent’s email address or website after having registered a comment. By using the numerical or email question type, the audience can register their comments only by entering a valid email address.

Common uses:

Contact us form, order registration form, customer survey, job application forms, and conference registration form.

A common example:

please enter your email address:

Email template

File upload questions

File upload question allows you to receive any file; From photo and document scanning to video and PDF!

Common uses:

Registration forms, order registration forms, and job application forms.

A common example:

Please upload your resume here:

File upload option.

Order/Sequence of an effective survey questions

The order of the effective survey questions should be determined based on the purpose of the research and its audience. Since one of the goals of the survey designer is to simplify answering questions for the audience, the best order of questions is as follows:

• From easy to difficult.
• From short to long and time-consuming questions.
• From general to more specific questions.
• From general to detailed questions.
• From close-ended to open-ended questions.

In this case, the audience answers the questions with less reluctance and more motivation.

Another important issue is the placement of personal questions in the survey. For example, in demographic questions, the audience may be sensitive to providing personal information such as age, gender, religion, sexual orientation, income, etc. Therefore, it is recommended to ask such questions at the end of the survey and only if necessary. Also, it is important to inform the audience about the confidentiality of their information or the anonymity of the answers.

Demographic Information: What to ask? Why ask? How to ask?

Bias in the survey

Bias refers to a state of mind and unconsciousness that causes a preference for one state over another or one opinion over another. Bias in research leads to ignoring some facts which increases the possibility of errors in the survey.

In research and surveys, there is a possibility of bias applied by the survey designer (researcher bias), the survey itself (research instrument bias), or the respondent (respondent bias).

Researcher bias occurs when the question designer’s presuppositions lead to a tendency towards certain questions or choices. For example, if the designer of the question is in favor of the separation of England from the European Union and designs his/her question in such a way that people would agree, then bias has happened. For instance, ask: “What do you think about the separation of the UK from the European Union?” instead of asking: “Do you agree with the separation of England from the European Union?”.

Survey bias is a type of bias that depends on the writing style and choice of words in the survey questions, the types of questions, the answer structure, or even the answer options. For example, when you ask the respondent about their religion without including an option indicating that he/she does not wish to answer, the respondent may choose an incorrect option or refuse to complete the survey.

Respondent bias is another type of bias that occurs due to the respondent’s inability to provide the correct answer or his/her reluctance to provide an honest answer.

Inability to provide the correct answer occurs for reasons such as unfamiliarity with the question, the complexity of the questions, or factors such as fear of judgment and embarrassment.

Strategies to prevent bias

1. To avoid respondent bias and receiving incorrect data, using options such as I don’t know, I’m not sure, or I’d rather not say can be helpful.
2. It is necessary to use the screening questions correctly at the beginning of the online survey. For example, if you are reading about the quality of 4G Internet services, first ask “Do you have experience using 4G Internet?” If the answer is yes, the respondent can continue to answer. Otherwise, direct them to the end of the survey.
3. Sometimes, unwillingness to provide honest answers is due to people’s social tendencies to preserve their reputations or to the fear of being judged. For example, when asking for an opinion about donating to a charity, if you ask your questions in the following way, you are likely to get dishonest answers:

The first question: Do you think working children need support from other members of society? yes/no

Second question: Are you willing to pay an amount to support children? yes/no

To avoid such a situation, conduct your effective survey anonymously so that the audience can honestly express their opinions without worry.

Survey settings

Each online survey can have special settings according to your goals and needs; from Adding logic and calculations to settings such as determining the response time.

In the following, you will get acquainted with these settings (for a detailed introduction to the various features of online surveys, refer to the Porsline Video Tutorials).

1. Add logic and calculations:

Adding logic to any research helps the questions’ designer gather valid results. You get meaningful data by defining relationships among questions by adding logic and calculations to each question.

2. Authenticate respondents:

Sometimes, it is very important for the designer of the survey that a certain statistical population answers his/her questions, because the characteristics of this statistical population are part of the research variables. By limiting access to the survey, you can allow your target community only to enter and fill out the survey.

On Porsline, authentication is possible with identification codes such as national code, personal code, student number, mobile number, or email address. Also, you can use this feature to make the survey anonymous. It is enough to randomly scatter a number of codes within your statistical population representing randomly-selected respondents and then ask them to participate in your survey, research, or evaluation by entering those codes.

3. Use of custom variables:

In research and surveys where previous information from contacts is available to you and you need this information to analyze the results, you should pass this data to your survey. For this purpose, you can ask the audience for this data at the beginning of the survey or send specific links to each respondent if you want to design an effective survey.

Using the «Custom variables» feature of inserting hidden information on Porsline, pass the existing data from your audience or customers to your effective survey and create special links based on these information and variables.

4. Preventing more than one submission of a survey:

In many organizational, scientific, or academic research and surveys, checking the frequency of a variable or characteristic is considered one of the main goals of designing those surveys. Therefore, identifying duplicate responses can greatly reduce the error rate of the results.

The designer of the survey can recognize the repeated responses by placing repeated questions or changing the order of the questions.

In addition, on Porsline, you can prevent receiving duplicate and invalid responses by enabling “Prevent more than one submission” or “Identify spam responses”.

Important tips before sharing the survey

To really say that what you did was an effective survey design, you have to do the following:

1. Send the survey to a small part of your target community and test it.
2. In each survey, seek only one goal and measure one or two variables.
3. Write your questions without ambiguity or applying your taste and opinions.
4. Pay attention to punctuation, grammar, and correct spelling of words.
5. Use close-ended questions more than open-ended ones. These questions are less ambiguous, give you more reliable data, and don’t have different interpretations.
6. Give the audience a reason to complete the effective survey. For example, remind them of the importance of their participation at the beginning of the survey. Make them aware of the small number of questions or the short time to complete the survey, or give them an offer or discount code in exchange for completing the survey.
7. Ask about only one concept or variable in each question.
8. Give the audience freedom of action. For example, include the option “I’d rather not say” in questions about religion, age, gender, etc. Also, give them a range in numerical questions. For example, instead of asking about the respondent’s monthly income in the form of an exact number, show them the numerical ranges of the salary in several options.
9. Do not start the survey with uncomfortable, personal, difficult, abstract, or time-consuming questions.
10. Keep the number of survey questions limited. No one likes to spend a lot of time filling out a survey!
11. Do not imply a positive or negative meaning. For example, if you are a supporter of a certain political party, the audience should not know your positive opinion about this party. This way they can answer your question honestly without bias.
12. Keep the tone of the questions friendly and non-challenging.
13. Do not use general questions and give the necessary details to the respondent in each question.
14. Choose an attractive and distinctive coloring for your survey. The audience should not be bored by completing it. Also, your effective survey is a good opportunity to influence the mind of the audience; Design it in line with your organization’s visual identity.
15. Do not finish the survey without an invitation to take action. Ask the audience to send your survey to others.

Survey testing

Before sending your questionnaire and online survey to your entire audience or target community, send it to a subgroup and make sure that all parts of the survey are understandable to this specific group. For example, check the responses carefully or include a question at the end of the survey and ask them to share their opinion about your survey with you. After testing the survey and ensuring its transparency, share it with your entire statistical population.

Sharing the final survey

Use different channels such as social networks, messaging apps, email, SMS, or push notifications to deliver the survey to your contacts.

Remind your audience of the importance of their participation in your research. If possible, consider offering a gift or discount code to encourage completing the survey. In addition, you can share your survey through online advertising and pay a small fee for each submission.

Last word

Designing clear, short, user-friendly, and standard surveys with a little care and using the right tools will help you save time and money for yourself and others. Follow this path easily with Porsline survey builder software.

On Porsline you can:

• Use a variety of question types; from short/long text and multiple-choice questions to opinion, numerical, ranking, and rating questions.
• Add logic and calculations to your questions.
• Use custom variables to make the analysis of the results both meaningful and simple.
• Through API or webhook, link the survey to your database and keep the results on your servers.
• Design the survey template with your desired colors and image. Display your logo or choose a background image.
• Prevent each respondent from submitting more than once. Enable the «Prevent more than one submission» feature to identify duplicate answers or limit the response time of the survey to a certain interval.
• Share the survey through email, SMS, and social media.
• View results and statistical graphs in real-time and get advanced reports by using the «Apply Advanced Filters» and «Add Logic» features.