UX Design: Between Subjects vs. Within Subjects

User experience (UX) research is an important component of product and service design to ensure what a company is offering is meeting the needs of the end user. In this article, we will explore what user research is, compare between-subject and within-subject study designs, and assess the advantages and disadvantages of each method.

What is user research?

User research is a key component of UX design that focuses on using different methodologies to understand what motivates users, what their needs are, why they make certain choices, and what their goals are. It’s a “needs assessment” that helps designers create more user-friendly products by integrating the user’s perspective into the design process. This can help your product or service stand out in the market and retain customers more effectively.

Types of user research study designs

You can use several types of user research study designs, each suited for different research questions and contexts. In many cases, you might need to use a combination of methods to validate findings and look for overlap. User research is generally split into two categories: attitudinal and behavioral. Attitudinal research involves recording your participants’ words (such as through an interview), while behavioral research involves watching their actions (such as recording task completion times).

All of these designs manipulate an independent variable (the condition you alter) to see how it affects a dependent variable (the outcome you measure). Independent variables can be variables that the researcher manipulates or variables that cannot be standardized across participants, such as subject characteristics (age, race, education, etc).

For instance, in UX research, the independent variable could be different designs of a website, while the dependent variable might be the time users take to complete a specific task. You could divide your test subjects into groups and present each with a different design option. The different design options will be a manipulation of your independent variable. The time it takes users to complete the task could change based on these modifications, making task completion your dependent variable. This type of experiment could help you gain insight into which website design is most intuitive for users to use.

Ways to assign conditions to test participants

Once you select your participants, you will need to determine how to assign your participants to each condition. You can assign conditions to test participants in a few ways. Random assignment is a common method, ensuring that every participant has the same chance of being assigned to any group. This method helps ensure that any differences between groups are due to the independent variable, not pre-existing differences among participants.

Between-subjects vs. within-subjects study designs

What mainly differentiates between-subjects and within-subjects study designs is the number of conditions of the independent variable the participants are exposed to. In between-subjects studies, each participant experiences one condition, whereas in within-subjects studies, each participant experiences all the conditions of the independent variable. Let's take a closer look at the characteristics of each type of study design.

Between-subjects study design

In a between-subjects study design, also called independent-groups design, you expose each participant to only one condition of the independent variable. In this type of design, you will typically have a control group and one or more experimental groups. You should expose each experimental group to a variation of the independent variable, and the control group should have no treatment, a false treatment, or a placebo. You can then measure changes in the dependent variable between groups to gain insight into its relationship with the independent variable.

Additionally, you can use this type of study design when you have participants that vary on a certain subject characteristic, such as a demographic factor. In this case, you could expose your three groups to the same product or service. This could help you gain insight into how their subject factor affected their response to the exposure.

Examples of between-subjects study design

In the first version of between-group study designs, you are modifying a certain product or service and seeing how your participants respond differently based on the modifications. Some examples of this would be:

• Testing two new versions of a website: Expose your control group to the original website and expose each of your experimental groups to one of the new designs. Then measure how long participants in each group stay on the website.

• Testing a fitness plan: Have your control group continue in their normal gym routine and your experimental group follow your fitness plan. Measure physical fitness changes in each group at the start and end of a certain period.

When using this method to test differences in subject characteristics, examples may look like this:

• Assessing how different age groups respond to a perfume scent: Divide your experimental groups into ages 20–29, 30–39, 40–49, and 50+. Then expose each group to the perfume scent and compare ratings on a scale of one to ten.

• Assessing how people with different mental health disorders respond to a treatment plan: Divide your participants based on their mental health diagnosis. Then record survey responses of symptoms before and after the treatment period.

Advantages of between-subjects study design

The between-subjects study design has its own set of advantages and disadvantages, which can make it more suitable for certain situations while posing challenges in others. Since each participant only experiences one condition, you don’t have a risk of order effects or changes in performance due to the order of presented conditions. This can be particularly useful when exposure to one condition might affect responses to the other condition. In addition to this, being only exposed to a single condition means the testing session can be shorter, have a simpler set-up, and decrease the likelihood of fatigue affecting the results.

Disadvantages of between-subjects study design

A between-subjects design typically requires a larger sample size to achieve the same statistical power as a within-subjects design. This is because individual differences can contribute more to the variability in the dependent variable, making it harder to detect a significant effect.

Within-subjects study design

In a within-subjects study design, each participant experiences all conditions of the independent variable. This type of experiment is also called repeated measures design or dependent groups because the measures are all collected from the same subject group. You can use this type of experiment in longitudinal studies, where participants may have data collected over time.

Examples of this study design

Some examples of a within-subjects study design being used in practice include:

• Determining which advertisement is more appealing to users: You show each participant five different advertisements and measure participant rankings on how likely they were to click the advertisement.

• Determining how people’s physical appearances affect how hirable they are: You have participants given similar resumes for six different candidates for a job. You show pictures of each candidate and measure how different appearance characteristics change the likelihood of being hired.

Advantages of within-subjects study design

Within-subjects study designs typically have higher statistical power than between-subjects study designs. In other words, the effect of the independent variables on the dependent variable is more effectively detected in this type of experiment. Because you expose each subject to each condition, you get less error variance caused by natural differences in subjects. Essentially, the subject is their own control group, and differences in responses to the exposures cannot relate to extraneous subject characteristics such as age, upbringing, education, and so on.

Disadvantages of within-subjects study design

That being said, this type of experiment design can be impacted by which order you expose the participant to the different conditions. Because you cannot expose different conditions simultaneously, the researcher has to gather responses from one condition, then the next, and so on. This can lead to a “carryover effect,” which represents how a participant’s behavior on the second or subsequent exposures is influenced by exposure to the first and previous exposures. Carryover effects can lead to participants performing better on subsequent tasks, rating attributes or qualities differently based on their first exposure, or decreased performance due to fatigue or boredom.

Between subjects vs. within subjects: Which one to choose?

The decision to opt for a between-subjects and a within-subjects study design depends on your research questions, your stakeholders, your data collection requirements, and logistical considerations.

Between-subjects designs can be beneficial when exposure to one condition could influence responses to other conditions. This type of design is also useful when the testing procedure is long or strenuous, as participants only need to attend one session.

Within-subjects designs are powerful for detecting differences between conditions because each participant is also their own control. However, they can be subject to order effects, and you may have to vary the order of conditions between participants to help mitigate this issue.

When choosing, think about the advantages and disadvantages of each method as compared to your study design. The methods typically differ in the number of sessions required, how they can assess conditions you’re testing, the setup of the experiment, and the length of each session. Each factor can influence the suitability of the study design for your needs.

Getting started with Coursera

Continue exploring user experience research with exciting course offerings on Coursera. To build your user design toolkit and gain job-ready skills, consider completing the Google UX Design Professional Certificate. You will have the opportunity to learn in-demand user design skills that prepare you for an entry-level career in under six months.