6 Key Concepts of Experimental Design

Experimental research design is a crucial skill for new students to understand and, eventually, master. A good experiment design is absolutely necessary in order to reach objective conclusions from a study. Here, we cover the main concepts that every student should keep in mind.

1. Clearly Define Your Experimental Variables

Variables must be clearly defined from the very beginning. Most studies will want to examine the relationship between a(n):

• Independent variable: The stimulus, condition, or intervention you are manipulating and controlling.
• Dependent variable: The subsequent effect of the change in the independent variable.

Consider the research question, “Does observing an emotion affect the participant’s perception of written language?” You might ask this if you are studying perception and interested in the effect of priming an emotion on how written language is interpreted.

For example, imagine the participant perceiving a happy person followed by reading various text prompts like “Don’t talk to strangers,” followed by the participant having to report how they perceive the text prompt.

In this example, your independent and dependent variables could be any number of things and it is your duty as a researcher to operationalize and define them.

• For the independent variable: images of different emotions or audio of voices expressing different emotions
• For the dependent variable: a Likert-scale assessing different dimensions of the text prompt or even a question measuring likelihood “How likely is the person you perceived to say this command?”

Independent and dependent variables are at the heart of all research, but good experimental design also considers other variables too, such as:

• Control variables
• Confounding variables
• Extraneous variables

If you conceptualize and rigorously take into account all of your variables at the planning stage, designing web-based experiments and setting up variables in Labvanced’s editor becomes just a matter of learning a few clicks.

You can also upload various stimuli, from visual to audio, and test participants’ behaviors through features like eye tracking. Check out this playlist on how various stimuli are created in Labvanced and treated as objects to build experiments!

2. Understand the Building Blocks of Experimental Design

To be good at experimental design and establish a design to support your research question and isolate your variables, you must know the building blocks of experimental design like the back of your hand.

Namely, these are:

• Trials: single instances of a particular task or cued response to a stimulus presentation, like completing one maze run or viewing a string of letters.
• Blocks of trials: A block is composed of multiple trials, usually an experiment will have multiple blocks.
• Sessions: Relating to longitudinal design, experimental sessions can stretch across multiple days (even years) where participants perform the trials again at different points in time.

Trials

Trials are the basic units of an experiment which when bundled together they form blocks which make up sessions. Trials are typically single instances of a stimulus.

Watch this video that explains how to build multiple trials showing images of pets (dogs and cats) and flowers, prompting the participant to pick the pet in each trial, using Labvanced:

Blocks

A collection of trials form a block. Researchers use these blocks for many reasons, including to randomize their study. Another common approach to blocks is to give them a theme that is related to the research question or to relate them to the different learning stages that it takes an experiment

Examples of blocks in research:

• Thematic question: Trial blocks can have a certain research theme they follow. For example, blocks can follow a certain cognitive theme like aspects of working memory or even the form of the actor where presented movements are grouped by directional and velocity characteristics (Cook, Pan, & Bianchi-Berthouze, 2013).
• Time-element: Blocks that have to do from ‘practice’ to ‘training’ to ‘experimental,’’ typically seen in a within-subjects study design. For example, a study using Labvanced assessing childrens’ performance on the serial reaction time task in a language experiment, used blocks and measured differences between participants’ performance in the first training block versus the final block (Marimon et al., 20021).

Sessions

Sessions refer to different points of time that participants provide responses to a study.

Consider this study by Polonen, Lappi, & Tervamieni (2019) in Frontiers of Psychology where the aim was to assess the impact of movement on the affective state and flow in Qigong practitioners where the first session is a mix between reflection and exercise blocks.

Later sessions are typically on different days (even months) and can either be the same trials and blocks used before, different ones, or a mix between new and old blocks.

In a recent study by Chládková et al., the researchers recruited participants for Session 1 in which 161 participants were tested and for Session 2 where 202 different participants. The goal was to study the effect that a month-long mask-wearing mandate has on speech perception. To do this, the researchers had Session 1 before the mandate and Session 2 after the month-long mandate. For both sessions, the stimuli were exactly the same, namely a collection of 48 trials that took about 10 minutes to complete, assessing the strength of the McGurk Effect.

3. Between-subjects and Within-subjects Design

In the example above, discussing the McGurk Effect experiment by Chládková et al., a between-subjects design (session 1 vs. session 2) was employed!

Sometimes there are different conditions or treatments that researchers want to study. In this case, either a between-subjects or within-subject design should be selected.

Between-subjects Design

Also known as ANOVA or independent measures design, in between-subjects design participants can only be in one condition or treatment from the many that are being studied in the experiment.

Typically, this is the classic experimental design where participants are either assigned to be in the control group (sometimes where a Placebo Effect is observed) or the experimental group.

On a surface level, use cases that typically use between-subjects design are studies in:

• Social Psychology
• Personality Psychology
• Decision-Making studies

Some advantages of repeated measures/between-subjects design includes:

• Shorter sessions: Because a participant is not expected to go through multiple conditions or treatments (as is the case of within-subjects design), the sessions are shorter.
• Reduces carryover effects: Since a single participant is not expected to complete multiple conditions or treatments, there is less of an effect from being in one condition followed by another. Thus, researchers don’t have to worry about things such as learning effects or fatigue effects as is the case in within-subjects design.

Within-subjects Design

Also known as repeated measures design, in within-subjects design participants participate in one condition or treatment and then another condition or treatment. All longitudinal studies that track participants overtime follow a within-subjects design because they are looking at the same participants over and over again.

This design is popular for when studying accumulative effects, something that is expected to emerge overtime. Thus, by measuring the same participant over and over again, the desired outcome can be recorded overtime as it emerges.

Counterbalancing is used by researchers to ensure that the order that the treatment is received does not affect the observable results. For example, in a counterbalanced repeated measures study, Sally experiences Condition A before Condition B but Laura goes through Condition B before Condition A.

Some advantages of repeated measures/within-subjects design include:

• Less participants needed: Because more data comes from a single participant, it is possible to detect correlationaship or even causal relationships using small samples.
• Minimize noise: Because data comes from the same participant, individual differences (i.e. noise) are minimized since factors like memory or intelligence do not change from condition to condition.

On a surface level, use cases that typically use within-subjects design are studies in/focusing on (Keren & Lewis, 2014):

• Cognitive & Neuro Psychology
• Perception
• Memory
• Learning
• Psychophysics
• Reaction time

4. Randomization & Balancing

Randomization is the random assignment of participants to different experimental conditions and is at the heart of experimental design because it helps eliminate bias and the effect of unknown variables which could statistically explain any significant results. (Suresh, 2011).

Balancing, on the other hand, is a way to ensure that the different experimental groups have equal (or as similar as possible) number of participants assigned to them. For example, consider that your ‘control group’ has 5 participants and your ‘experimental group’ has 95 participants. Sounds disastrous, doesn’t it? A more balanced design would be having about 50 subjects in the ‘control group’ and 50 in the ‘experimental group.’

While all areas of psychology benefit from randomization, clinical and health psychologists pay extra attention to this because they are oftentimes dealing with interventions and treatments.

Using Labvanced, your experiment can have randomized trials, tasks, blocks, and groups, depending on the nature of your study. It is possible to randomize anything that you want and you can ensure balancing between groups ensuring that there is an equal number of subjects between groups.

Here are some ways to balance and randomize your study using Labvanced:

• Random factors: achieve simple, nested, or between-subject balance of stimuli
• Random trial sequence: upload a predefined trial sequence for each subject or create an adaptive / dynamic trial sequence using events
• Event logic: create highly individualized balancing mechanisms, e.g. assign files dynamically to objects

Having a balanced and randomized study is a must because it helps to ensure the integrity of your statistical analysis and results.

5. Replication

Replication is one of the oldest issues in the field of psychology, dating back to observations from leading psychologists in the 19th and early 20th century (Laws, 2016).

To this day, replication, which refers to how repeatable a study can be (typically to determine if the findings from an original study can be observed in other situations and participants), is still an issue in experimental research.

Replication is important because for other researchers to confirm or test the findings of a study, they must be able to replicate the study. This is difficult in psychology because behavior, as a general rule, is very complex and difficult to predict.

One of the advantages of having your study online is that you can share the study template to other researchers and they can use the experiment or tasks within to test other populations and try to replicate your results.

6. The Power of Evidence

The more rigorously defined and set up a study is, the stronger the evidence is behind any subsequent conclusions. Thus, if you want to establish something with your study, you will need the evidence to support it.

There are many different systems git statusthat classify evidence (which can even vary by country and organization). As a general rule, you can consider evidence on a spectrum from ‘not very stringent or convincing’ to ‘tried and tested’.

Imagine someone’s aunt tells you her opinion on the effect of caregiving on infant perception based on her experience raising her three children. While this may be an amusing and delightful conversation to have, these insights are not reliable as the conclusions drawn from a randomized study with over 1000s of caregivers that were assigned to specific parenting styles and their childrens’ response was measured (all the while controlling for confound variables)!

By understanding the power of evidence, you can design an experiment where the subsequent conclusions are sound, reliable, and not tainted by confounds and/or bias.

Concluding Remarks

There is nothing more exciting than designing your first experiments as a budding researcher. With these key concepts in mind, you can start strong and develop the foundation for more understanding advanced concepts like interactions. And, building experiments online with Labvanced can help you continue growing as a researcher as you gather data quickly, design valid experiments, and increase your publication output!

And, remember, how you address and handle the 6 components outlined in this article determines the strength of your conclusion and even the outcomes of your findings. Also, keep in mind when performing experiments online experimental control plays an important role about how the experiment is handled in an online environment. Learn more about how Labvanced ensures experimental control here.

References

Chládková, K., Podlipský, V. J., Nudga, N., & Šimáčková, Š. (2021). The McGurk effect in the time of pandemic: age-dependent adaptation to an environmental loss of visual speech cues Psychonomic Bulletin & Review, 28(3), 992-1002.

Cook, J., Swapp, D., Pan, X., Bianchi-Berthouze, N., & Blakemore, S. J. (2014). Atypical interference effect of action observation in autism spectrum conditions. Psychological medicine, 44(4), 731-740.

Keren, G., & Lewis, C. (2014). A Handbook for Data Analysis in the Behaviorial Sciences: Volume 1: Methodological Issues Volume 2: Statistical Issues. Psychology Press.

Laws, K. R. (2016). Psychology, replication & beyond. BMC psychology, 4(1), 1-8.

Marimon, M., Hofmann, A., Veríssimo, J., Männel, C., Friederici, A. D., Höhle, B., & Wartenburger, I. (2021). Children’s Learning of Non-adjacent Dependencies Using a Web-Based Computer Game Setting. Frontiers in psychology, 12.

Pölönen, P., Lappi, O., & Tervaniemi, M. (2019). Effect of meditative movement on affect and flow in Qigong practitioners. Frontiers in psychology, 2375.

Suresh, K. P. (2011). An overview of randomization techniques: an unbiased assessment of outcome in clinical research. Journal of human reproductive sciences, 4(1), 8.