Precise Timing

Background & Context

Reaction time in psychology research is used to quantify cognitive processes and behaviors. A clear-cut definition of reaction time has to do with the amount of time passed between an appeared stimulus and the response.

There are two components to measuring reaction time, the stimulus’ time of onset and when the participant’s response occurred, illustrated by Fig.1.

Fig. 1: The two main components of measuring reaction time.

For reaction time to be measured accurately, the exact time of the stimulus onset (Point A) must be known, as well as when the participant’s response (Point B) happened as reaction time is the difference between these two points. From the two points, it is easy to determine when a participant’s response occurred, but it is challenging to know exactly when the exact stimulus onset occurred (Point A).

Why is it challenging to determine when Point A occurs? There are three main reasons that influence when a stimulus appears:

1. Screen refresh rate: The rate of monitor refreshing occurs at 60Hz so if something is scheduled to occur, it can occur only when the monitor is refreshed. While this is on a millisecond scale, it’s an important factor to quantify (which we discuss later how it is measured with the request animation frame) as it directly impacts the experimental sequence.

2. Nature of programming: All experiments are based on coding and for code to be executed, it must be processed as nothing is instantaneous, this usually takes 1-2 refresh cycles.

3. Device capacity: Though this is not common, if the participant’s device capacity is really slow, the stimulus presentation can lag as all of the system delays (like a computer freeze). We discuss later on how we check for this issue (the JavaScript Event Loop).

In summary, reaction time is affected by many factors upon which technological processes are built in order to accurately determine the time between stimulus onset and the participant’s response.

Publication in Behavior Research Methods:

Check out this peer-reviewed paper published in Springer Nature’s Behavior Research Methods in May 2022. The authors conclude that Labvanced has the most accurate reaction time measures compared to other web-based tools.

Our Process: Labvanced’s pipeline for precise timing

Fig. 2: The general pipeline for precision timing and capturing accurate reaction times in Labvanced.

To provide precise timing and reaction times, our software follows these steps (Fig. 2):

• Preloading (caching): Ensuring all experimental stimuli are loaded a priori to the experiment beginning and locally available so loading does not happen in the midst of experimental progress. So, if a participant wants to take part in a study, all the stimuli (images, audio,and video) are already fetched and loaded locally on their computer from our server.

• Pre-rendering: When the experiment begins, the content is recursively created so that the next frame and trial is loaded in the background and ready to go as soon as the participant is ready to move on. This is driven by a pre-rendering mechanism.

• Participant-Specific Measurements: Since online studies begin in the browser, each participant has finite computer resources (GPU, CPU) which must be kept under consideration as they affect performance. We capture any potential delay and provide it as a correctional variable to the researcher which can also be used as an exclusion criteria.

Saving Participant’s Responses

All experiments happen locally on the participant’s computer. Therefore, the internet is technically not mandatory for running an experiment. The internet is only needed at the beginning to preload the experiment locally and then at the end to upload the data and responses back to the server.

However, if the provisions are available, our software is set up so that data recording and responses are saved automatically after each trial. This is important because:

1. A local browser cannot hold or cache an infinite amount of memory. By backing up frequently, memory is freed and the system does not risk lagging.
2. If a participant stops or drops out, there is at least some data saved for the trials that they did complete and provide responses to prior to terminating their participation.

About the Timestamp

While the experiment is active, the Labvanced app does not have access to all of the other processes or parts on the computer. However, when recording something with reaction time, a timestamp is needed and the app can access the system time from the computer clock in order to determine when Point A (stimulus onset) and Point B (the participant’s response) occurred. Since the computer has a general system clock, this is the same no matter where / what you are doing or using.

About System Architecture and Reaction Time Data Flow

While the pipeline described above captures the basic steps of the reaction time process, below is a more detailed explanation of everything that is going on in Labvanced to make the reaction time measurement accurate and precise.

Preloading (Caching)

Fig.3: The main steps of the preloading/caching mechanism in Labvanced.

Preloading or caching occurs before the experiment even begins. Labvanced is set up so that all of the study’s experimental stimuli are downloaded before the study starts. This includes all of the elements, such as images and videos. They are all fetched from the Labvanced servers and downloaded locally to the participant’s device so that no downloading has to occur during the experiment itself (Fig. 3).

Pre-Rendering Mechanism

Fig. 4: The main steps of the pre-rendering mechanism in Labvanced.

We have a pre-rendering mechanism in place to build the structure of the experimental tasks, trials, and frames in advance. For example, if you are in Trial #1 of a task, we pre-render all frames in the current and upcoming trial so that loading does not happen during the experiment, including the instruction, text, audio objects, fixation cross, etc. By building the trials and frames in advance, it prevents the browser from slowing down or being overwhelmed (Fig. 4).

Participant-Specific Measurements

Because of the innate variability between devices and computers, performance is affected by the definition. Simply by running an experiment on a local system which are inherently limited with resources (ie. speed and memory are not infinite but constrained by their tech specifications), stimuli may not get shown as expected (there may be a delay of a few milliseconds, for example.

To capture these device- and participant-specific fluctuations, we have the following mechanisms in place:

• The request animation frame
• The JavaScript Event Loop

Request Animation Frame

Fig. 5: Demonstration of the request animation frame mechanism in Labvanced.

Every 60ms the monitor is independently updating and refreshing, this is a constant for all computers and screens. To determine whether there is a delay in presentation of the stimulus (on the millisecond scale), the request animation frame is used for all instances where a timed stimulus is occurring.

Let’s say you execute code to show stimuli at 2000ms, when you execute it nothing happens, the stimuli will be automatically presented at the next refresh rate, 60 milliseconds (Hz) later, at the 240ms mark. You can measure this tiny lag and account for it post-hoc. Because we use the request animation frame, you can know exactly when a command was executed (when it really happened/appeared on the monitor) and adjust accordingly (Fig. 5).

JavaScript Event Loop

Fig. 6: The steps of the JavaScript Event Loop, the CallBack Function, for determining computer speed using Labvanced.

Another example of participant-specific measurements has to do with determining the speed of their device.

If your computer is slow, it may be because there are active system processes running that use available CPU. Thus, the browser is working the limited resources that are available and as a result, everything gets slower.

To determine whether this is happening on the participant level, we use the ** JavaScript Event Loop using CallBack Functions** which runs automatically (by default) in the background to measure the amount of time it takes for the function to call back on itself. If it doesn't return within 5 ms, it means the participant’s browser/computer is slow which could affect the integrity of experimental results measuring reaction time (Fig. 6). We report the mean value in milliseconds that it takes for the CallBack Function to return for the participant.

For the thousands of studies that have been completed by participants in Labvaned, we have found that over 95% of participants have a reported value that falls below 3ms, sometimes below even 1ms. But in some cases, there are results that average 200-300ms which could indicate to the researcher to consider excluding that particular user’s data from the final data set analysis.

Key Features of Labvanced’s Reaction Time and Precision Timing Capabilities:

Our top features for measuring participants’ responses include (Fig. 7):

• Temporal accuracy of stimulus presentations
• Spatial accuracy of stimulus presentations
• Eye tracking accuracy and sampling rate
• Quantification and measurements of any device- and screen-related delays for each participant, allowing for standardization, comparability between subjects, and corrections. This occurs through the event loop precision and request animation frame described in the previous section.
  

  Fig. 7: The key features of Labvanced’s precise timing / reaction time solution.

Advantages of Labvanced’s Precision Timing

Because of these steps and mechanisms, Labvanced offers an accurate and precise solution to measuring reaction time during online experiments. We highlight the following advantages of our platform:

• Controlled timing of stimuli: Researchers have knowledge of the exact time that stimuli are presented on screen, allowing for adjustment and accurate measurements.
• Strong computational and programming mechanisms: To assure the researcher the most accurate data is being reported, we use strong computational and programming mechanisms in order to accurately quantify the onset of stimuli on the participant’s screen.
• Tried and tested: We have worked with researchers from all over the world to fine-tune our platform and as a result our features have been tried and tested by countless of research and academic institutions using our online reaction timing measuring as a basis for their studies and published works.

Sample Data & Metrics for Reaction Time

Fig. 8: Data report from a participant’s session performing the Stroop Task using Labvanced; 3rd column from the right demonstrates recorded reaction times.

Things You Can do with Labvanced’s Precision Timing:

• Alertness
• Cognitive decline
• Perception
• Performance measures
• Feature recognition
  

LV Library Studies:

There are many studies that measure how long it takes for a response to a stimulus to occur, here are a few examples of tasks that have reaction time measurement at their core:

• N-back task: A cognitive test measuring working memory capacity. Stimuli are presented and the participant is challenged to match whether it is the same stimulus presented n-steps back.
• Stroop task: This classic task demonstrates how increased reaction times occur when a participant is presented with incongruent stimuli (a word that says ‘yellow’ but is colored in blue).
• Face recognition: Face recognition is deeply ingrained in human nature and reaction time can be measured for how long it takes the participant to recognize or discriminate between two or more faces.

Popular Areas of Research Utilizing Labvanced’s Precision Timing:

• Alertness
• Cognitive decline
• Perception
• Performance measures
• Feature recognition

In theory, you can add reaction time to any experiment simply by creating a Reaction Time Variable in your editor.

See how you can add a reaction time task to your study with this video: