What is a "Headphone Check?"
The headphone check, to our knowledge, was first coined as a term in 2017 when Kevin Woods and colleagues created the first method of determining whether or not a participant in an online study was listening through headphones. Based on this foundational study, a headphone check is simply a task that identifies if a participant is using headphones versus some form of speaker when listening to specific stimuli.
When research is conducted in a controlled lab setting, there is no need for a headphone check because the researcher chooses whether or not to place headphones on the participant. The listening environment is controlled and closely monitored.
However, in online studies, there is no way to control what the participants are doing. There is only the element of trust, the “honor code” so to say, that comes from asking participants to wear headphones and hoping that they will comply.
Why are headphone checks necessary?
Researchers hope that participants will comply with headphone requirements for many reasons. When presenting auditory stimuli, it is crucial to standardize how your participants are hearing your audio clips. This prevents confounding variables such as “interference from background noise, the poor fidelity of laptop speakers, and environmental reverberation,” (Woods et al. 2017) as well as distance from speakers and even stereo versus mono sound.
Within types of headphones, there is a wide variety. Participants could be using wired or wireless devices. Headphone styles include over-the-ear headphones and earbuds, as well as monaural or binaural setups. Some headphones are noise canceling and thoroughly occlude the ear canal, while others sit towards the outside of the canal and allow background sounds to still be heard. Despite this within-headphones variability, the category of headphones as a whole differs from loudspeakers in measurable ways.
How is this measurement accomplished?
How was the first headphone check created?
One of the key differences between listening through headphones and listening through loudspeakers is phase cancellation. Woods and colleagues used this principle to make their determinations by using three tones: two tones with the same phase and one tone that was exactly 180 degrees out of phase as compared to the others, specifically between the stereo channels.
Phase cancellation, also called destructive interference, occurs when two waves are identical in all aspects except phase, where they differ by 180 degrees. This difference causes the waves to cancel each other out, resulting in no sound when the waves are played at the same time.
When listening over loudspeakers, there is a difference in the phases of sounds than there is when using headphones. Woods and colleagues found that this difference was due to the speaker(s) position and the listener’s position. The trick is that headphones do not allow the phases of the sounds to cancel out, but loudspeakers do.
Fig. 1: Diagrams of phase cancellation from Woods et al. 2017
The task itself was simple: participants had to listen to 3 tones and decide which tone was the quietest of the group. The authors found that a low frequency tone (specifically 200 Hz) worked best for this measurement. Three tones were played: two at the same loudness level (measured in decibels), one of which was 180 degrees out of phase, and a third tone that was in phase with the first tone but 6 decibels quieter. The order of the tones was randomized between trials.
Overall, the researchers found that due to phase cancellation, participants who listened through loudspeakers often incorrectly chose the tone that was out of phase as the softest tone. When using headphones, participants could often correctly identify the tone that was 6 decibels softer than the other two tones, despite being in phase with one of the louder tones. This resulted in about 70% of true headphone users passing the screening.
Fig 2: Results of the Woods et al. 2017 headphone screening
New Techniques for Headphone Checks
In the 4 years since the Woods et al. headphone check made its debut, two more methods for assessing participants’ listening technology have emerged. Though not yet as well cited, these new methods show that research is ever growing and evolving on this topic.
Huggins Pitch
ChaitLab from University College London, 2020
In an effort to improve the selectivity of the Woods et al. screening (also called AP for anti-phase), the ChaitLab from UCL developed a test that resulted in the correct detection of 80% of headphone users as opposed to the previous selectivity of about 70%. However, they did note that combining the two methods resulted in even better results: a false positive rate of only about 7%.
The Huggins Pitch also involves a phase shift, but in a slightly different way. The three stimuli used in this task are 2 identical white noise sounds, and a third one that has a “hidden” warble tone inside it. The science behind this third sound is “a white noise stimulus to one ear, and the same white noise—but with a phase shift of 180°—to the other ear” (Milne et al 2020). The trick to this task is that the hidden warble is only detectable by wearing headphones due to the binaural (or dichotic) nature of the sound being delivered to the ears. This is a detection task which, the authors note, is a lower cognitive demand than a discrimination task such as the Woods et al. screening.
Fig. 3: Comparing anti-phase and huggins pitch stimuli, from Milne et al. 2020
Headphone and Loudspeaker Test (HALT)
Hanover University of Music, Drama and Media, 2022
The HALT, developed by Wycisk et al. in 2022, is an even more sophisticated screening: results indicate that it is a reliable and efficient way to detect what kind of device the participant is using to listen to auditory stimuli, differentiating between four playback devices: two kinds of headphones and 2 kinds of speakers.
Volume Check
Using a musical excerpt, pink noise, and looped pink noise segments as the different stimuli, participants first completed a series of volume calibration tasks. Next, the participants completed a counting task involving different numbers of “noise events.” For the task to be marked correct, participants should only hear a specific number of events: between 5 and 7 of the 9 presented events. If they heard less, their volume was too soft, and hearing more meant the volume was too loud. The task was then repeated later to ensure that the participant had not adjusted their volume in the meantime (they were instructed not to do so).
Stereo vs. Mono Check
To determine if the participant’s playback device was stereo or mono, they were instructed to count all of the pink noise segments they heard on the right side only. For mono, all events would be audible, but only a few (a random number) would be perceived in the stereo setup.
Low Frequency Limit
The HALT also checked for what the lowest pitch was that the playback device would output to help determine what device was being used. The participants head a series of pure tones and had to report how many they heard, assuming that number would correspond to their device’s capabilities.
To norm the test, researchers conducted electroacoustic analysis on their own equipment and compared the results of controls to the experimental participants.
Results of the HALT followed a normal distribution and indicated high reliability in predicting if participants had adjusted their volume, were listening through stereo or mono setups, and what type of device they were using (headphones or speakers).
Headphone Checks in Labvanced
The Woods et al. 2017 headphone check involving anti-phase stimuli is implemented in Labvanced and is available for your use. Simply visit our Sample Studies page and click import to copy it to your account!
Fig. 4: The headphone check designed by Woods et al. 2017, now available in Labvanced!
Although the Huggins Pitch headphone screening from ChaitLab was originally implemented in Gorilla, this screener is also available in Labvanced. The authors provided the full project on the ChaitLabUCL GitHub which allowed our team to create this version for your use. It is also on the Sample Studies page and is available to import!
Fig. 5: The headphone check designed by the ChaitLab from UCL, now available in Labvanced!
Last but not least, the HALT Part 1 is also available on GitHub thanks to Kilian Sander. If you would like to see this test in Labvanced as well, please send us a message via Discord or to [email protected]!
Conclusion
All of the headphone checks presented here have been validated by data, but science is always evolving and improving. Each check has its own advantages and disadvantages and is useful in its own way. We recommend trying each check and combining their methods to match the experiment that you are conducting.
Happy researching!
Note: When using any of these resources, please give credit to the original creators! Labvanced does not claim ownership of any materials created by the aforementioned researchers.
References
Milne, A. E., Bianco, R., Poole, K. C., Zhao, S., Oxenham, A. J., Billig, A. J., & Chait, M. (2021). An online headphone screening test based on dichotic pitch. Behavior research methods, 53(4), 1551–1562. https://doi.org/10.3758/s13428-020-01514-0
Woods, K., Siegel, M. H., Traer, J., & McDermott, J. H. (2017). Headphone screening to facilitate web-based auditory experiments. Attention, perception & psychophysics, 79(7), 2064–2072. https://doi.org/10.3758/s13414-017-1361-2
Wycisk, Y., Kopiez, R., Bergner, J. et al. The Headphone and Loudspeaker Test – Part I: Suggestions for controlling characteristics of playback devices in internet experiments. Behav Res (2022). https://doi.org/10.3758/s13428-022-01859-8