Webhook API
Usage: The main use case for the Labvanced External Data Storage Webhook API is to transfer participant data in “real time” to an external / remote server, instead of hosting the recorded data on the Labvanced servers.
PLEASE NOTE: Contrary to many API use cases, what is described here is NOT how to call the Labvanced API / endpoints (such a functionality does not exist for now), but rather how you must configure/implement your backend, such that our platform will be able to automatically call the endpoints you have implemented to sent the participants data directly to your backend/database instead of ours.
Availability: This functionality is only available to Labvanced Lab license holders.
1. Setting up the Webhook API in Labvanced
To use the Webhook API, one first has to activate the “external data storage” option in “Study Settings” of the respective Labvanced study under the “Experiment Features” section.
1.1 Parameters:
- IP Address: Enter your static IP address where your external server is running / publicly available.
- Port: The port which should be used to send the data
- URL Path: The URL path which should be appended after the port. This is particularly useful to “namespace” your projects / incoming data, such that data from different experiments are sent to a different URL path. The URL path is optional, but we strongly recommend using it.
2. Create the Server / Script for Receiving the Data from the Webhook API
Based on your settings in step 1, you have to implement a server which is available at the specified IP address, port, and url path. A simplified version / example of such a server is available on our Github page at here.
2.1 Data format: Data is sent in JSON format for all routes but the “/file_upload” route. The “/file_upload” route (used for uploading binary files such as recorded videos or audio) is encoded as “multipart/form-data” so please make sure you parse the incoming data correctly as JSON or multipart/form-data / binary data.
2.2. CORS Settings: When implementing your backend you have to implement / allow Cross Origin Resource Sharing (CORS). Particularly, you have to set the following headers: “Access-Control-Allow-Origin” (include https://www.labvanced.com/), “Access-Control-Allow-Methods” (include POST and OPTIONS), “Access-Control-Allow-Headers” (include Content-Type)
2.3 Request Types: All request types are POST requests as they are used to send data out. At this point, a return value is only required for the “/file_upload” route. All other routes do not need a special return value.
2.4 Types of requests / routes: Each time a participant takes part in a study, in total 8 different types of POST requests will be sent to the (external /your) server. The “/file_upload” route will only be used if your study includes an action to upload some recorded binary data (e.g. audio or video). Each type of POST request has a different route (url path), which will be appended to the specified URL path in step 1. Also each type of request has a different structure/payload, which should be accounted for when parsing and storing the data. See details in section 3.
3. Webhook API Endpoint Definitions / Routes:
In the following, you will see a detailed explanation of each route / endpoint that is used/called when a participant takes part in the experiment. Ideally we suggest that you implement all of these endpoints (besides 3.1), and make sure all the data is appropriately parsed and stored in your database.
Which endpoints are used, which server is called when?
- Endpoint 3.1 “/startExpPlayer” is only sent to the Labvanced server to load the experiment definition/JSON file and associated stimuli from our server.
- Endpoints 3.2, 3.3, 3.4, 3.7, and 3.8 are sent both to the Labvanced server (needed for group balancing and general experiment flow) and the external/your server.
- Endpoints 3.5 and 3.6 are only sent to the external/your server. Please note all trial/participant data is sent via route 3.6, which therefore is the most important endpoint to implement.
- Endpoint 3.9 is only sent to the external/your server and called every time an ”upload action” is executed to upload binary (audio or video) data.
3.1 Loading of the experiment
- Route: /startExpPlayer
- Type: POST
- Called when: The participant visits a certain experiment URL on labvanced.com Main function on Lavanced server: Initializes the loading of the experiment.
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| isTestrun | Whether the data is recorded or note | Boolean | NO |
| subject_code | Identifier of the subject via URL param | String | NO |
| token | A unique identifier for participants in longitudinal studies | Token identifier | NO |
| askSubjData | Whether to show an initial survey, used to put subjects into different groups | Boolean | YES |
NOTE: Does not need to be implemented on external servers, as this only initializes the loading of the experiment from the Labvanced server to the client computer
3.2 Group and session selection
- Route: /startFirstPlayerSession
- Type: POST
- Called when: Route 3.1 “ /startExpPlayer” returns successfully from the Labvanced server.
- Main function on Lavanced server: Used to assign group and session numbers to a subject (server side balancing)
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| subject_code | Identifier of the subject via URL param | String | NO |
| token | A unique identifier for participants in longitudinal studies | Token identifier | NO |
| survey_data | JSON of the pre-study questionnaire, fields described below. | JSON | NO |
| survey_data.selectedGender | Selected gender of the participant | String | NO |
| survey_data.selectedAge | Selected age of the participant | Integer | NO |
| survey_data.selectedCountry | Selected country/location of the participant | String | NO |
| survey_data.selectedLanguage | Selected (first) language of the participant | String | NO |
| isTestrun | Whether data is recorded or not | Boolean | YES |
| runOnlyGroupNr | Only used to test run a certain group number without data recordings | Integer / False | NO |
| runOnlySessionNr | Only used to test run a certain session number without data recordings | Integer / False | NO |
| groupNr | Group number for this subject | Integer | YES |
| sessionNr | Session number in a longitudinal study | Integer | YES |
| group_name | The name of the group for this subject | String | YES |
| session_name | The name of the session for this subject | String | YES |
| experiment_name | The experiment name (not the publication name) | String | YES |
3.3 Start of the experiment
- Route: /setPlayerSessionStartedTime
- Type: POST
- Called when: The participant presses “Start Experiment”
- Function on Lavanced Server: Saves the start time, session and group name of the experiment and starts the experiment.
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| start_time | Start time of the experiment | UNIX Timestamp | YES |
| sessionNr | Session number in a longitudinal study | Integer | YES |
| groupNr | Group number for this subject | Integer | YES |
| token | A unique identifier for participants in longitudinal studies | Token identifier | NO |
3.4 Add metadata information
- Route: /addMetaInfo
- Type: POST
- Called when: The participant presses “Start Experiment”
- Function on Lavanced Server: Saves the meta information on the server.
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| var_data | The JSON holding the meta information | JSON | YES |
| var_data.browserSpec | The browser which is used by the subject | String | NO |
| var_data.versionSpec | The browser version which is used by the subject | String | NO |
| var_data.systemSpec | The device type / OS which is used by the subject | String | NO |
| var_data.agentSpec | The complete user agent string | String | NO |
| var_data.fullscreen | Indicates if the study is always in fullscreen | Boolean | NO |
| var_data.timeDelayMean | The JavaScript callback precision mean offset in milliseconds | Float | NO |
| var_data.timeDelayStd | The JavaScript callback precision standard deviation in milliseconds | Float | NO |
| var_data.crowdsourcingCode | The crowdsourcing / completion code for subjects | String | NO |
| var_data.crowdsourcinSubjId | The crowdsourcing / worker ID of the subject | String | NO |
| var_data.subjCounterGlobal | A global subject counter for number of subject in the study | Integer | NO |
| var_data.subjCounterPer Group | A subject counter per group in the study | Array of Integers | NO |
| var_data.roleId | The unique role ID for multi-user studies of the subject | Integer | NO |
| var_data.multiUserGroupId | The global unique multi-user group ID for multi-user studies | Uuid | NO |
| var_data.displayedLanguage | The selected display language for multi-language studies | String | NO |
| var_data.pixelDensityPerMM | The pixel density of the screen | Float | NO |
| var_data.screenHeight | The screens’ height in pixels | Integer | NO |
| var_data.screenWidth | The screens’ width in pixels | Integer | NO |
| var_data.windowHeight | The window /viewport height in pixels | Integer | NO |
| var_data.windowWidth | The window/viewport width in pixels | Integer | NO |
3.5 Record information about a task
- Route: /recordStartTask
- Type: POST
- Called when: A new tasks starts in the experiment flow
- Function on Lavanced Server: Not called when external API requests are activated
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| blockNr | Current block number (increasing counter) | Integer | YES |
| blockId | Unique ID of the current block | Uuid | YES |
| blockName | Name of the block | String | YES |
| taskNr | Current task number (increasing counter) | Integer | YES |
| taskId | ID of the current task as defined in the editor (same between subjects) | Uuid | YES |
| recTaskId | Server-generated ID for the current recording task (different between subjects) | Integer | YES |
| taskName | Name of the current task | String | YES |
| start_time | Start time of the task | UNIX Timestamp | YES |
3.6 Record trial data
- Route: /recordTrial
- Type: POST
- Called when: A trial is finished OR a custom record action is executed
- Function on Lavanced Server: Not called when external API requests are activated.
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| recTaskId | Server-generated ID for the current recording task (different between subjects) | Integer | YES |
| taskId | ID of the current task as defined in the editor (same between subjects) | Uuid | YES |
| trailNr | Trial number | Integer | YES |
| recData | The main data / all user-created variables per trial are stored here | JSON | YES |
NOTE: The exact data which is recorded per trial depends on the specific experiment and task. An overview for each study is available under the "Variables" tab of the specific experiment (see screenshot). Also please note that the external API variable names will be used as a key in the JSON structure. So please make sure that variable names are unique. Otherwise you would overwrite the data. The Labvanced system usually enforces the user to use unique variable names.
3.7 Finish the experiment with success
- Route: /finishExpSession
- Type: POST
- Called when: The experiment is finished successfully
- Function on Lavanced Server: Finishes the study and flags the data set as completed (important for balancing).
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| end_time | Time when the experiment finished | UNIX Timestamp | YES |
| var_data | Same as the var data in the “/addMetaInfo” route (updated info) | JSON | YES |
| nextStartTime | Next start time (for longitudinal studies only) | UNIX Timestamp | NO |
| nextEndTime | Next end time (for longitudinal studies only) | UNIX Timestamp | NO |
| reminderTime | Time until next start time (for longitudinal studies only) | Time String | NO |
| selectedEmail | Email to send participation reminder (for longitudinal studies only) | Email Address | NO |
| emailReminder | When to send the reminder (for longitudinal studies only) | String | NO |
3.8 Finish the experiment with failure
- Route: /errExpSession
- Type: POST
- Called when: The experiment is aborted with an error
- Function on Lavanced Server: Aborts the study and flags the data set as incomplete
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expId | Unique identifier for the study | Integer | YES |
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| err_msg | The error message | String | YES |
3.9 Upload binary (audio or video) data
- Route: /file_upload
- Type: POST
- Called when: An upload action is executed in the event system associated with some audio or video recording object. Function on Lavanced Server: Not called when external API requests are activated.
- NOTE: The server must send return values for this route (explained below). If this is not done the “onUploadComplete” trigger is not executed and the experiment cannot propagate correctly. Also note that the data in this route is encoded as “multipart/form-data” as it includes both binary and non-binary data.
- Payload: JSON with the following fields:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| expSessionNr | Unique identifier for the recording session for this participant (counting upwards) | String | YES |
| newFileName | The suggested file name to store the file. The file name includes the associated variable name, and the current, block, task and trial number. | String | YES |
| myFile | The actual file / binary data | Binary | YES |
Required Response:
| Field Name | Description | Data Type | Required |
|---|---|---|---|
| file_guid | Global Unique Identifier (guid) for the recorded file. This has to be generated server side and will be saved in the variable associated with the file such that later it is easy to understand which file came from which recording object. | Uuid | YES |
| file_name | The resulting filename the server used to save the file. This can be (but does not have to be) the same file name suggested by the client in the request. This will also be stored in the associated variable. | String | YES |