WIP: Expanding jsPsych Pavlovia plugin: 'save' command for asynchronous saving, checkpoints, interaction data and more
Thank you @tpronk for your work addressing user's feedback and expanding the Pavlovia jsPsych plugin -I've definitely found your work useful (e.g. this thread). So useful that I decided to attempt to build on it further, hoping it would benefit other users as well.
New Features
The current MR attempts to add a few features:
- allow users to save data during the experiment without having to close and reopen the pavlovia connection or disrupt the experiment. This is especially important as many users have complained of data loss in the past (see here, here, or here)
- to create custom filenames with checkpoints in the names so that when data are saved during the experiment, it is clear what stage the participant reached
- allow users to save interaction data in addition to the main jsPsych dataset
Changes
To that effect, I've made some changes to the pavlovia plugin and the index file. I will consider those changes work in progress until you review them as you seem much more knowledgeable when it comes to JS than me (I am more familiar with jsPsych and have some experience running experiments).
The reason there are many changes to my MR is because 1) I've added local jsPsych files from version 6.3. These files come from my own Simulation method update on jsPsych 6.3. (see here for detailed explanation) which in summary goes through experiments automatically and generates random responses. The reason for adding this is to test the limits of the pavlovia plguin - e.g. can it handle multiple requests sent in unusually quick succession (as you would never have response with a reaction time of sub-100ms).
The changes in the pavlovia plugin can be summarised as follows:
- 3 new parameters: 1) checkpoint - allows you to add a string in the middle of your saved datasets to identify at what stage the participant's data were saved; 2) save_interaction_data - true/false, if true, it will save the interaction data alongside the main jsPsych dataset; 3) display_msg_while_saving_data - a string, allowing users to create a custom message while the data are being saved (currently only work in the finish case)
- Adding save case in the trial method that will asynchronously call the _save function and log the response once received. If interaction data need to be saved, then this will happen after the main data are saved as multiple simultaneous queries to Pavlovia cause errors (due to using git, I think -- at least that's what the error message was telling me)
- Update to finish case as well as the finish function to save interaction data and use the message from the trial's parameters
- Update to the file naming - I wholehearted agree that this is extremely personal choice but usually when data are sorted, I like to see the data first ordered by participant and then by the time of save so I can see how the participant progress (if there are multiple saves per participant that is)
- Some additional information provided to the response from the server when logging stuff in the console (based on the trial's new parameters)
How to use
Here's how a typical 'save' command would look like if you would want to save data during the experiment:
var pavlovia_save_checkpoint_1 = {
type: "pavlovia",
command: "save",
participantId: "custom-subject-id",
checkpoint: 'finished test procedure 1',
save_interaction_data: true
}
The command
is telling pavlovia to simply save the data up to this point of the experiment. It will save both the main jsPsych dataset as well as the interaction one (as this is requested by the save_itneraction_data
parameter, being set to true
. This trial can be pushed at any point in the timeline. You can enter any participant id or checkpoint string. The resultant files would have the following names (timestamps and experiment names will obviously vary):
custom-subject-id_2021-07-07_19h14.53.1_MAIN_checkpoint-finished test procedure 1_MYEXPERIMENT.csv
custom-subject-id_2021-07-07_19h14.53.1_INTERACTION_checkpoint-finished test procedure 1_MYEXPERIMENT.csv
The template for the file naming is:
{participantId}_{timestamp}_{either 'MAIN' or 'INTERACTION' depending on which datasets is saved}_checkpoint-{name of checkpoint}_{experiment name}.csv
The useful thing here is that this can be used as part of a timeline with timeline variables, where the timeline variables provide values for the checkpoints (e.g. if there are multiple different stages at which you want to reject the participant and save their data).
Notably, at the end, when you call the 'finish' command, the plugin will ensure that all requests are processed before closing the connection.
Next steps
@tpronk I would genuinely appreciate your help pulling this together as it seems to me that there will be people who can take advantage of it - as long as it works reliably.
One thing that I would also want to look at in the future perhaps is if it is possible to add additional handling such that if a specific request ails, it is re-tried a couple of times (say, limited to 3 attempts) before officially being rejected -- sometimes if many participants are taking part at the same time this might cause many data commits to Pavlovia and it might start getting fussy and reject some of them (which I believe is the root issue of why some people in the past have experienced data losses).
P.S. apologies for the hundreds of commits - the only way I could think of testing the plugin was online and I am not that proficient managing stuff with git :(