Handling Submissions

NOTE: this page assumes you are familiar with notions covered in this page.
This page describes how to keep track of your participants submissions, and how to add welcome and exit screens before and after your series of trials.

We will add two pages before the trial series: a mock consent form and a page describing the task. We will add two pages after the trial series: a page where the participants can leave some feedback comment, and a final page inviting them to click a link (either to learn more about the study, or to confirm their participation on an external platform).

Welcome screens

Consent form HTML document

When you want to display a large amount of text on the screen, one option is to use many Text elements, but this rapidly becomes a hassle. What you can do instead is create your own HTML file that you upload in the folder chunk_includes of your experiment (Ibex Farm already provides you with an example file called example_intro.html when you create a new project). Then you can use the command newHtml to incorporate the content of the file in a PennController trial.

One option to add HTML files to chunk_includes is to click on example_intro to open it in a new tab, then save it on your computer (Ctrl+S or Apple+S) under a new name (e.g. consent.html) and upload it back to your Ibex project. This way, you can click edit next to it, and replace its whole content with the code below.

consent.html

<h1>Consent form</h1>

<div><span style='font-weight: bold;'>Voluntary participation:</span> I understand that my participation in this study is voluntary.</div>

<div><span style='font-weight: bold;'>Withdrawal:</span> I can withdraw my participation at any time during the experiment.</div>

<div><span style='font-weight: bold;'>Risks and Benefits:</span> There is no risk or benefit associated with my participation in this study.</div>

Link to the file

Description HTML document

description.html

<h1>Description of the task</h1>

<div>This experiment has two steps. In the first step, you will read a question and click on one of two pictures displayed side by side below the question to provide your answer.</div>

<div>In the second step, you will read the beginning of a sentence and click on one of five radio buttons to choose its ending.</div>

<div>Now the first step of the experiment will begin.</div>

Link to the file

PennController trials

The scripts for the pre- and post-experiment trials are pretty basic: a newHtml command block followed by a newButton command block.

Make sure to refer to their labels in the shuffleSequence variable to show them before your experimental items.

Note that we now define the items variable again, because the consent form and the description screen are not generated through the template+spreasheet method, so we cannot use FeedItems for them.

PennController.Sequence( "consent" , "description" , randomize("picture") , randomize("rating") );
PennController.ResetPrefix(null);
PennController.AddHost("http://files.lab.florianschwarz.net/ibexfiles/PennController/SampleTrials/");

PennController( "consent" ,
    newHtml("consent form", "consent.html")
        .print()
    ,
    newButton("consent button", "By clicking this button I indicate my consent")
        .print()
        .wait()
);

PennController( "description" ,
    newHtml("description form", "description.html")
        .print()
    ,
    newButton("start", "Start the experiment")
        .print()
        .wait()
);

// We then add our trials
PennController.Template(
    // ...
    // See code from previous page

Feedback page

For the feedback page, your probably don’t want much more than an input box, so you can simply use newTextInput:

PennController.Sequence( "consent" , "description" , randomize("picture") , randomize("rating") , "feedback" );
PennController.ResetPrefix(null);
PennController.AddHost("http://files.lab.florianschwarz.net/ibexfiles/PennController/SampleTrials/");

// Note that 'feedback' is defined first, but what matters is the order of the labels in 'shuffleSequence' above
PennController( "feedback" ,
    newTextInput("feedback", "Type any comment you have here")
        .settings.log() // Record the participant's feedback
        .settings.lines(0) // Unlimited number of lines
        .print()
    ,
    newButton("validate", "Validate")
        .print()
        .wait()
);

PennController( "consent" ,
    newHtml("consent form", "consent.html")
        .print()
    ,
    newButton("consent button", "By clicking this button I indicate my consent")
        .print()
        .wait()
);

PennController( "description" ,
    newHtml("description form", "description.html")
        .print()
    ,
    newButton("start", "Start the experiment")
        .print()
        .wait()
);

PennController.Template(
    // ...
    // See code from previous page

Exit page

For the exit page you probably want to use a HTML document again. You have the option of showing it after the results have been sent to the server, which is quite useful when you want to give a link for participants to confirm their participation to your study. Here is how to do that:

exit.html

<h1><a href='http://validationurl/'>Click here to validate your participation</a></h1>

<div>Thank you for taking our study. Your results have now been saved, but you still need to validate your participation by clicking the link above.</div>

<div style='color: red; font-weight: bold;'>Validating your participation is a necessary step to grant you credits.</div>

Link to the file

The trick is to insert the command PennController.SendResults to send the results right before the exit page in your sequence of items:

// Here we insert the item labeled 'send' before the one labeled 'exit'
PennController.Sequence( "consent" , "description" , randomize("picture") , randomize("rating") , "feedback" , "send" , "exit" );
PennController.ResetPrefix(null);
PennController.AddHost("http://files.lab.florianschwarz.net/ibexfiles/PennController/SampleTrials/");

// Creates an item labeled 'send' that will send the results
PennController.SendResults( "send" );

// The exit page
PennController( "exit" ,
    newHtml("exit", "exit.html")
        .print()
    ,
    // A dummy timer we never start in order to stay on page forever
    newTimer("dummy", 10)
        .wait() // This will wait forever
);

PennController( "feedback" ,
    newTextInput("feedback", "Type any comment you have here")
        .settings.log() // Record the participant's feedback
        .settings.lines(0) // Unlimited number of lines
        .print()
    ,
    newButton("validate", "Validate")
        .print()
        .wait()
);

PennController( "consent" ,
    newHtml("consent form", "consent.html")
        .print()
    ,
    newButton("consent button", "By clicking this button I indicate my consent")
        .print()
        .wait()
);

PennController( "description" ,
    newHtml("description form", "description.html")
        .print()
    ,
    newButton("start", "Start the experiment")
        .print()
        .wait()
);

PennController.Template(
    // ...
    // See code from previous page

Participant’s information

You may want to be able to associate a set of responses with a unique identification code for your participants. This is particularly useful when you want to make sure they completed your study before paying them or awarding them course credit. It also proves convenient when analyzing the data.

We will consider two cases:

  1. you pass a unique ID in the experiment’s URL (a method often used when recruiting participants from the SONA systems)
  2. you ask your participants to enter a unique ID at the beginning of the experiment (a method often used when recruiting participants from Prolific or Amazon Mechanical Turk)

The URL method

You will need to provide your participants with links of the form http://spellout.net/ibexexps/YourAccount/YourExperiment/experiment.html?id=ParticipantsID or if you are using a group design and want full control over group assignment, links of the form http://spellout.net/ibexexps/YourAccount/YourExperiment/server.py?withsquare=0&id=ParticipantsID.

For each trial, you want to associate the set of responses with the actual value of ParticipantsID from the URL. You can retrieve that value by using the function PennController.GetURLParameter("id") (make sure the keywords in the URL and in the function—here, id—are exact case-sensitive matches). All you need to do then is use the .log method we saw before, like this:

PennController.Sequence( "consent" , "description" , randomize("picture") , randomize("rating") , "feedback" , "send" , "exit" );
PennController.ResetPrefix(null);
PennController.AddHost("http://files.lab.florianschwarz.net/ibexfiles/PennController/SampleTrials/");

PennController.SendResults( "send" );

PennController( "exit" , 
        newHtml("exit", "exit.html")
            .print()
        ,
        newTimer("dummy", 10)
            .wait() // This will wait forever
);

PennController( "feedback" ,
        newTextInput("feedback", "Type any comment you have here")
            .settings.log()
            .settings.lines(0)
            .print()
        ,
        newButton("validate", "Validate")
            .print()
            .wait()
)
.log( "ParticipantID", PennController.GetURLParameter("id") );

PennController( "consent" ,
        newHtml("consent form", "consent.html")
            .print()
        ,
        newButton("consent button", "By clicking this button I indicate my consent")
            .print()
            .wait()
)
.log( "ParticipantID", PennController.GetURLParameter("id") );

PennController( "description" ,
        newHtml("description form", "description.html")
            .print()
        ,
        newButton("start", "Start the experiment")
            .print()
            .wait()
)
.log( "ParticipantID", PennController.GetURLParameter("id") );


PennController.Template( PennController.defaultTable.filter("Type","rating") ,
    row => PennController( "rating" ,
        newText( "sentence" , row.Sentence )
        ,
        newScale("judgment",    "cold", "cool", "lukewarm", "warm", "hot")
            .settings.log()             // Record
            .settings.labelsPosition("top")
            .settings.before( getText("sentence") )
            .settings.size("auto")
            .print()
            .wait()
    )
        .log( "ParticipantID", PennController.GetURLParameter("id") )
        .log( "Group" , row.Group )
        .log( "Sentence" , row.Sentence )
);

PennController.Template( PennController.defaultTable.filter("Type","picture") ,
    row => PennController( "picture" ,
        defaultImage
            .settings.size(200, 200)
        ,
        newText("test sentence", row.Sentence)
            .print()
        ,
        newCanvas("patches", 500, 200)
            .settings.add(   0, 0, newImage("color1", row.Color1) )
            .settings.add( 300, 0, newImage("color2", row.Color2) )
            .print()
        ,
        newSelector("patch")
            .settings.log()             // Record
            .settings.add( getImage("color1") , getImage("color2") )
            .wait()
    )
        .log( "ParticipantID", PennController.GetURLParameter("id") )
        .log( "Group" , row.Group )
        .log( "Sentence" , row.Sentence )
);

The Input method

You will need to provide your participants with a TextInput element early in the experiment, ideally on the first screen. Here, we will insert a TextInput element right below the consent form, above the Consent button.

What is special about this case though is that we want to retrieve some piece of information from a trial and pass it to all the other trials, so they can save that piece of information in the results file along with other data points. In contrast to the URL method, from the perspective or the script evaluation, the value of the participant’s ID is not known prior to running the experiment. We will have to use the special Var element type, which lets you dynamically store values during runtime that can optionally be accessed across trials.

The script below illustrates how to use this method.

PennController.Sequence( "consent" , "description" , randomize("picture") , randomize("rating") , "feedback" , "send" , "exit" );
PennController.ResetPrefix(null);
PennController.AddHost("http://files.lab.florianschwarz.net/ibexfiles/PennController/SampleTrials/");

PennController.SendResults( "send" );

// Note that we define this first, because it defines the Var element we will refer to later
PennController( "consent" ,
        newHtml("consent form", "consent.html")
            .print()
        ,
        newTextInput("id")
            .settings.log()
            .settings.before( newText("before", "Please enter your unique participant ID") )
            .print()
        ,
        newText("warning", "Please enter your ID first")
            .settings.color("red")
            .settings.bold()
        ,
        newButton("consent button", "By clicking this button I indicate my consent")
            .print()
            .wait(  // Make sure the TextInput has been filled
                getTextInput("id")
                    .testNot.text("")
                    .failure( getText("warning").print() )
            )
        ,   // Create a Var element before going to the next screen
        newVar("ParticipantID")
            .settings.global()          // Make it globally accessible
            .set( getTextInput("id") )  // And save the text from TextInput
)                   // Now we can save the ID
.log( "ParticipantID", getVar("ParticipantID") );

PennController( "exit" ,
        newHtml("exit", "exit.html")
            .print()
        ,
        newTimer("dummy", 10)
            .wait() // This will wait forever
);

PennController( "feedback" ,
        newTextInput("feedback", "Type any comment you have here")
            .settings.log()
            .settings.lines(0)
            .print()
        ,
        newButton("validate", "Validate")
            .print()
            .wait()
)
.log( "ParticipantID", getVar("ParticipantID") );

PennController( "description" ,
        newHtml("description form", "description.html")
            .print()
        ,
        newButton("start", "Start the experiment")
            .print()
            .wait()
)
.log( "ParticipantID", getVar("ParticipantID") );

PennController.Template( PennController.defaultTable.filter("Type","rating") ,
    row => PennController( "rating" ,
        newText( "sentence" , row.Sentence )
        ,
        newScale("judgment",    "cold", "cool", "lukewarm", "warm", "hot")
            .settings.log()             // Record
            .settings.labelsPosition("top")
            .settings.before( getText("sentence") )
            .settings.size("auto")
            .print()
            .wait()
    )
        .log( "ParticipantID", getVar("ParticipantID") )
        .log( "Group" , row.Group )
        .log( "Sentence" , row.Sentence )
);

PennController.Template( PennController.defaultTable.filter("Type","picture") ,
    row => PennController( "picture" ,
        defaultImage
            .settings.size(200, 200)
        ,
        newText("test sentence", row.Sentence)
            .print()
        ,
        newCanvas("patches", 500, 200)
            .settings.add(   0, 0, newImage("color1", row.Color1) )
            .settings.add( 300, 0, newImage("color2", row.Color2) )
            .print()
        ,
        newSelector("patch")
            .settings.log()             // Record
            .settings.add( getImage("color1") , getImage("color2") )
            .wait()
    )
        .log( "ParticipantID", getVar("ParticipantID") )
        .log( "Group" , row.Group )
        .log( "Sentence" , row.Sentence )
);

Notes on group designs

If you choose to rely on Ibex’s internal counter to assign each participant a group (remember the last paragraph of the within-item manipulation section of this page) a problematic situation that often happens is that you get many participants as soon as you publish your experiment on your favorite pooling platform, but the counter does not increase until one complete the whole experiment, and then all of your (first) participants see the same condition, resulting in dramatically unbalanced observations. This is because, by default, Ibex increases its counter only when a participant has finished completing the experiment.

Ibex offers a simple method to control when, in the flow of the experiment, the counter should be updated. If you get a massive flux of participants upon publication, you are better off incrementing the counter as soon as a participant arrives on the first page of your experiment:

 // First thing: setcounter
PennController.Sequence( "setcounter" , "consent" , "description" , randomize("picture") , randomize("rating") , "feedback" , "send" , "exit");

PennController.ResetPrefix(null);
PennController.AddHost("http://files.lab.florianschwarz.net/ibexfiles/PennController/SampleTrials/");

// The counter is incremented when this is executed
PennController.SetCounter( "setcounter" );

PennController.SendResults( "send" );

// ...
// See code above


Good job, you should now be ready to set up your own experiment and collect data!

You may also want to take a look at more advanced issues:

And of course, take a look at the reference manual for a list of existing element types, commands and their descriptions.