oTree Eye-Tracking integration

General info

The set-up was prepared at ELFE lab in collaboration with SR Research and in particular with Sam Hutton. I would also like to acknowledge Alexander Ralphs for his invaluable assistance with all technical arrangements.

Phisical level

In the multi-device setup, eye-tracking equipment typically operates using a dedicated local network that connects several key machines.

[Host PC 1]----
...           |
[Host PC 10]----- [Switch] ------- [Central Terminal]
                    |
            --------|
            |  [Display PC 1]
            |      ...
            \__[Display PC 10]

• Host PC (small box PC): Runs the eye-tracking software and receives data from the eye-tracking camera.

• Display PC (terminal in lab): Shows experimental stimuli to participants.

• Central Terminal (CT): Connects and monitors the overall experiment infrastructure.

• Switch: Serve as local network hubs connecting all machines.

Default configuration (single mode)

In most cases, the term [host PC] (as used in the documentation) refers to a single mode configuration, where the host PC is directly connected to both the monitor and the display PC.

A typical single mode setup looks like this:

         DisplayPort (DP)         Ethernet
[ monitor ] -------- [ host PC ] -------- [ display PC ]
             usb [camera]--┙| |
    [ mouse ] __ [ keyboard ] └──── [ power ]

In this setup:

• The host PC and display PC are directly connected via an Ethernet cable.
• During boot, the first BIOS option is selected (default video output to monitor), allowing the system to be configured and monitored directly via the connected screen.
• This setup is often used for debugging or isolated single-device use.

Multi-device mode

Each host PC and camera configuration

• During boot, the second BIOS option must be selected — headless mode, where the system boots without expecting a local display.
• This setup is required for multi-terminal coordination and centralized experiment control.

In the typical lab configuration, each of the 10 stations is set up as follows:

Launch in multi devise mode

(could done with single mode, using monitor and not a headless mode as well)

SR research has a specific software for many solutions, we are using 3 of them:

• Eye Tracker Control Center (script on CT for remote work with cameras)
• Weblink (running the records from displays PC)
• Data viwer (data processing and displaying)

To run record: (part1)

To run record: (part2)

oTree and Weblink communication Using WebSockets

• Trials
  • WebLink opens the oTree link and starts the experiment in a browser by using
  • oTree takes the participant label and uses it to establish a websocket connection
  • Important: WebLink creates trials automatically by itself
  • Within each recording, if you start a new trial, you must close the previous one first
  • Any custom event with trial opening and closing should use both open and close commands

• Command syntax
  • On relevant experiment pages (like decision pages):
    • JavaScript events detect when you need to start new trial
    • Send a message to mark a new trial start (TRIALID)
    • Track participant interactions (keypresses, inputs, etc.)
    • When the participant completes a task, mark the trial end (TRIAL_RESULT)

General Flow:

[weblink] -> initial link
    ↓ IP adress via p.pabel
[oTree Server] -> create page
    ↓ Bind page of specific participant with specific weblink
[Participant's Browser]
    ↓ liveSend({ action: 'key_down', key: 'Control', ... })
[oTree Server]
    ↓ live_method → live_key_event
    ↓ Format message:
      "KEY_DOWN KEY=Control MESSAGE=help_activated ..."
[WebLinkConnection]
    ↓ TCP connection (port 50700)
[Eye-tracking System (WebLink)]

Implementation fragments examples

1. Socket Connection

self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

This creates a TCP socket connection:

• socket.AF_INET: IPv4 addressing
• socket.SOCK_STREAM: TCP protocol (reliable, connection-oriented)

We use TCP (instead of UDP) to ensure reliable delivery of messages to WebLink, which is crucial for proper trial marking.

• Socket Creation: Create a socket object with appropriate protocol
• Timeout Setting: Set connection timeout to prevent hanging if WebLink is unreachable
• Connection: Connect to the WebLink IP address and port (default 50700)
• Message Sending: Send properly formatted messages with newline termination
• Disconnection: Close the socket connection after sending messages

2. Connection Utilities

Add these functions to your __init__.py:

# Connection class for WebLink communication
class WebLinkConnection:
    """Handles connection to the WebLink for eye tracking"""
    
    def __init__(self, host, port=50700, use_tcp=True, timeout=2.0):
        self.host = host
        self.port = port
        self.use_tcp = use_tcp
        self.timeout = timeout
        self.socket = None
        
    def connect(self):
        """Establish connection to WebLink"""
        try:
            if self.use_tcp:
                self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            else:
                self.socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
                
            self.socket.settimeout(self.timeout)
            
            if self.use_tcp:
                self.socket.connect((self.host, self.port))
            
            return True
        except Exception as e:
            print(f"Connection error: {e}")
            if self.socket:
                self.socket.close()
                self.socket = None
            return False
    
    def disconnect(self):
        """Close the connection"""
        if self.socket:
            self.socket.close()
            self.socket = None
    
    def send_message(self, message):
        """Send a message to WebLink"""
        if not self.socket and not self.connect():
            raise ConnectionError("Failed to connect to WebLink")
            
        try:
            # Ensure message ends with newline
            if not message.endswith('\n'):
                message += '\n'
                
            # Convert message to bytes and send
            message_bytes = message.encode('utf-8')
            
            if self.use_tcp:
                self.socket.sendall(message_bytes)
            else:
                self.socket.sendto(message_bytes, (self.host, self.port))
                
            return True
        except Exception as e:
            print(f"Send error: {e}")
            self.disconnect()
            raise

3. Helper Functions

# Helper function to get WebLink IP from participant label
def get_weblink_host(player):
    """Get WebLink host from participant label or return None"""
    if player.participant.label and len(player.participant.label.strip()) > 0:
        if '.' in player.participant.label and not ' ' in player.participant.label:
            return player.participant.label.strip()
    return None

4. Managing Trials Between Pages

Example of closing first automatic trial earlier:

class GroupingWaitPage(WaitPage):
    title_text = "Waiting for all participants"
    body_text = "Please wait for all participants."
    
    @staticmethod
    def before_next_page(player):
        try:
            print('trial result 0')
            host = get_weblink_host(player)
            if host:
                weblink = WebLinkConnection(
                    host=host,
                    port=50700,
                    use_tcp=True,
                    timeout=2.0
                )
                # Explicitly connect before sending message
                if weblink.connect():
                    # Send a single message, not a list
                    weblink.send_message("TRIAL_RESULT 0")
                    weblink.disconnect()
                else:
                    print(f"Failed to connect to WebLink at {host}")
        except Exception as e:
            print(f"Error sending TRIAL_RESULT in before_next_page: {e}")
            pass

5. Event Handling Function

def live_eye_tracking_event(player, data):
    """Handler for eye tracking events"""
    host = get_weblink_host(player)
    if not host:
        return {player.id_in_group: {'status': 'error', 'message': 'No WebLink host found'}}
    
    messages_to_send = []
    standard_message = None
    
    # Handle different event types based on 'data' content
    if data.get('event_type') == 'custom_trial_start':
        # Always close any existing trial first
        messages_to_send.append("TRIAL_RESULT 0")
        
        # Then start a new custom trial
        trial_id = f"{player.round_number}_{data.get('condition', 'standard')}"
        messages_to_send.append(f"TRIALID {trial_id}")
        
    elif data.get('event_type') == 'custom_trial_end':
        # End the current custom trial
        result = data.get('result', '0')
        messages_to_send.append(f"TRIAL_RESULT {result}")
        
    elif data.get('event_type') == 'key_press':
        # Log a key press event
        key = data.get('key', 'unknown')
        standard_message = f"KEY_PRESS {key} TIME={data.get('timestamp', 0)}"
    
    # Send messages
    try:
        weblink = WebLinkConnection(host=host, port=50700)
        if standard_message:
            weblink.send_message(standard_message)
        
        for msg in messages_to_send:
            weblink.send_message(msg)
        
        weblink.disconnect()
        return {player.id_in_group: {'status': 'success'}}
    except Exception as e:
        return {player.id_in_group: {'status': 'error', 'message': str(e)}}

6. Page Integration

class ExperimentPage(Page):
    @staticmethod
    def live_method(player, data):
        # Determine what type of data we're receiving
        if 'event_type' in data:
            # Eye tracking related event
            return live_eye_tracking_event(player, data)
        elif 'game_action' in data:
            # Game-specific action (handle separately)
            return handle_game_action(player, data)
        else:
            # Unknown data format
            return {player.id_in_group: {'status': 'error', 'message': 'Unknown data format'}}

7. JavaScript Implementation

// Function to send an eye tracking event
function sendEyeTrackingEvent(eventType, additionalData = {}) {
    const payload = {
        event_type: eventType,
        timestamp: Date.now(),
        ...additionalData
    };
    
    liveSend(payload);
}

// Start a custom trial (will automatically close any existing trial first)
function startCustomTrial(condition) {
    sendEyeTrackingEvent('custom_trial_start', {
        condition: condition,
        player_id: js_vars.player_id
    });
}

// End a custom trial
function endCustomTrial(result) {
    sendEyeTrackingEvent('custom_trial_end', {
        result: result
    });
}

// Track interactions during the trial
document.addEventListener('keydown', function(e) {
    sendEyeTrackingEvent('key_press', {
        key: e.key
    });
});

// Detect when player becomes active and start a custom trial
function liveRecv(data) {
    // If this is a state update and player is becoming active
    if (data.type === 'state_update') {
        if (data.is_active && !window.playerWasActive) {
            window.playerWasActive = true;
            
            // Start a custom trial for this player's turn
            startCustomTrial('player_active_' + js_vars.my_position);
        }
    }
    
    // Handle other state changes...
}

JS Example Application Decision-Making Game

For a decision-making experiment l:

// When a player becomes active (their turn to make a decision)
function liveRecv(data) {
    if (data.type === 'state_update') {
        if (data.is_active && !window.playerWasActive) {
            window.playerWasActive = true;
            
            // Determine the stage (first guess or final guess)
            const stage = data.decision_stage === 1 ? "first_guess" : "final_guess";
            
            // Start a custom trial for this decision phase
            sendEyeTrackingEvent('custom_trial_start', {
                condition: `round${js_vars.round_number}_player${js_vars.my_position}_${stage}`
            });
        }
    }
}

// When player submits their decision
document.getElementById('submitButton').addEventListener('click', function() {
    // Get the guess value
    const guessValue = document.getElementById('guess-input').value;
    
    // End the custom trial with the guess result
    sendEyeTrackingEvent('custom_trial_end', {
        result: guessValue
    });
    
    // Then submit the form or call the regular submission function
    submitGuess();
});

Points to Remember

1. ** Close before opening**: When starting a new trial, first close any existing trial with TRIAL_RESULT 0

2. Handle automatic trials: WebLink automatically creates trials when pages load - be aware of these

3. Pair TRIALID with TRIAL_RESULT: Every trial start must have a corresponding end

Troubleshooting