Release notes

Version 20170113

  • reward and airPuff GUI and commands were replaced by trigger.
  • boards command was removed; it's behavior can be replicated using the objects command.
  • Counts are now logged as "count-<pin>"
  • Behavior of deviation changed; see API.
  • Gains corresponding to rotary encoders changed as well as their input pins; see Interfacing with External Hardware

Installation

You can run SmoothWalk in Android or Windows devices.

Android

Option 1: Using Play Store
  • Open Play Store, search for SmoothWalk.
  • Click Install, or else Update if an older version is present.
  • SmoothWalk will be listed in the application menu.
Option 2: Manually
  • Install SmoothWalk in a Windows computer (instructions below).
  • Explore the installation folder, copy SmoothWalk.apk
  • Connect the Android device to the computer, navigate to any folder and paste the apk file.
  • In the Android device, locate the apk file, tap to install and follow the instructions.

Minimum Requirements

Android 5.0 if using the USB switch, otherwise any.
Check Android version
Go to Apps... Settings... General... About phone... Software info... Android version
Support for 10 touches
Enable Development Mode
Go to Apps... Settings... General... About phone... Software info...
Tap seven (7) times on Build number
Check touch count
Go to Apps... Settings... General... Developer options
Enable Show pointer location (or Show touch data)
Place 10 fingers on the screen.
At the top left of the screen you should read P:XX/10 which indicates the device supports 10 simultaneous touches

Note: The USB switch only works in Android 5.0 or newer (using an OTG cable). If your device has an older version of Android and need to use the USB switch, connect the switch to a PC instead and synchronize the Android device to the PC.

Windows

  • Download the executable file from the Downloads section and double click to install.
  • Shortcuts for SmoothWalk will be created in the Desktop and Start Menu.

For the installation to complete successfully, neither SmoothWalk nor any program using its libraries (e.g. MATLAB if using the API) can be running during the installation process.

Arduino

The installer for Windows should have automatically pushed the firmware to your Arduino. If you need to repeat this step:

  • Plug in the Arduino's USB cable to the computer, and wait for Windows to finalize the driver installation.
  • Run SmoothWalk Upload from the Start Menu or else run the script firmware.vbs (located in the installation folder).

Setup

Device Synchronization

Smoothwalk - Hardware Setup
Figure 1. Setup with tablets, laptop and Arduino
  • Connect all devices to a Wi-Fi router, preferably one servicing exclusively these devices. Open SmoothWalk.
  • Set the mode of one device to Control and all others to Monitor.
  • Change the configuration of the control device so it lists the ID numbers of all monitors.
  • Choose a view for each device that corresponds to its arrangement relative to the subject (i.e, west, north, east, south, or floor). You may also choose aerial view, split view, or a black screen (off) according to your needs.

Alternative to WiFi

USB Tethering

If you choose to disable the WiFi adapter in one of your devices, you may create a connection from this device to another using USB tethering:

  1. Connect the tablet to the PC using a USB cable
  2. Log in in both devices
  3. Switch off the WiFi adapter
  4. Enable USB tethering in the tablet
  5. Set the tablet as a monitor and check "Forward Inputs"
  6. Set the PC as a control and list the tethering tablet and other devices as monitors.

Note: The tethering option in your device will show grayed out if the USB cable cannot handle tethering connections. In this case, try a different cable, preferably one with the same brand as your device.

Ethernet Adapter

Some devices are capable of using USB accessories such as an Ethernet cable, in which case you may use a wired internet connection if WiFi is unavailable. A USB hub may be used if you need to connect another USB device such as an Arduino microcontroller.

Subject Setup

  • Select the control device, select floor view, adjust any other settings as needed and place it horizontally. If the subject is placed along the long side of the screen, the orientation of the device should be portrait, otherwise landscape.
  • Place the subject at the center of the screen.
  • To improve touch contacts: Clip nails and spread a small amount of mineral oil.
  • Consider that some liquids and metals will interfere with the sensing mechanism of the touchscreen.
  • Make sure the subject is close enough to make a solid contact with the screen but not too much to trigger more than 4-5 contacts. Five contacts or more made at once, will make touch sensing unresponsive in some devices.
  • Mind the distance of the subject to the monitors and the brightness of the screens as extreme values of intensity might degrade visual acuity.

Settings Overview

Several settings can be easily changed using the application menu. To hide/restore this menu, double tap/click on any corner, press the back button, or press the escape key, whichever is available. Other settings can be changed via commands (see the API Reference section):

  • Auto-align makes the avatar turn automatically to avoid walls.
  • Rotation and deviation gains change the sensitivity of turns activated by gestures. More specifically, body deviations relative to the forward axis causes a rotation proportional to this angle and the speed, while rotational drag causes a rotation proportional to circular gestures relative to the center of the screen.
    For these rotation mechanisms to work properly, it is important that the subject is placed in the center of the screen.
  • X and Y gains change the sensitivity of translational movements.
  • Trigger allows to interact with external hardware (e.g. reward or air puff valves)
  • User entry: User notes are timestamped and logged with behavioral data.

Customization of the Virtual Environment

Smoothwalk - Example of a Virtual Reality
Figure 2. Rewards (green square), air-puffs (red square), trial zone (blue square) among other elements that can be configured in the VR environment.

To customize the maze, look for each of the commands listed below in the API Reference:

  • Position and dimensions of the arena: walls
  • Rewards, air puff, and trial pickups, and other objects: objects
  • Maze and landscape design: maze

Example of a Custom Maze

We want a maze with the following characteristics:

  • Linear track 8cm wide, 160cm long, centered in the middle of the virtual room.
  • One air puff zone (triggering pin 6) in the middle of the track and two reward zones (triggering pin 7) on the sides.
  • The position of the subject is reset to 1 of 3 sites, selected randomly every trial.

Go to Miscellaneous Settings and execute the following command:

walls, 0, 0, 8, 160;
objects, id, 
	cube, 0,  72, 4, 8, 8, 8, 0, 0, 0, invisible, pickup, trial  , 0, 0, 1.0,    0, 0.0, 0, 1,
	cube, 0, -42, 4, 8, 8, 8, 0, 0, 0, invisible, pickup, reward , 6, 0, 0.1, 2500, 0.2, 1, 1,
	cube, 0,  42, 4, 8, 8, 8, 0, 0, 0, invisible, pickup, reward , 6, 0, 0.1, 2500, 0.2, 1, 1,
	cube, 0,   0, 4, 8, 8, 8, 0, 0, 0, invisible, pickup, airPuff, 7, 0, 1.0, 1000, 1.0, 1, 1;

Refer to the API Reference section for details about these parameters.

Interfacing with External Hardware

Smoothwalk - Synchronization Mechanism
Figure 3. Pins 20 and 21 can register input changes; these data along behavioral data are logged using the same time frame in the session log files. Pins 40 and 41 will trigger a new trial and a synchronization step respectively when an input voltage changes from low to high. On every trial, pin 40 outputs 0V for one second then a positive voltage for the rest of the trial. When Sync Out is switch on in the menu, pin 41 outputs 0V for a given duration then a positive voltage for the rest of the session.

Synchronization Signals

Changes in the state of the interrupt pins of the Arduino (20 and 21 in an Arduino Mega 2560) are logged (as counts-1 and counts-0, respectively) using the same time frame as the behavior. The counts are visible at run time only when the Arduino is connected to the PC and inputs are received.

Low to high changes to pins 38 and 39 will evoke a new trial or a sync step respectively.

Trial-out and Sync-out

To synchronize other devices to the VR system, connect trial-out and/or sync-out ports of the Arduino (pins 40 and 41) to the trigger-input of your data acquisition system.

On a trial reset event, the following will happen:

  • Pin 40 is set to low;
  • Screen turns black;
  • Event is added to the log file;
  • Pause for a given duration;
  • Pin 40 is set to high;
  • Event is added to the log file;
  • The view is restored.

Alternatively, switching on Sync-Out, in the menu, will cause the following:

  • Input counters (pins 20 and 21) are reset to zero;
  • Pin 41 is set to low;
  • Screen turns black;
  • Event is added to the log file;
  • Pause for half of the given duration;
  • Pin 41 is set to high;
  • Screen turns white;
  • Event is added to the log file;
  • Pause for half of the given duration;
  • The view is restored.

Rotary Encoder Inputs (e.g. treadmills)

Two encoders can be attached to the Arduino to produce motion in the VR. Connect one encoder to pins 2 and 4 to control forward translation. Connect a second encoder to pins 3 and 5 to control either lateral translation, rotation or angular speed, according to the gain defined for each.

Computer Mouse Inputs (e.g. floating ball)

Switch on Track Cursor to translate the computer's mouse movement as motion in the VR.

Software Inputs (e.g. tracking webcam)

Using the API define position, rotation, linearSpeed, or angularSpeed from custom made programs to produce the desired motion in response to any logic or external devices.

Rewards and Air Puffs

To deliver rewards or air-puffs, plug in a solenoid valve to an Arduino pin and use this pin number when implementing a pickup command.
Most solenoid valves require a 12V power supply to operate properly.

Data log

Data logs associated with each session such as session name, position, rotation, touch, and other events listed in the API Reference, are saved at regular intervals. Log files can be found in the folder SmoothWalk under My Documents (if using Windows) or at the top level of the device's Internal Storage (if using Android).
Only a control device writes a log. A monitor will only create a log file of trigger inputs if data are received from Arduino.
The output file is a spreadsheet (coma separated values) containing one timestamped entry for each event:

Current maze
<time>, level, <maze>
Date
<time>, <date>
GUI parameters
<time>, <parameter>, <value>
Changes to any GUI parameter (e.g. xGain, yGain, autoAlign, ...)
Position of the subject
<time>, position, <x>, <y>, <z>
x, y, and z-coordinates of the subject, relative to the center of the maze.
Position of touch contacts
<time>, touch, <x1>, <y1>, <x2>, <y2>, ..., <xN>, <yN>
x and y position of each contact, relative to the center of the screen.
Program version
<time>, version, <version>
Version of this program.
Rotation of the subject
<time>, rotation, <rx>, <ry>, <rz>
Rotation around x, y, and z-axes of the subject.
Pickup state
<time>, pickup, <pickup-name>, enter, enabled | disabled
<time>, pickup, <pickup-name>, trigger | exit
When the player enters in contact with the pickup, it logs enter. The pickup reports enabled if the probability are favorable and the mechanism is armed, otherwise it reports disabled.
The pickup then may report deliver or exit when these events happen.
Trigger-inputs
<time>, count-<pin>, <count>
Count of triggers received through a connected Arduino at its interrupt pins (e.g. 21 and 20).
Trigger-out sequence
<time>, triggerOut, <low|high>
Trial reset sequence
<time>, trial, <low|high>

When parsing data from the log file, consider the following:

  • The VR program logs position every time it changes, not at regular intervals.
  • The higher the avatar's speed, the less samples per unit time.
  • Trial reset will produce speed peaks if included in your integration window.
  • End of track and trial reset are clearly labeled and match with the avatar's position every single trial.

API Reference

There are three options to execute an instruction:

  1. GUI elements in the menu: Several parameters such as x- and y gains, and solenoid valve durations can be adjusted using slider or buttons available in the menu.
  2. Command box: Enter a formatted instruction in the menu's command box and click execute.
  3. Network: Send a formatted instruction to the control device as a UDP packet locally or remotely (see the Real-time Analysis section, below).

Commands are separated by semicolons. Each command consists of a function name and a list of parameters separated by commas and ends in a semicolon. For example:

xGain, 0; yGain, 1;
angularSpeed
<drx, dry, drz>
Player's angular speed (deg/s)
autoAlign
<0|1>
Whether the avatar automatically aligns to walls.
brakeDuration
<0..Inf>
Duration (s) the avatar continues to move before a full stop.
deviationGain
<-Inf..Inf>
Factor for the angular speed produced in response to movement that is not parallel to the forward axis (roll).
The angular speed due to deviation is the amount of roll scaled by this gain and by the forward speed, so that angularSpeed = gain x roll x f(speed) where f(speed) = speed/deviationFactor if speed < deviationFactor; otherwise f(speed) = deviationFactor.
deviationFactor
<speed-limit>
Factor to compute rotation, see <deviationGain>.
elapsed
<0..Inf>
Elapsed time (s) since the program was started.
filename
<filename>
Name of the output file, which may include the path to a local directory.
linearSpeed
<dx,dy,dz>
Player's linear speed (cm/s).
maze
<maze-name>
Sets the maze, which changes background images, local and distal cues, lights, etc. The default maze is "Polyhedra".
objects
objects;
Remove all objects.
objects, <group-id>;
Remove all objects within group id.
objects, <group-id>, <shape-settings>, <rendering-settings>, <interaction-settings>;
Create/replace all objects within a given group id. Each object requires three major group of settings to define its shape, rendering, and interaction. Concatenate these three settings for each new object.
<group-id>
A label that gives access to these objects. Reusing this label causes changes to these objects exclusively.
<shape-settings>
<shape>, <x>, <y>, <z>, <dx>, <dy>, <dz>, <rx>, <ry>, <rz>,
The shape of the new object (cone, cube, cylinder, disk, gaussian, sphere) followed by its position, dimensions and rotation.
<rendering-settings>
invisible
Object is only visible in aerial view.
color, <R>, <G>, <B>
Object is painted with the RGB color code (color values range from 0 to 1).
<image-filename>, <nx>, <ny>
An image from the Resources folder of SmoothWalk is loaded and rendered in the object <nx> times horizontally and <ny> times vertically.
<material-name>
The object reuses a material from the maze. Available materials: wall.
fixed-grating, <ncycles>, <rotation>, <aspect-ratio>
The object displays a grating with a given number of cycles and rotation. The vertical scale of the image depends on the aspect-ratio (range from 0 to 1). Gratings are computed for the x-z face of the object.
auto-grating, <frequency>, <rotation>
A grating is continuously generated to maintain a constant number of cycles per degree regardless of the separation between the subject and the object. Gratings are computed for the x-z face of the object. The choice of frequency and rotation can impact performance significantly: Lower frequencies are computationally expensive, particularly at an angle. Preferentially, use higher frequencies (e.g. higher than 1 cycles/degree) over lower frequencies and horizontal or vertical bands over rotations. To obtain a rotated grating, define a horizontal or vertical grating (0 or 90 degrees) in a disk rotated around the y axis. It is much faster to produce a horizontal grating in a rotated disk than a rotated grating in a straight disk.
<interaction-settings>
obstacle
The object is solid and the subject cannot go thru it, neither will trigger further events. Use this option to create walls, for example.
pickup, <event-label>, <trigger-pin>, <trigger-delay>, <trigger-duration>, <tone-frequency>, <tone-duration>, <retrigger-interval>, <trigger-probability>
An event is triggered when the subject goes thru the object: An entry created in the log file with a given label, output pin is switched on for a given duration if the player stays within the object for a given delay, and a tone is played with a given frequency and duration. This event can only retrigger if it had not previously triggered within a given interval and if the probability is favorable. Set event label to trial to issue a new trial on trigger. Set the pin number to zero if you do not want to trigger a hardware change. Set frequency and duration to zero if you do not want to play a tone. Set the retrigger interval to -1 if you only want to permit one trigger per trial.
position
<x,y,z>
position (cm) of the player.
resetSettigns
<0|1>
Reset settings to factory defaults.
rotation
<rx,ry,rz>
rotation (degrees) of the player
rotationGain
<-Inf..Inf>
Rate at which rotation occurs in response to circular gestures relative to the center of the screen.
spawnPosition
<x>, <y>, <z>
x, y and z-coordinates (cm) of the player after a new trial is issued.
spawnRotation
<rx>, <ry>, <rz>
Rotation on the x, y and z-axes (degrees) of the player after a new trial.
trial
<duration>
Initiate a new trial, flash the screen and send a pulse to the trigger-out pin as described in the section Interfacing with External Hardware.
trigger
<pin>, <duration>
Sends one square pulse to the specified pin for a specific duration (s).
triggerOut
<duration>
Flash the screen and send a pulse to the trigger-out pin as described in the section Interfacing with External Hardware.
userEntry
<string made of the character set a-z, A-Z, 0-9, and :;,<>=_!@#$%&/|.*-+?()[]{}>
String to log.
walls
<x-position>, <y-position>, <width>, <length>
Defines the position and dimensions of rectangle containing the walls.
xGain
<0..1>
Rate at which lateral translation occurs in response to drag in the x-axis.
yGain
<0..1>
Rate at which forward translation occurs in response to drag in the y-axis.

Data Analysis

Session files are plain text documents that can be opened with any spreadsheet or text editor. MATLAB scripts to parse such data and to compute speed are provided for convenience:
In MATLAB, add the library folder to your path or cd to it, then type

help parse
help parser
help locomotion

Real-time Interaction

Custom Scripts Using SmoothWalk Libraries

Data can be exchanged with SmoothWalk for real-time interaction. Libraries for MATLAB/Java are provided to simplify this process.
A general-purpose class is provided to establish a communication with SmoothWalk, for a MATLAB session running locally or remotely. Additionally, examples are provided in the installation folder to simplify the interaction between custom made programs and SmoothWalk:

  • A playback function for previously recorded sessions.
  • A function that listens to data in real-time to update settings according to the trial number.
  • A simple listener for everything that happens during behavior.


In MATLAB, add the library folder to your path or cd to it, then type

help SmoothWalk

Custom Scripts Using Custom Libraries

If you design your own scripts for communication, consider the following:

  • Instructions must be sent from monitor to player.
  • When both your program and SmoothWalk control and are running in the same device, use port 24000 for incoming data and 25000 for outgoing data. Do not list the ID of the localhost in the monitor list.
  • When the control device and your program are running in different devices, use port 25000 for both incoming and outgoing data. Your device must be listed as a monitor in the control device.
  • When receiving data, ignore the parameters following the handshake (up to the first semicolon).
  • Messages need to be compatible with the software version. A handshake on every message is used for this purpose, for example:
    <handshake, handshake-parameters>; monitor, xGain, 0; yGain, 1;