Evelyn (Shiyi) Peng

Design Engineer

Reflex

UnityC#AndroidEye trackingMotion sensorsFFTFluid simulation

Much of what we feel arrives before we have decided anything. Fear travels a shortcut that reaches the amygdala ahead of the visual cortex, so the body has already begun responding while the mind is still catching up, and the loop that follows — arousal, hormones, heart rate, the return of all that as sensation — runs largely beneath our notice. I wanted to build a place that sat inside a loop of that shape: an environment that reads what your body is doing and answers, so that the state you arrive in becomes the weather you find.

Gaze

Gaze is tracked with a Tobii eye tracker. Where you look disturbs the surface directly, the way a dropped stone would, so attention leaves an immediate physical trace. The gaze points are also held in a rolling buffer and examined for behaviour rather than position — whether you are scanning restlessly or holding still — and sustained attention on one place changes the light and the weather around you.

Attention: the surface risen into sharp gold peaks
Attention, the state sustained gaze brings on

Motion

Motion comes from the accelerometer and gyroscope of a phone running a companion app, built in Unity alongside the environment itself. Rather than mapping movement straight onto the scene, I recorded it into a 256-sample buffer and ran a Fourier transform over it, which turns a stretch of movement into a frequency spectrum. The peaks in that spectrum, and how much they vary, describe the character of the motion — quick and agitated, or slow and even — and it is that character, rather than any single gesture, that the environment responds to. Reading the spectrum rather than the gesture is the point: a body doesn't register a single movement either, but the rhythm and intensity of many over time.

The companion app while recording. Above, the gyroscope rotation rate and the low-pass filtered acceleration as they arrive; below, the frequency spectrum the Fourier transform returns for that stretch of motion

The environment

The landscape is a fluid simulation written in C#: a mesh whose vertices each carry their own velocity and displacement, with waves that propagate outward, damping and surface tension pulling everything back toward rest, and a slow Perlin drift so the water is never quite still. Fog scrolls above it and the light shifts with the weather. Nothing in the scene is a menu or a control; the only way to affect it is to look and to move.

Placid: a near-flat surface under heavy cloud
Placid, the resting state, and the enum the others are drawn from

Phases

The environment holds five states — placid, awaken, rhythmic, nervous, and attention — each a set of values for wave frequency, speed and height, and for the surface tension and damping that decide how a disturbance settles. Motion moves it between the first four; sustained gaze brings on attention. No input maps to a single ripple. The readings decide which state the world is in, and the world behaves accordingly.

The five phases laid out across motion and gaze, with the wave and disturbance parameters that define them
The five phases, and the parameters each one sets
Awaken: a single bright disturbance spreading across the surface
Rhythmic: regular waves running through the surface
Nervous: broken, irregular water
The three states motion moves between, escalating as the readings do

Future steps

The movements worth reading are the ones nobody chooses — subtle and involuntary, carrying state without conscious control. Detecting patterns in them rather than levels would let the atmosphere correspond to an inner state instead of approximating it. Further out the landscape forms from the input rather than being disturbed by it: terrain as the sedimentation of what a body has been feeling.