Spidey SenseGitHub

A sixth sense for what you missed.

What it is
Smart glasses that tap your wrist about things you haven't looked at.
How it works
The glasses follow your eyes. Claude spots what matters. A small Dot buzzes only if you missed it.
Why it helps
Our eyes are often on a phone or lost in thought. One tap at the right moment, and you see the cable in time.
See it workRead the code
The glasses' view: gaze on the phone, a box drawn around a cardboard box ahead, and the buzz banner 'Something in my way, 1.7 s'
What the glasses saw. My eyes were on my phone with a box a few steps ahead, so the Dot buzzed 1.7 s after the frame. Had I looked at the box, it would have stayed quiet.

Three more from our test walks

Real takes from the hackathon floor, as the glasses saw them.

The glasses' view while walking: a box drawn around a power cord on the floor

A cable on the floor

Walking while reading my phone. It flagged the cord before I reached it.

Amber buzz

The glasses' view: a friend close by, flagged as someone about to attack

A fist from the side

Danger gets a stronger buzz, whether or not you saw it coming.

Red buzz, 1.8 s after the frame

Hold to tell SpideyThe drafted trigger to confirm

"Warn me if someone waves at me"

Hold the button, say what to watch for, and confirm the trigger it writes.

The app tells you why

Every buzz is kept with the picture the glasses saw and a sentence on what they noticed.

Home with recent buzzes
Recent buzzes
Event detail with the picture the glasses saw
What the glasses saw
Choose when to warn you
When to warn you
Heart rate from Oura
Heart rate from Oura

Details

What it's made of
  • Pupil Labs Neon eye-tracking glasses, streaming through their companion phone to a Mac.
  • Claude Haiku checks each frame against your triggers. Your real gaze decides whether you noticed.
  • DataFeel Dot on the wrist, over Bluetooth, with a colour per trigger.
  • An iPhone app with Spidey's voice (ElevenLabs) and spoken triggers. Optional Oura heart rate.
How fast, how accurate
Glasses video to the Mac~90 ms
Something in view to a buzz, in real takes1–2 s
Dot buzz over Bluetooth117 ms
Obstacles caught in 56 hand-labelled frames96%
Fist frames caught after rewording the trigger14 / 14
What doesn't work yet
  • The glasses' microphone gives no audio, so hearing is built but switched off.
  • Oura readings are only as fresh as the ring's last sync.
  • The Mac does the thinking, so it rides along in a backpack.