EXPLORATION · IN PROGRESS

Label, please: AI glasses that read ingredients aloud

Clear
The shopper picks up a box of crackers
“No peanuts or tree nuts listed.”
When the shopper picks up a box of crackers
Hear all six moments

TL;DR

In progress · Voice prototype built

Silence must never mean safe: four answers and four rules for AI glasses

AI glasses can already read text aloud, but there’s no clear rule yet for when an always-looking assistant should speak, or how it should say “I’m not sure” about food that could make someone sick. I’m designing the glasses’ built-in assistant for people with food allergies or diabetes: four kinds of answers, four rules for when to speak, and a voiced prototype of six moments in the aisle. Research in a real store comes next.

Voice and multimodal design
Voice and multimodal design
Trust design
Trust design
AI prototyping
AI prototyping
MY ROLE

Self-initiated

TEAM

Solo, with research participants starting soon

TIMELINE

4 weeks · 2026

TOOLS

OVERVIEW

The glasses can read the label.
They don’t yet know when to speak.

Google and Samsung’s first Android XR glasses arrive with no screen at all, and Meta’s glasses can already read text aloud. Meanwhile, more than 1 in 10 US adults has a food allergy and more than 38 million Americans have diabetes, and many still check every label by hand.

Designing for

People with food allergies, or parents of a child with one · People managing diabetes

THE CHALLENGE

Checks exist, but they still need your hands

Barcode apps can already check ingredients against a diet, but only with a phone in one hand and the product in the other. AI glasses free the hands, yet today they mostly answer when asked. What’s missing is a clear rule for when an always-watching assistant speaks up, and how it says “I’m not sure” about something that could make you sick. That’s the problem I’m working on. The goal is calibrated trust: people rely on the assistant when it’s sure, and double-check when it isn’t.

Barcode apps can already check ingredients against a diet, but only with a phone in one hand and the product in the other. AI glasses free the hands, yet today they mostly answer when asked. What’s missing is a clear rule for when an always-watching assistant speaks up, and how it says “I’m not sure” about something that could make you sick. That’s the problem I’m working on. The goal is calibrated trust: people rely on the assistant when it’s sure, and double-check when it isn’t. The goal is calibrated trust: people rely on the assistant when it’s sure, and double-check when it isn’t.

THE APPROACH

Working draft

Every check ends in an answer

Silence must never mean safe. Once a shopper picks something up or asks, the assistant answers in one of four ways.

1 · Clear

“No peanuts or tree nuts listed.”

Said once, then quiet.

2 · Can’t tell

“I can’t read this label.”

Plus one next step to try.

3 · Heads up

“The label says may contain tree nuts.”

You decide, by your own rule.

4 · Stop

“Stop: this lists peanuts.”

The only answer that interrupts.

Calm

More urgent →

Each step up is more insistent. Only a match on the shopper’s own avoid list earns a “Stop.”

HOW IT DECIDES

Three questions lead to every answer

The assistant asks itself the same three questions each time, in the same order. Tap an answer to hear its sound, then its line.

01

Did the shopper pick it up, or ask?

If yes, the next question
If not

It stays quiet. Products on the shelf get no comment.

02

Can it read the label?

If yes, the next question
If not
Can’t tell

“I can’t read this label. Turn it so the ingredients face you.”

Never a guess. Always one next step.

One falling note
03

Is anything from the shopper’s avoid list on it?

Listed as an ingredient
Stop

“Stop: this lists peanuts.”

The only answer that interrupts.

Three firm, low notes
Only “may contain”
Heads up

“The label says may contain tree nuts.”

The label’s exact words. The shopper decides.

Two even notes
Nothing listed
Clear

“No peanuts or tree nuts listed.”

Said once, then quiet.

One soft, high note

One exception. If a product reads differently from the last time it was bought, that change is said first.

WHEN IT SPEAKS UP

Four rules for an assistant that’s always looking

The glasses see the whole aisle, but most of it isn’t worth a word. These rules decide when the assistant talks, and when it stays quiet. They’re working rules and may change as research comes in.

01

Pick it up, or ask

Picking something up or asking a question is the signal. Products on the shelf stay quiet, so the aisle never turns into a running commentary.

02

Only Stop interrupts

Only a match on the shopper’s own avoid list cuts into music, a call, or a conversation. Every other answer waits for a pause.

03

Say it once

Clear is said once per product. Picking it up again stays quiet unless someone asks.

04

Changes come first

If a product bought before now reads differently, that’s said before anything else, because recipe changes are where trust breaks.

KEY DECISIONS SO FAR

Three decisions shaping the assistant

These are working decisions and may change as research comes in.

Silence never means safe

If it only spoke up when something was wrong, people would read silence as approval, even when it never saw the label.

Say what the label says

“This lists milk” beats “this is unsafe.” The assistant reports what it read, and never rounds “may contain” to a yes or a no.

Built in, not an add-on

It’s the glasses’ own assistant, so there’s nothing to install. It switches to stricter rules the moment food safety is at stake.

THE VOICE SCRIPT

Six moments, each a sound and one line

Every answer starts with a short sound, so the shopper knows how serious it is before the sentence ends. Then comes one plain line that repeats what the label says. The glasses have no screen, so the color and the moving wave here stand in for what the shopper hears. Four moments show the four answers. Two more show a recipe change and a direct question. Turn on sound to hear each one, or read the captions.

Try it

Hear what the glasses say, or pick any moment

WhenThe shopper picks up a box of crackers
The ruleSaid once, then quiet. Picking it up again stays silent unless someone asks.

Sound uses your device’s built-in voice, standing in for the glasses.

HOW I’LL TEST IT

Two scenarios, two kinds of stakes

If the rule holds for a rare, dangerous mistake and for frequent everyday choices, it should hold for most of the store. Testing happens in a real aisle, with a real AI model reading real labels. The goal is calibrated trust: people should rely on the assistant when it’s sure, and double-check when it isn’t.

Is this safe for Leo?

A parent checks snacks for a 7-year-old with peanut and tree nut allergies.

STAKES

High · A few new products a week

TESTS

Alerts · “May contain” · Recipe changes

Which one fits my numbers?

Someone recently diagnosed with type 2 diabetes compares carbs and added sugar.

STAKES

Lower · Many small choices every trip

TESTS

Comparing by voice · Serving sizes · Tone

The prototype

Coming next: a phone camera and earbuds standing in for the glasses, with a real AI model reading real labels. It will appear here once it’s built.

WHAT SUCCESS LOOKS LIKE

Signs of calibrated trust

Calibrated trust means people trust the assistant as much as it deserves, no more and no less. In testing, I’ll look for these signs.

01

Silence isn’t read as safe

People can tell “Clear” apart from “didn’t check,” and ask when they aren’t sure.

02

Doubt leads to a second look

After “Heads up” or “Can’t tell,” people turn the box over and read it themselves.

03

Faster, not careless

Decisions take less time than reading every label, with no more mistakes.

04

It knows when to be quiet

People keep it on for a whole shop, instead of switching it off after one aisle.

RISKS AND OPEN QUESTIONS

What I haven’t solved yet

These are the hard problems the assistant has to hold up against. I don’t have full answers yet, and research will shape them.

A wrong “Clear”

The one failure that is never acceptable. When the model isn’t sure, it has to say “Can’t tell,” even when that’s less convenient.

Not medical advice

The assistant reports what the label says. It never gives dosing advice or tells someone a food is medically safe for them.

The camera sees other people

In a busy store, bystanders need protecting too: a clear cue when it’s looking, no saved images by default, and no processing of faces.

BEYOND GLASSES

A rule for anything that watches in the background

This started with food labels, but the rule is wider. Any product that senses in the background, from a health wearable to a car, mostly speaks up when something is wrong. People then read its silence as an all-clear, even when it had nothing to go on. Designing what “nothing to report” means is part of designing the alert.

WHAT’S NEXT

Follow along

The speak-up rules, the voice script, and a scripted audio prototype are drafted. Research starts with store ride-alongs with people managing food allergies and diabetes. Next comes a working version with a real AI model reading real labels, tested in a real aisle. When the work is done, a full case study with test results will replace these notes.

Want to talk about it? I’d love to hear from you at lynngee@gmail.com.

More work

Let’s work together

© 2026 by Lynn Gee