Case Study · 2024 · HMI . . . . . . . Designed by Lori Cai

Smart Brook

Multimodal HMI
Autonomous vehicles
A passenger experience built around the human–AI trust loop.
Type · Multimodal Product Design Platform · HMI Experience Role · UX Designer & Researcher Team · Lori, Shreya, Yuan, Will
VoiceTouchVisual24 / 7 Agent
01 · The Project

A passenger experience that shows its work.

Smart Brook is a multimodal passenger experience for autonomous vehicles that centers on trust, transparency, and perceived control. Rather than hiding complexity, the system reveals how the vehicle thinks and acts in real time — letting passengers understand, personalize, and engage with the journey through an AI companion.

The brief from our team was to redefine the in-vehicle journey: empower passengers to customize their travel, surface autonomous decisions in plain language, and provide multimodal support across voice, touch, and visual channels — all so that trust can be built rather than assumed.

Qualitative interviews Quant survey · 50+ responses Competitive analysis Personas User flows HMI screens Design system
02 · The Trust Paradox

The car can be safe and still feel unsafe.

Autonomy works on a metric that passengers can't see — collision avoidance, lane confidence, perception scores. So even when the vehicle is statistically safer than a human driver, the passenger's body still flinches at the unexplained slowdown, the silent reroute, the hand that won't grip a wheel.

The gap isn't a sensor problem. It's a communication problem. Trust collapses in the seconds between an AI deciding something and the human realizing why.

HMW How might we redesign the smart-passenger experience to improve sense of control and safety without reinforcing stereotypes of AI systems as opaque, paternal, or untrustworthy?

68% of survey respondents named "not knowing what the car is doing" as their top discomfort
9 / 10 interview participants wanted a real-time explanation when the car changed plan
passengers were twice as likely to relax when the car narrated routine actions in plain language
03 · Research

What we heard before we drew a single screen.

Method 01

10 qualitative interviews

A mix of frequent business travelers, casual vacationers, and daily commuters — chosen for the spread of contexts in which someone might use a self-driving ride.

Most participants didn't fear the technology. They feared not understanding what it was about to do.

Method 02

50+ survey responses

A multiple-choice survey distributed to students, professionals, and frequent ride-hailers via Google Forms. Quantitative validation for what the interviews suggested.

Respondents who described autonomous rides as "stressful" overwhelmingly cited opacity, not danger.

Method 03

POV ride video analysis

We logged in-vehicle features and passenger behaviors across competitor POV videos, mapping every action a passenger took into a comparative chart.

Existing systems narrated what the car was doing, but rarely why — leaving the passenger to infer reasoning.

"It's not the speed that scares me. It's that the car will just do something and I don't know if it saw what I saw."

— Interview participant, daily commuter

"I want to know it's thinking. Not the math — just that there's a reason."

— Interview participant, vacationer

"If I can override it once, I'll trust it the next ten times."

— Interview participant, business traveler
Process snapshot

Six phases. Trust earned step by step.

  1. User Research
    10 interviews · 50+ surveys · POV ride videos
  2. Personas
    Frequent travelers · vacationers · commuters
  3. Spec & IA
    Customizable routes · entertainment · controls
  4. Wireframe
    Onboarding · map & infotainment · key cards
  5. UI Design
    Multimodal screens · status hierarchy · cards
  6. Reveal Logic
    Real-time decision visibility · safety alerts

Each phase produced an artifact validated against the same question: does this passenger feel more in control, or less?

04 · The Insight

Trust isn't a score. It's a loop.

Across our research, one pattern kept surfacing: passengers don't decide once whether to trust the car. They re-decide on every action. Every slowdown, every reroute, every silent moment becomes a turn of the same five-step loop — and trust either accumulates or quietly leaks.

The trust loop

Each turn is a chance to calibrate — not to maximize.

01

Predict

The passenger forms a mental model: "the car should slow for that stop sign."

02

Act

The vehicle takes an action. Brakes engage. Wheel turns. Route shifts.

03

Reveal

The system announces the action and its reason — in plain language, before the passenger has to ask.

04

Verify

The passenger checks reveal against prediction. Did the car see what I saw?

05

Adjust

The passenger calibrates: relax, override, or change a preference. The next turn begins from a slightly different baseline.

Smart Brook is engineered around this loop. Every screen, every voice line, every haptic moment was designed to keep all five stages legible — so trust doesn't have to be claimed, it can simply be demonstrated.

05 · Failure Modes

Skip a stage and the loop breaks.

Three patterns in our research showed what happens when one of the five stages is missing. Each one is a place where a well-intentioned autonomous system silently erodes the trust it just spent miles earning.

Missing · Reveal

The silent slowdown

Pattern: Car acts but doesn't explain. The passenger feels a brake they didn't predict, with no reason given.

Cost: Anxiety. The passenger now narrates the trip in their head — and every silent moment after this one feels heavier.

Smart Brook fix: Decision Reveal — every non-trivial action surfaces a one-line reason within 600 ms.

Missing · Adjust

The locked passenger

Pattern: Car explains, but offers no override. The passenger understands and disagrees — and discovers there's no way to act on the disagreement.

Cost: Resentment. Transparency without agency feels patronizing, not collaborative.

Smart Brook fix: Tap-anywhere override and a route-edit affordance on every recommendation card.

Missing · Predict

The unannounced shift

Pattern: Car changes mode (manual handoff, route reroute) without giving the passenger time to form an expectation first.

Cost: Startle. The passenger reacts to the action instead of cooperating with it.

Smart Brook fix: Pre-act narration — "approaching construction at Pike St, considering reroute" — surfaced before the action lands.

06 · User Persona

Built for the cautious commuter, not the early adopter.

Persona · synthesized from 10 interviews + 50+ surveys

Maya

32 y/o · Seattle · Solo commuter

"Maya isn't a tech enthusiast — she just wants her morning ride to be quiet, predictable, and explained when something changes. She's used ride-hailing for years, and she's open to autonomous, but only if the car can tell her what it's doing without her having to ask."

Background

Long commute, often during stressful mornings. Reads in the car. Doesn't drive. Trusts technology when it earns it.

A composite persona, not a single interview subject. Built from research synthesis across business travelers, casual vacationers, and commuters.

Goals · Needs
  • A ride she can predict
  • Plain-language status
  • Override when needed
  • No anxiety around route changes
Pains
  • Opaque AI decisions
  • "Why is the car slowing?"
  • Aggressive driving feel
  • No way to talk back
07 · Insight → HMI

One loop, mapped onto every screen.

Each stage of the trust loop became a concrete UI commitment — not a value statement, a behavior the system actually performs on every relevant frame.

Loop stage
What we built
How a passenger feels it
01Predict
Pre-act narration. The car previews intent ("approaching pedestrian crossing") before the action lands.
"I saw it coming. The car's not surprising me — we're agreeing."
02Act
Action chips on the cluster — small, persistent badges that name the live behavior (Slowing · Routing · Holding) without stealing focus.
"Something just changed. I can see what."
03Reveal
Decision card with reason. One sentence, plain language, with the trigger named ("Construction at Pike St — adjusted route by 0.3 mi").
"Now I know why. I don't have to guess."
04Verify
Map mirroring — the trigger (pedestrian, hazard, construction) is rendered on the map at the moment of reveal, so the passenger can match world to system.
"The thing the car saw is the thing I see. We're looking at the same world."
05Adjust
Tap-anywhere override + persistent route-edit. Every reveal includes a way for the passenger to disagree — a pinned secondary action, never buried in a menu.
"I have a way out. I probably won't use it. That's exactly why I trust the car."
08 · The Reveal in motion

Every choice the car makes is labeled, in real time.

A · Decision "Slowing for pedestrian at 30 m"
B · Mode "Voice + visual feedback active"
C · Reroute "Construction at Pike St — adjusted"
D · Override "Tap anywhere to take manual control"

"Rather than hiding complexity, the system reveals how the vehicle thinks and acts in real time."

09 · Multimodal States

One task. Voice. Touch. Visual. All consistent.

Smart Brook · onboarding screen
Smart Brook · map and infotainment screen
Smart Brook · key card features
Smart Brook · safety notification
Smart Brook · multimodal control
Smart Brook · entertainment selection

From onboarding through map & infotainment to the key card features — every interaction is reachable through voice, touch, or visual paths, designed to behave the same regardless of how you reach for it.

10 · Three Challenges

Each one solved by stripping, not adding.

Challenge 01

HMW design an intuitive UX so passengers use the system effortlessly without feeling overwhelmed?

  • Solution 01Simple layout with clear status indicators. Visual hierarchy that resolves itself in under a second of glance.
  • Solution 02Scrollable function cards. Organized, discoverable interface that doesn't force a full-screen mental load.
Challenge 02

HMW create personalized, localized experiences while preserving the passenger's sense of control?

  • Solution 01Route-based recommendations with seamless route edit. Suggestions feel collaborative, not prescriptive.
  • Solution 02AI-powered in-car entertainment. Curates without locking the passenger into one mode.
  • Solution 03Smart infotainment control. Air, seat, travel modes — all surfaced through the same hierarchy.
Challenge 03

HMW improve safety awareness while keeping features accessible and unobtrusive on routine trips?

  • Solution 01Increase awareness of autonomous decisions. Real-time visibility into the vehicle's reasoning, not buried logs.
  • Solution 02Real-time notifications for safety. Proactive alerts that earn attention rather than demanding it.
  • Solution 0324/7 multi-format agent. Always-available companion across voice, touch, and visual channels.
11 · Calibration, not maximization

The goal isn't maximum trust. It's calibrated trust.

A common failure in AI product design is treating "more trust" as the win condition. But over-trust is a safety failure too: a passenger who never glances at the road, never overrides, never doubts is a passenger who can't catch the system when it's wrong. Smart Brook is designed to keep passengers in the middle of this spectrum — confident enough to relax, alert enough to participate.

Calibrated Smart Brook target
Under-trust

Passenger overrides constantly. Autonomy provides no value. Riding feels like babysitting a student driver.

Over-trust

Passenger disengages entirely. When the system errs, no human is in the loop to catch it. Comfort masquerades as safety.

Principle 01
Reveal, don't reassure

Stating "you're safe" doesn't earn trust. Showing what the car is reasoning about does.

Principle 02
Quiet by default, loud on change

Routine driving doesn't need narration. Mode shifts, reroutes, and hazards do — and they should be impossible to miss.

Principle 03
Override is a feature, not an emergency

If the only way to disagree with the car is in a panic, the passenger never disagrees calmly — and never builds calibrated trust.

12 · Reflection

Trust isn't earned by being safer. It's earned by being legible.

The technical capability of an autonomous vehicle is necessary but not sufficient. What converts capability into comfort is the relentless work of making the system's reasoning visible at human speed — pre-acted, narrated, mirrored on the map, and never delivered without an out.

01
Transparency only works when it's quiet

Surface the vehicle's reasoning, but never make the passenger debug it. The reveal is for reassurance, not engineering review.

02
The loop is the product

Each interaction isn't a feature — it's one turn of a continuous trust loop. Designing the loop, not the screen, is how you scale safety perception across thousands of rides.

03
Multimodal isn't redundancy. It's resilience.

Voice, touch, and visual aren't three ways to do the same task. They're three failure modes the system can recover from when one channel is unavailable to the passenger.

Next steps
  • Companion mobile app. Extend the trust loop beyond the cabin — pre-ride briefings, post-ride reasoning recaps, and remote override of pickup parameters.
  • Accessibility & inclusivity. Compliance audit and adaptation for passengers with diverse sensory and motor abilities, especially in the multimodal layer.
  • NFC, QR & voice ID. Secure, low-friction passenger identification so personalization can persist across rides without compromising privacy.
  • Field validation. Usability testing across diverse user groups under varying road, weather, and traffic conditions to stress-test the reveal logic.

See more projects.

Each one a different shape of ambiguity.

Back to all projects