Manual phone orders→Autonomous agent loop
The agent handles the full workflow end-to-end: receive an incoming call, converse with the customer, look up the real menu, confirm the order, and write it straight to the kitchen system.
01AgentEats
An AI phone agent that takes restaurant orders through natural conversation, built in 3 hours at an ElevenLabs hackathon.
Team
Matt Weston, Walter Lim, Jackson Lawrence, William Seagar
Period
2026 (hackathon)
Format
AI phone-ordering agent
Disciplines
Product DesignVoice AIFull-stackElevenLabs Hackathon
What if a restaurant never missed a phone order again?
Restaurants lose thousands every year to missed calls. When they do pick up, staff juggle the phone while managing the floor. And if they route orders through delivery apps, they hand over 20–30% in commission fees.
AgentEats is an AI phone agent that answers restaurant calls 24/7, takes orders through a natural voice conversation, and sends the final order directly to the kitchen or POS — with zero staff involvement and zero commission.
We built a working prototype in 3 hours at the ElevenLabs × BlackBird Foundry hackathon, with a team of four. The question was not whether AI could take a phone order. It was whether a hackathon-scoped prototype could handle a real conversation well enough to feel like talking to staff.
24/7
availability
0%
commission
3 hrs
to build
4
team members
Voice AI made the hard parts cheap enough to prototype in one session
Building a phone agent used to mean months of telephony integration, speech recognition pipelines, and dialog management. The ElevenLabs agent framework compressed those into API calls, letting us focus on the ordering experience instead of infrastructure.
Chat-first ordering→Voice-first natural conversation
Instead of another chatbot or web-ordering widget, the interface is a phone call. It feels like talking to a trained staff member, not navigating a menu tree.
02Expensive integrations→3-hour hackathon build
The ElevenLabs agent framework made the hard parts — low-latency bidirectional voice, multi-turn dialog, structured output — cheap enough to prototype in a single session.
03Commission-heavy platforms→Zero commission, zero staff
Delivery apps take 20–30% per order. A phone agent the restaurant owns captures every call, operates 24/7, and keeps the full margin.
04Challenges we solved at hackathon speed
Three hours is not a lot. Every decision had to earn its place, and every integration had to work on the first try or get cut.
Keeping conversation grounded in the real menu
The agent retrieves the restaurant's live menu via API so it never hallucinates items or prices. Every suggestion it makes is backed by real data, not a training-set guess.
Multi-turn order accuracy
Orders have modifiers, corrections, and confirmations. The agent handles the full dialog — add an item, change a size, remove a side — across a natural conversation without losing context.
End-to-end in 3 hours
We scoped aggressively: start from a Next.js + Railway starter, integrate the ElevenLabs agent framework, and ship a complete call-to-kitchen demo. Everything else got cut.
How it works
A complete agentic loop from incoming call to kitchen ticket, with no human in the middle.
Customer calls
A customer dials the restaurant's number. The agent picks up instantly via ElevenLabs bidirectional voice.
Natural conversation
The agent engages in multi-turn dialog — greeting, taking the order, handling modifiers, confirming details.
Live menu lookup
Items and prices are retrieved from the restaurant's real menu via API. No hallucinated dishes.
Order confirmation
The agent reads back the complete order, handles corrections, and collects customer details.
Kitchen submission
The confirmed order is written directly to the kitchen system or POS via Prisma + Postgres.
Interaction logged
Every call is recorded and logged for the restaurant owner to review in the admin dashboard.
Tech stack
Chosen for speed-to-demo at a hackathon. Every piece had to integrate in minutes, not days.
ElevenLabs Conversational AI
Agent framework with bidirectional voice, multi-turn reasoning, and structured output over a single low-latency connection.
Next.js + React
Frontend with a live phone-call simulation UI for the browser demo.
Clerk
Authentication for the restaurant admin dashboard.
Prisma + Postgres
Data models for menus, orders, and restaurant configuration.
Railway
Hosting, deployments, and database infrastructure.
What I took away
We ended up with a working phone agent, but the useful lesson was how little infrastructure voice AI needs now. The hard parts, bidirectional speech, multi-turn reasoning, structured output are API calls. The design work is the conversation itself.
Voice agents can replace manual workflows
A 3-hour prototype proved the full ordering loop works autonomously. Call to kitchen, no human in between.
Hackathon speed clarifies scope
Aggressive time constraints forced us to cut to the core: call, converse, order. Nothing else survived.
Bidirectional AI voice is a new design surface
Designing for spoken conversation is different from chat UI. Tone, pacing, and error recovery matter more than layout.
ElevenLabs × BlackBird Foundry Hackathon