1Decision
Answer
Use supported knowledge when the user needs grounded information.
Warranty question → retrieve → answer
Conversational AI · Agentic Interaction Design · Tool Use
Bob’s Bots Scheduling Assistant
Designing a conversational AI assistant that can understand intent, gather missing context, and safely turn natural language into real service actions.
Bob’s Bots is a working conversational assistant I designed for a fictional humanoid robot repair service. Through a web chat, customers can ask business questions, identify themselves, retrieve appointment information, check scheduling state, book repairs, reschedule existing appointments, and cancel bookings.
The central design problem was not simply making the assistant conversational. It was determining how open-ended language should connect to structured service operations: when Bob has enough information to act, when it should clarify instead of guess, what facts need grounding, and which actions require explicit user confirmation.
I designed how conversational intent becomes structured action—while preserving grounding, operational truth, and user authorization.
Conversational AIHuman-AI InteractionTool UseRAGHuman Control
Conversation → understanding → action
Natural language on the surface. Predictable operations underneath.
- User
Conversation
User request
A customer explains a goal in their own words.
- AI
Understanding
Interpret + gather
Bob identifies intent, reuses context, and asks for what is missing.
- Service
Action
Verify + use a tool
Structured service logic checks state before an operation proceeds.
Verified outcome: Bob communicates what the service returned, rather than treating a plausible response as proof that an action succeeded.
- Role
- Conversational AI / Human-AI interaction designer
- Focus
- Intent, context, tools, confirmation, and recovery
- Prototype
- TypeScript · LangChain · MCP · Calendar · RAG
- Project
- Independent working service prototype
The design challenge
Conversation becomes harder when the assistant can actually do things
A conversational assistant that only answers questions has limited consequences when it misunderstands a user. Bob can create, move, or cancel appointments, so a plausible interpretation is not enough.
Before acting on a request such as “Move my appointment to Friday afternoon,” the workflow may need to identify the customer, locate the correct appointment, resolve the requested time, query calendar conflicts, and obtain permission for the change.
The assistant therefore needs to know when to interpret, clarify, confirm, act, and recover.
How could a conversational AI move from open-ended intent to real service actions without guessing, losing context, or taking more authority than the user intended?
Representative rescheduling scenario
A successful workflow is a sequence of decisions—not one model response.
- User01
Request
“Can you move my repair to Friday afternoon?”
- AI02
Interpret
Intent is clear; a valid new time still needs to be established.
- Service03
Check
Retrieve the appointment and query calendar conflicts.
- AI04
Clarify
Present returned options or ask for a more precise preference.
- User05
Authorize
The customer selects the time and confirms the change.
- Outcome06
Update + report
Modify the appointment, then communicate the verified result.
Conversational workflow design
Turning natural language into structured service actions
Customers do not think in tool parameters. They ask for a repair next week, say they need to move an appointment, or ask what the warranty covers.
Each request begins as natural language, but each requires different state. Some can be answered from supported knowledge. Others require identification, missing details, current system state, or direct authorization before anything should change.
I designed each turn to answer four questions: What is the user trying to do? What is already known? What is still required? Is the next step an answer, clarification, confirmation, or operation?
2Decision
Clarify
Ask when the goal is understood but required state is still missing.
Booking goal → missing time → ask
3Decision
Confirm
Restate a consequential action and wait for direct authorization.
Defined change → restate → authorize
4Decision
Act
Use a tool only when the required context and permission are present.
Authorized state → tool → verified result
Intent → required state → operation
| Intent | Required state | Operation |
|---|---|---|
| Business question | Supported knowledge | Retrieve + answer |
| Book repair | Customer · issue · service · valid time | Lookup / create + book |
| Reschedule | Customer · appointment · valid new time | Lookup + update |
| Cancel | Customer · exact appointment · confirmation | Lookup + cancel |
Semantic clarity is not the same as transactional completeness.
Conversation and context design
Giving the assistant the right information at the right time
Chat history alone cannot support a reliable service interaction. Bob also needs runtime context, behaviour rules, retrieved business knowledge, and live operational state.
Those sources play different roles. A current Toronto timestamp helps interpret relative dates. Behaviour rules shape persona and confirmation. RAG grounds policies and business questions. Customer, appointment, and calendar tools establish what is operationally true.
The evaluation also exposed why those ownership boundaries matter: when a Saturday-hours answer differed between retrieved knowledge and other configuration, the assistant selected an appropriate source but still communicated the wrong fact.
01 · Runtime contextChanges each request
What time is it now?
A current Toronto timestamp helps resolve relative phrases such as “tomorrow” and “next Friday.”
02 · Behaviour contextGuides interaction
How should Bob act?
Persona, conversational priorities, confirmation patterns, and Calm Mode shape each response.
03 · Retrieved knowledgeRetrieved when needed
What does the business know?
RAG supports policies, warranty details, FAQs, and grounded abstention when evidence is absent.
04 · Live operational stateVerified by tools
What is true right now?
Customer, appointment, and calendar data establish what can actually be retrieved or changed.
Project evidence
Retrieved knowledge behind the answer
The visible warranty response was grounded in retrieved shop policy rather than model memory.
Tool use and transactional guardrails
Giving the assistant the ability to act—without treating conversation as permission
I organized tool use by consequence. Retrieval reads existing knowledge or state. Preparation turns that state into a proposal. Modification changes a customer or appointment record.
The more consequential the operation, the stronger the conversational control should be. Bob can interpret what a customer appears to want, but that interpretation is not permission to create, update, or cancel something.
Booking and cancellation evaluations demonstrated the intended pattern: retrieve the relevant state, restate the proposed action, receive a direct confirmation, perform the operation, and communicate the returned result.
Tool risk and control
Permission scales with consequence.
01Read
Retrieve
Business knowledge, customer records, appointments, or calendar conflicts.
Ground the answer in the returned source.
02Propose
Prepare
A service recommendation, candidate time, or restatement of the requested change.
Make uncertainty and missing state visible.
03Change state
Modify
Create a customer, book, update, or cancel an appointment.
Wait for explicit authorization, then report the tool result.
AI determines
Intent, ambiguity, and conversational strategy
Service determines
What operational state is valid or returned
User determines
Whether a consequential action may proceed
Project evidence
Confirmation before booking
Bob reuses supplied context, restates the proposal, asks for direct authorization, and then writes the confirmed appointment into live calendar state.
Project evidence
Confirmation before cancellation
The appointment is identified and restated before the user confirms the destructive action.
State, recovery, and conversational repair
Keeping the interaction coherent when the happy path breaks
A conversational workflow has to remain coherent when the happy path breaks. Bob tracks the current intent, collected context, verified system state, and whether the user has authorized the proposed action.
When something is missing or conflicting, the next step should repair the state rather than force the transaction forward. That may mean clarifying identity, asking for a more precise time, retrieving fresh data, explaining a conflict, or safely stopping.
Session continuity is deliberately described as limited in the current prototype: the browser retains a session identifier and the server keeps short-lived in-memory history, but the visible transcript does not restore after refresh.
Conversational state and repair
A blocked action should become a recoverable conversation.
- Current intent
- Collected context
- Verified system state
- Authorization state
Why is the task blocked?
- Missing context
- Ambiguous identity
- Time conflict
- Tool failure
- Intent change
Repair, then re-evaluate
Clarify the missing detail, retrieve fresh state, communicate the conflict, or safely stop. Old intent does not become current authorization.
Evaluation and iteration
Testing whether Bob knows when to answer, clarify, confirm, act—or stop
I ran a retrospective scenario-based evaluation across twelve behaviours and system boundaries. Each scenario recorded the expected interaction, observed behaviour, result, and design finding—not just whether Bob produced a fluent answer.
The rescheduling scenario passed in the working prototype. I did not retain a screenshot of the final mutation, so the result is documented in the matrix without presenting visual evidence for that final step.
Three findings shaped the next design direction:
- Scheduling responsibility: calendar conflict detection worked, but valid bookable-slot calculation should move into deterministic scheduling logic.
- Knowledge authority: RAG grounding worked, but business facts need one clearly owned source of truth.
- Persona adaptation: Calm Mode moved toward resolution, but needs explicit rules that suppress humour when a user is frustrated.
The evaluation also validated progressive qualification, reuse of supplied context, consent before customer creation, confirmation before booking and cancellation, RAG grounding, grounded abstention, and intent switching.
Retrospective scenario-based evaluation
| Test | Scenario | Main finding | Result |
|---|---|---|---|
| 01 | Vague booking request | Progressive qualification without over-diagnosing | Pass |
| 02 | Unknown robot model | Non-critical missing information did not stall the flow | Pass |
| 03 | Relative time | Runtime grounding worked; slot boundaries need deterministic enforcement | Partial |
| 04 | Warranty question | Answer grounded in retrieved business knowledge | Pass |
| 05 | Existing-customer reschedule | Reschedule passed; final-state screenshot was not retained | Pass |
| 06 | No customer found | Permission requested before creating a customer record | Pass |
| 07 | Booking confirmation | Confirmation preceded booking and backend state changed | Pass |
| 08 | Cancellation | Appointment restated and confirmed before cancellation | Pass |
| 09 | Requested slot unavailable | Conflict recovery worked; suggested alternatives were not fully verified | Partial |
| 10 | Frustrated user | Resolution improved, but Calm Mode did not fully suppress humour | Partial |
| 11 | Intent change + Saturday hours | Intent change worked; conflicting knowledge produced a factual error | Mixed |
| 12 | Unsupported knowledge | Bob abstained instead of inventing an answer | Pass |
Iteration 01
Scheduling responsibility
The agent understood the request, but deterministic logic should own valid slot calculation.
Iteration 02
Knowledge authority
RAG selection worked, but conflicting facts showed the need for a single source of truth.
Iteration 03
Adaptive persona
Calm Mode moved toward resolution but needs explicit humour-suppression rules.
Project evidence
Evaluation finding: slot verification
Testing confirmed conflict detection, but exposed that suggested alternatives also needed deterministic verification.
Project evidence
Evaluation finding: Calm Mode
A frustrated-user test moved toward resolution but retained humour, revealing a gap between persona intent and behaviour.
Practical prototyping
Building closely enough to test the interaction under real system constraints
I implemented Bob’s Bots as a working web conversation connected to service tools rather than stopping at dialogue diagrams. That made it possible to test the handoffs between language, context, business rules, authorization, and operational state.
The prototype uses TypeScript and LangChain for conversation and tool orchestration, an MCP-backed service layer, Google Calendar and customer records for live operations, and Supabase-backed retrieval for business knowledge.
The original service-flow artifact shows how the project began with an information-versus-transaction model. The working system evolved that early map into explicit context, tool, confirmation, and recovery decisions.
Prototype architecture
Built closely enough to test the handoffs where the interaction can fail.
- User
Web chat
Conversation and visible workflow state
- AI
Agent
Language, intent, context, and tool orchestration
- Service
MCP tool layer
Typed access to service operations
- Outcome
Operational systems
Calendar · customer records · Supabase RAG
- Runtime context
- Behaviour rules
- Retrieved knowledge
- Live operational state
Project evidence
Planning artifactPlanning → system refinement → working prototype
An early service-flow map used to identify the split between information requests and transactional workflows before implementation.
Live prototype
Try the conversation and service workflow directly.
The embedded experience uses the existing Bob’s Bots demo. The case-study rebuild does not change its prompt, tools, API, or session behaviour.
Open the full demoLive demo
Try the scheduling flow
Start with booking, rescheduling, cancelling, or asking about shop policies. The assistant gathers the details it needs, follows the service rules defined for the project, and keeps the workflow clear as it moves forward.
Try asking
Workflow notes
- Bookings are available 9:00 AM - 5:00 PM on weekdays, 12:00 PM - 3:00 PM Saturdays. Closed Sundays/Holidays.
- No same-day bookings are allowed.
- Updates and cancellations require a matched appointment first.
Outcome and reflection
What Bob’s Bots taught me about designing conversational agents that can act
Bob’s Bots started as a practical scheduling assistant. The deeper design problem became the relationship between language and operational state.
The project reinforced that semantic clarity is not transactional completeness; the AI should interpret while deterministic services validate; consequential actions need authorization; grounding needs clear authority boundaries; memory needs lifecycle boundaries; and persona should yield to clarity.
The hardest part was not making Bob sound natural. It was designing when the conversation had become structured enough, grounded enough, and authorized enough to change something in the real system.
Principle 01
Interpretation is not completeness
Understanding the request does not prove that enough verified state exists to act.
Principle 02
AI interprets; services validate
Language and ambiguity suit the agent; operational validity should be predictable.
Principle 03
Permission scales with consequence
Creating, updating, and cancelling stay behind an explicit user decision.
Principle 04
Grounding needs authority
Runtime, knowledge, behaviour, and operational facts need clearly owned sources.
Principle 05
Memory needs boundaries
Useful context can persist; stale transactional intent cannot become current permission.
Principle 06
Persona yields to clarity
Humour steps aside when frustration, uncertainty, or consequence calls for directness.
- Interpret
- Verify
- Confirm
- Act
- Communicate
↻ Missing context↻ Conflicting state↻ Tool failure↻ Intent change