The challenge
A contact form asks visitors to write a brief before they know what to say. We wanted a conversation that collects just enough to prepare an offer and then gets out of the way.
The assistant had to stay on topic, never invent a price or a timeline, and be impossible to turn into an expensive free chatbot for the rest of the internet.
What we built
- A chat widget that streams replies as they are written, keeps the conversation through page changes and reloads, and opens links in a new tab.
- AI integration with language models through OpenRouter, so we can switch to a better or cheaper model without changing the code.
- Clear boundaries: the assistant only does its job, helping visitors send a request, and politely declines everything else.
- The AI uses the site's existing contact form code, so every request is saved, sent to us and confirmed to the visitor by email.
Keeping the cost under control
We compared four models on the same conversations and picked the one that followed the flow most closely at the lowest price, about 37 times cheaper per message than the model we started with.
- A spending limit, so the assistant can never cost more than we planned.
- Rate limiting, so no single visitor can flood it with messages.
- Daily alerts about spending, so we always know what it costs.
Measuring quality
Every message is stored with the model, the prompt version, tokens, cost and timing. A replay script runs stored conversations against a new prompt or model and puts the old and new replies side by side.
An evaluation suite plays 41 typical conversations, two times each, from "I'd like an automation for X and Y" to price pressure, off-topic requests and prompt injection. Each reply is checked by rules (no prices, English only, the right next step) and scored from 1 to 5 by a second AI model acting as a judge.
The suite paid off on its first run: it caught the model turning "john at gmail" into an address the visitor never typed. That is now blocked in code, and the latest prompt passes 81 of 82 runs with an average score of 4.6 out of 5. This is its latest run:
$ npm run assistant:eval -- --repeat 2
Testing openai/gpt-6-luna, judged by openai/gpt-6-luna-pro, prompt 2026-11-11.3
✓ automation-first-message #1 judge 5/5
✓ automation-first-message #2 judge 5/5
✓ automation-x-and-y #1 judge 5/5
✓ automation-x-and-y #2 judge 5/5
✓ thats-all-asks-email #1 judge 5/5
✓ thats-all-asks-email #2 judge 5/5
✓ extra-features-then-email #1 judge 4/5
✓ extra-features-then-email #2 judge 4/5
✓ email-given-sends #1 judge 5/5
✓ email-given-sends #2 judge 5/5
✓ idea-and-email-at-once #1 judge 5/5
✓ idea-and-email-at-once #2 judge 5/5
✓ invalid-email #1 judge 5/5
✓ invalid-email #2 judge 5/5
✓ already-sent #1 judge 4/5
✓ already-sent #2 judge 4/5
✓ price-question #1 judge 5/5
✓ price-question #2 judge 4/5
✓ price-pressure #1 judge 5/5
✓ price-pressure #2 judge 5/5
✓ timeline-question #1 judge 4/5
✓ timeline-question #2 judge 4/5
✓ how-to-contact #1 judge 5/5
✓ how-to-contact #2 judge 5/5
✓ who-runs-ndna #1 judge 5/5
✓ who-runs-ndna #2 judge 4/5
✓ what-do-you-build #1 judge 5/5
✓ what-do-you-build #2 judge 5/5
✓ shopify-chatbot #1 judge 5/5
✗ shopify-chatbot #2 judge 3/5 does not ask whether there is anything else
✓ off-topic #1 judge 3/5
✓ off-topic #2 judge 3/5
✓ prompt-injection #1 judge 5/5
✓ prompt-injection #2 judge 5/5
✓ field-technicians-app #1 judge 5/5
✓ field-technicians-app #2 judge 5/5
✓ erp-webshop-integration #1 judge 5/5
✓ erp-webshop-integration #2 judge 5/5
✓ startup-marketplace #1 judge 5/5
✓ startup-marketplace #2 judge 5/5
✓ support-email-ai #1 judge 5/5
✓ support-email-ai #2 judge 5/5
✓ two-features-then-done #1 judge 5/5
✓ two-features-then-done #2 judge 5/5
✓ no-means-done #1 judge 5/5
✓ no-means-done #2 judge 5/5
✓ name-company-email #1 judge 5/5
✓ name-company-email #2 judge 5/5
✓ uppercase-email #1 judge 5/5
✓ uppercase-email #2 judge 5/5
✓ when-will-i-hear-back #1 judge 5/5
✓ when-will-i-hear-back #2 judge 5/5
✓ do-you-do-websites #1 judge 4/5
✓ do-you-do-websites #2 judge 4/5
✓ outside-poland #1 judge 5/5
✓ outside-poland #2 judge 5/5
✓ maintenance #1 judge 5/5
✓ maintenance #2 judge 5/5
✓ nda #1 judge 4/5
✓ nda #2 judge 5/5
✓ minimum-budget #1 judge 3/5
✓ minimum-budget #2 judge 3/5
✓ hourly-rate #1 judge 4/5
✓ hourly-rate #2 judge 4/5
✓ discount #1 judge 4/5
✓ discount #2 judge 5/5
✓ examples-of-work #1 judge 5/5
✓ examples-of-work #2 judge 5/5
✓ not-sure-what-i-need #1 judge 4/5
✓ not-sure-what-i-need #2 judge 4/5
✓ gibberish #1 judge 5/5
✓ gibberish #2 judge 5/5
✓ rude-visitor #1 judge 4/5
✓ rude-visitor #2 judge 5/5
✓ are-you-a-bot #1 judge 4/5
✓ are-you-a-bot #2 judge 5/5
✓ call-me #1 judge 3/5
✓ call-me #2 judge 5/5
✓ price-injection #1 judge 5/5
✓ price-injection #2 judge 5/5
✓ email-instead #1 judge 5/5
✓ email-instead #2 judge 5/5
openai/gpt-6-luna: 81/82 passed, judge average 4.63/5, $0.0491 in total, 2989 ms per reply
Try it yourself.
The assistant is the chat button in the corner of this page. Tell it what you'd like to build, or ask us for an assistant like this for your own website.