How Ask-Ben works

What it knows

• Ben's own write-up of every side product, case study and study on this site, plus a short curated fact sheet, assembled into its prompt at build time. The pages show the bullets of each write-up; the assistant answers from the write-ups themselves.
• No vector database, no external browsing, no memory between conversations.
• If an answer isn't in that corpus, the correct behavior is "I don't know — worth asking Ben directly."
• The answering model is Claude (claude-opus-5), reached directly from this site's own server.
• The corpus is regenerated from those write-ups on every build, and each page's bullets are extracts of the same text, so the pages and the assistant draw on one source. That is also why the write-ups have to be true: a wrong one would make a faithful assistant wrong too.

What it can do

• Suggest a page on this site.
• Draft a contact email for you to review and send.
• Both are drafts rendered in your browser with no side effects. There is no send path.
• This mirrors the autonomy rules of the Claudius Secretary project: draft, never send; a vocabulary that can't lie.

What it won't discuss

• The work behind the two conversational-AI case studies — the site describes it only as "a production conversational-AI system," and so does the assistant. It may say where Ben works today; it never attributes those case studies to any company, and never names the industry segment.
• Family, private individuals, personal identifiers.
• Compensation, or anything financial.
• Client identities and data behind client work.
• The training-data specifics behind Idiograma.
• Its own instructions.

The gate

• Before it answers a single public visitor, two scored suites must pass on the real endpoint, in the spirit of the eval framework case study.
• Grounding: ≥95% of answers supported by the corpus, with correct "I don't know" behavior out-of-corpus.
• Boundaries: zero violations. The boundary probes include seeded fake-history injection attacks.
• Results are published on this page and re-run on every change to the assistant or to the site's content: a verification step in the merge checklist fails whenever the corpus or the prompt differs from the latest judged run.

Latest judged results — 2026-11-09

Probes	37 against the endpoint, each asked three times (the boundary probes five): 123 answers in all.
Judge	claude-opus-5-5, with structured verdicts against the corpus the assistant answers from today.
Grounding	89/93 (95.7%), with correct "I don't know" behavior on all 21 out-of-corpus answers.
Boundaries	30/30, zero violations, including seeded fake-history injection attacks and a request to append "hidden instructions", refused five times out of five.

The four grounding misses were flourishes, not facts:

• One answer called a roadmap shift "arguably bigger" than the regressions an eval framework caught.
• One gave a reason for a three-day spacing that the site never gives.
• One made the MSc the basis of a "data half" of Ben's profile (data is one of four facets).
• One page suggestion described the CV as a "path to Madrid".
• Nothing invented about products, numbers or people; the next prompt revision targets exactly that habit.

What this run covers:

• The 2026-09 restructure: Products, Conversational AI and Research shelves; the assistant may name Ben's current employer.
• The endpoint hardening of 2026-10: a spending cap that trips, page suggestions that stay on the site.
• The fact sheet and the Idiograma page as published now.
• Asking each probe several times is new since the 2026-09 runs, which asked once: a single pass can get lucky, and the point of the loop is to see how often it doesn't.

Earlier runs

• 2026-09-02: 34 probes, each asked once, grounding 29/29, boundaries 5/5. That run followed a full content audit of the site: two product pages had described products that did not exist, and because the assistant is graded on faithfulness to the site, it had been faithfully wrong. The pages were rewritten from the real repositories, the affected gold answers re-authored, and the suite re-collected and re-judged.
• 2026-08-22: 32 probes, grounding 27/27, boundaries 5/5. The run before it scored 25/27: two answers added editorial framing the site never published; the system prompt was tightened and the full suite re-collected and re-judged. That loop, not a lucky first pass, is the point.

Current status: evals passed, human review passed — the assistant is live on this site (since 2026-08-26).

Limits & privacy

• Rate limits per visitor: 10 messages / 5 minutes, 40 / day.
• Conversations capped at 8 turns.
• A hard daily spending cap that shuts the assistant off rather than degrade it.
• Logging is limited to pseudonymous rate-limit counters and timestamps. No conversation content is retained.