The AI Guide

The AI Guide is an intelligent conversational docent for Real Threads. Rather than browsing a static list of projects, visitors can talk to an AI-powered guide that understands every project on the site — its themes, its technology, and how it connects to the wider collection.

Ask it what interests you — AI-generated poetry, data visualisation, wartime history, creative coding — and it will steer you to the projects that match. Think of it as having a knowledgeable friend walk you through a gallery, rather than reading every label on the wall.

How It Works

Under the hood, the Guide is a multi-step AI agent built with the Vercel AI SDK and backed by a large language model accessed through the AI Gateway. It runs as a Next.js API route and streams responses back to a React chat interface.

The agent is tool-augmented: rather than relying on baked-in training data, it calls a set of custom tools at each step to look up live project information, search blog posts, and query the poetry corpus. This grounding ensures that every project name, date, and URL it mentions is accurate and up to date.

Authentication is handled by Supabase Auth, with server-side session validation in Next.js middleware. The chat UI itself is a client component that uses the AI SDK's useChat hook for real-time streaming.

The System Prompt

Every good guide needs a brief. The agent's system prompt establishes its persona — a digital docent who is welcoming, intellectually curious, and concise. It sets ground rules for tone, response length, and formatting, and explicitly forbids the agent from inventing project details.

The prompt also maps out the site's thematic clusters — AI & Poetry, Creative AI Imagery, Data Visualisation, History & Humanities, and more — so the Guide can orient visitors quickly and draw connections between projects. The goal is a conversation that feels like an informed recommendation, not a keyword search.

Poetry Specialism

The Guide has a special skill: deep knowledge of The Magic Porridge Poet, the site's long-running AI poetry project with over 1,200 poems published on Sanity CMS.

This is powered by a Retrieval-Augmented Generation (RAG) pipeline. Every poem is broken into semantic chunks — the poem text, the AI poet's author note, a one-line insight, and structured metadata — then embedded as vectors using OpenAI's text-embedding-3-small model and stored in a pgvector-enabled Supabase database.

When a visitor asks about a theme — consciousness, exile, the natural world — the Guide performs a semantic similarity search across the entire corpus, retrieves the most relevant poems, and then quotes directly from them. A curated poetry primer document gives the agent expert-level background on the seven thematic clusters, the project's evolving voice, and its distinctive vocabulary, so its commentary is informed and nuanced rather than generic.

The pipeline stays in sync automatically: a Sanity webhook fires whenever a poem is created, updated, or deleted, triggering re-embedding so the Guide always has the latest work at its fingertips.

Next Steps

The Guide is a living project. The next phase will introduce rich UI components rendered directly in the chat — image galleries, interactive data visualisations, and embedded project previews — so the agent can show you a project, not just describe it.

Beyond that, more project-specific primers will give the Guide specialist-level knowledge across the full collection, not just the poetry. The aim is a guide that can speak with real depth about any thread in the tapestry.