We're looking for an AI Engineer to own Charlie, the personal banker at the heart of Quartz. Charlie is live with UK members today, answering questions about their pensions, ISAs, savings and investments from data we aggregate across their accounts. You'll take responsibility for how Charlie reasons, what it is allowed to say, and how we know it is getting better. This is a production role, not a research one: you'll be judged on answer quality, safety and cost in front of real customers, in a regulated category where a confident wrong answer is worse than no answer.

Responsibilities

• Own and evolve the Charlie runtime end to end: agent roles and handoffs, prompt and tool architecture, model routing across providers, conversation memory, and the MCP integrations that give Charlie typed access to data in our Kotlin backend
• Design and maintain the guardrails that keep Charlie accurate, compliant and on-brand, including what it must not do in a business that is not yet authorised to give regulated financial advice
• Build and run our evaluation pipeline: define quality and relevance metrics, curate eval sets from real conversations, and gate releases on them so regressions never reach members
• Drive reliability and cost: instrument every conversation, hunt down failure modes such as loops, duplicated actions and tool misuse, and keep inference spend proportional to the value delivered
• Evaluate and integrate external capabilities (web search, market data, model providers) and make clear build-versus-buy recommendations
• Work with product and the wider engineering team to turn wealth guidance logic into behaviour Charlie can execute safely, and feed what you learn from conversations back into the roadmap
• Set the standards for how we build with LLMs at Quartz: testing, observability, prompt versioning and release discipline

Requirements

• 5+ years of software engineering experience, including meaningful ownership of a customer-facing LLM system in production
• Strong Python and FastAPI and strong relational data modelling in Postgres with SQL
• Hands-on experience with agentic or tool-calling systems: function calling, retrieval, structured outputs, multi-step workflows
• Practical experience with evaluation of non-deterministic systems: you have built eval sets, chosen metrics and defended a release decision with them
• Solid grasp of guardrails and safety patterns for LLM applications, and the judgement to know when a model should decline
• Startup or scale-up experience; you are comfortable owning a system alone and making it robust
• Quick to learn, ambitious and driven by results
• Fluency in English