A practical JD for the person who makes AI systems actually work in production — not a prompt hobbyist, not a research scientist. Genericized from how Engineering Square writes and vets the role. Replace the bracketed placeholders, cut what doesn't apply, and keep the skills section honest: hire for the work that determines whether an AI system survives contact with production.
The role in one paragraph
An AI Systems Engineer builds and operates LLM-powered systems end to end: defining what 'correct' means as a measurable eval, plumbing the data that feeds the model, designing the prompts and tools the model uses, and wiring the observability that tells you when it drifts. This is a software engineering role with a specialty, not a prompt-writing role. The output is a production system with a golden set, a cost/latency budget, and monitoring — not a clever demo.
What you'll do
- Turn fuzzy 'make it use AI' requests into specified tasks with acceptance criteria and a golden eval set.
- Build the data pipelines that feed models: retrieval (RAG) over governed corpora, chunking and indexing, freshness and access control.
- Design prompts, tool/function schemas, and agent control flow — and the guardrails that keep them inside the data boundary.
- Stand up evals as code: golden sets, scoring (exact-match, rubric, LLM-as-judge validated against humans), and CI that runs them on every change.
- Instrument everything: token cost, p50/p95 latency, pass rates, refusal correctness, and drift — with alerts, not dashboards nobody reads.
- Right-size the model. Test the cheaper option; ship the smallest model that clears the bar; keep the model swappable when the frontier moves.
- Own the system in production: on-call for the AI surface, incident response, and turning every surprising failure into a new golden case.
Skills that actually matter
Weighted for what breaks in production, not what's fashionable.
Must have
- Strong general software engineering — writes production code in a modern language, reads a codebase, ships tested increments. AI systems fail on plumbing far more than on model quality.
- Evals and measurement — can design a golden set, pick a scoring method, and defend a pass threshold with numbers. Treats 'which model is better' as a table to sort, not a vibe.
- Data plumbing — retrieval pipelines, embeddings and vector search, chunking strategy, and, critically, data-access boundaries and PII handling.
- Prompt and tool design — structured outputs, function/tool schemas, agent control flow; understands context windows, token budgets, and why deterministic structure beats clever phrasing.
- Observability — logs, traces, cost and latency metrics, and alerting for a non-deterministic system. Can tell you what a request cost and why it was slow.
- Judgment about failure modes — prompt injection, hallucination, silent degradation, cost blowups. Designs for the wrong answer, not just the happy path.
Nice to have
- Experience with an agent framework (LangChain / LlamaIndex / CrewAI or equivalent) and the Model Context Protocol (MCP).
- Fine-tuning or distillation experience, and the judgment to know when not to reach for it.
- Self-hosted / open-weight model deployment for in-boundary inference.
- Background in the domain the system serves — enough to know what a correct answer looks like.
Explicitly not required
- A PhD or published research. This is an engineering role, not a research role.
- '10 years of LLM experience.' The field is younger than that; anyone claiming it is rounding up.
- Memorized benchmark leaderboards. We care that you can build a golden set for our task, not recite scores.
What we don't want
- A prompt tinkerer with no software engineering foundation.
- Someone who reaches for the biggest frontier model by reflex and has never measured whether a smaller one would do.
- Anyone who ships an AI feature with no eval set, no cost budget, and no monitoring, and calls the demo 'done.'
Compensation
- Base salary band: $[LOWER] – $[UPPER] [CURRENCY] — state a real range; candidates screen out silently on 'competitive.'
- Equity / bonus: [RANGE OR N/A].
- Location / remote: [POLICY].
- Level: [IC3 / Senior / Staff — map to your ladder].
Interview loop
Mirrors how we vet: a structured screen, then evidence they can actually build, then judgment under pressure. No brain-teasers, no take-homes that cost a week of unpaid work.
| Stage | Format | What it tests |
|---|---|---|
| 1. Structured screen | 30–45 min, fixed questions, rubric-scored | Real project ownership; can they explain a system they built and why it worked? |
| 2. Live technical | 60–90 min, pairing on a real problem | Can they write and debug production code, not just talk about it? |
| 3. Eval design exercise | 45–60 min, whiteboard/doc | Given a fuzzy AI task, can they define acceptance criteria, a golden set, and a scoring method? The load-bearing round. |
| 4. Systems & failure-mode | 45 min | Cost/latency trade-offs, data boundaries, what breaks in production and how they'd catch it. |
| 5. Values / collaboration | 30 min | How they handle disagreement, review, and being wrong. |
Scoring discipline: every interviewer scores against a written rubric before seeing anyone else's scores. Structured, comparable evidence — not a vibe check in the hallway afterward.
How to use this template
- 01Cut ruthlessly. A JD listing thirty 'requirements' screens out the good candidates who are honest about not having all thirty. Keep the must-haves short and true.
- 02State the salary band. It is the single biggest filter for wasted interview cycles on both sides.
- 03Keep the eval-design round. It is the one stage that separates people who can build AI systems from people who can talk about them. If you cut one round, do not cut that one.
Hire the engineer who defines 'correct' as a number and instruments the system to prove it — not the one with the most model names on their résumé.