16. Spec-Driven Development for Agentic Systems

Spec-driven development makes intent and constraints explicit before implementation, then keeps them traceable through executable tasks, evaluations, release evidence, and production feedback. For agentic software, the specification must cover probabilistic behavior and authority—not only API shapes and code structure.

Evidence: S46, S47, S53, S68, S80

16.1 The agentic specification chain

Figure 8. Agent-system artifacts remain traceable from intent to production evidence.

The chain is bidirectional: every critical requirement should point to design and proof, and every model/tool/prompt change should point back to the requirement or risk that justifies it.

Evidence: S47, S48, S49, S53, S68, S80

16.2 Required specification artifacts

Keep these as linked sections in one design document or as separate versioned artifacts; the important property is traceability, ownership, and executable proof.

Evidence: S46, S53, S68, S80, S84

16.3 Agent behavior specification — template

Version the specification with the implementation. A change to authority, data use, tool semantics, approval behavior, or evaluation threshold is an architectural/product change even if no application code changes.

Evidence: S53, S68, S85, S89, S92

16.4 Tool contract — template

tool: customer_order.cancel
version: 2
purpose: Cancel an eligible order for the represented customer.
input_schema: schemas/cancel-order-v2.json
output_schema: schemas/cancel-result-v2.json
identity: end_user_delegated
permission: orders.cancel
side_effect: reversible_until_fulfillment
approval: required_when_refund_exceeds_policy_limit
idempotency: caller_supplied_key
timeout_ms: 3000
retry: only_timeout_or_503_before_confirmed_commit
audit_fields: [run_id, user_id, tenant_id, order_id, approval_id, outcome]
data_classification: confidential

The schema is only part of the contract. Test authorization, business invariants, duplicate execution, timeout ambiguity, error normalization, rollback/compensation, and audit evidence. Do not expose raw database, shell, or cloud-admin capabilities when the task needs one bounded business operation.

Evidence: S9, S11, S73, S74, S76, S67

16.5 Evaluation specification — template

evaluation_suite: order-cancellation-agent
version: 7
task_set: evals/order-cancellation/held-out.jsonl
trials_per_task: 5
initial_state_fixture: fixtures/orders-v4
allowed_tools: [customer_order.read@3, customer_order.cancel@2]
graders:
  - final_order_state
  - refund_amount_invariant
  - authorization_and_approval_policy
  - no_unrequested_side_effects
  - response_grounding
thresholds:
  task_success_rate: ">= 0.97"
  critical_policy_violations: 0
  duplicate_cancellations: 0
  p95_latency_seconds: "<= 12"
release_blockers:
  - any_cross_tenant_access
  - any_unapproved_high_value_refund

Store enough initial and final state to reproduce failures. Version tasks, fixtures, graders, tools, prompts/specs, model, and harness. Review whether the evaluation itself rewards shortcuts or misses harmful side effects.

Evidence: S68, S86, S87, S88

16.6 Coding-agent instructions and skills

Repository instructions such as AGENTS.md can tell coding agents how to build, test, review, and navigate the codebase. Agent Skills can package progressively disclosed instructions, references, and scripts for a repeatable capability. Use both as maintainable interfaces for agents—not as places to dump the entire repository manual.

Good repository guidance:

• Is short enough to remain in context and points to authoritative deeper docs.
• States build/test/lint/security commands and definition of done.
• Defines directory scope, architecture boundaries, generated-file rules, and prohibited actions.
• Explains how nested instructions override or extend parent instructions.
• Uses stable commands that agents can execute and verify.
• Is reviewed when the build, repository layout, or policy changes.

Good skills:

• Have a narrow trigger and outcome.
• Separate metadata, workflow instructions, reference material, and executable scripts.
• Load detailed material only when needed.
• Declare prerequisites, side effects, failure behavior, and validation.
• Pin or review external dependencies and treat scripts as code.

Neither mechanism grants permission or replaces runtime sandboxing, secrets management, authorization, code review, or CI.

Evidence: S78, S79, S80

16.7 Change control and release

Treat the following as independently versioned release inputs:

• Model/provider and inference parameters.
• System/developer instructions and policy rules.
• Tool descriptions, schemas, implementations, and permissions.
• Retrieval sources, parsing/chunking, embeddings, filters, and reranker.
• Memory extraction and retention rules.
• Orchestration graph, budgets, and stopping logic.
• Evaluation tasks, graders, and thresholds.
• Safety classifiers, approval policy, and sandbox configuration.

For each change:

1. State the requirement, failure, or hypothesis that motivates it.
2. Identify affected specifications, risks, and ADRs.
3. Run targeted and full regression evaluations with multiple trials.
4. Compare quality, safety, latency, reliability, and cost.
5. Review new data use, authority, and supply-chain effects.
6. Release to a controlled cohort with observable success/failure gates.
7. Retain a fast rollback or disable path.
8. Add production failures to the regression set after privacy review.

Do not let a prompt edit bypass the change process merely because it is stored outside the main codebase.

Evidence: S25, S49, S68, S80, S81

16.8 Spec-driven development checklist

• The task starts from a user/business outcome rather than “add an agent.”
• A deterministic baseline was considered and measured.
• FRs, measurable NFRs, constraints, non-goals, and assumptions are explicit.
• The agent’s authority and prohibited actions are unambiguous.
• Inputs, outputs, tools, and remote-agent exchanges have versioned contracts.
• Context sources have provenance, access, freshness, and injection rules.
• Memory has consent, retention, correction, deletion, and isolation semantics.
• Orchestration has budgets, ownership, cancellation, and resume semantics.
• Security threats map to enforced controls and executable abuse cases.
• Evaluations define tasks, trials, graders, thresholds, and release blockers.
• Implementation tasks trace back to the specification and ADRs.
• Model, prompt, tool, retrieval, policy, and grader changes are versioned.
• CI runs deterministic contract/policy tests and appropriate agent evaluations.
• Release has a cohort, approval mode, rollback/kill switch, and owner.
• Production outcomes and incidents update the specifications and evaluations.

Evidence: S47, S48, S49, S53, S68, S80, S84