Design effective prompts for large language models in production systems. Covers chain-of-thought prompting, few-shot learning, system prompt design, structured output, prompt testing, and the patterns that make LLM interactions reliable and repeatable.
Prompt engineering is the practice of structuring input to a language model to elicit precise, reliable, and scalable outputs. It matters when models are the primary processing layer—whether in APIs, batch pipelines, or interactive systems—and when the cost of error or latency outweighs the cost of training or fine-tuning. This page details the most effective, repeatable ways to design prompts across engineering workflows, grounded in real configuration, real data flow, and real failure modes.
When the downstream system expects a rigid structure—JSON, XML, or a specific schema—prompting must enforce consistency. The core challenge is ensuring the model produces output that parses without error, even when the input is noisy or ambiguous.
Use the output_format directive in the prompt to explicitly define the expected structure. This directive includes:
schema block in JSON or a format example block.strict flag that requires the model to validate every field against the schema.validation_rules list of constraints.For example, to generate a customer onboarding workflow, the prompt is:
{
"prompt": "Generate a customer onboarding workflow for a SaaS product based on the following description.",
"input": "The product is a project management tool with team collaboration, task tracking, and real-time reporting.",
"output_format": {
"schema": {
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"title": { "type": "string", "minLength": 3 },
"duration_days": { "type": "integer", "minimum": 1 },
"responsible_role": { "type": "string", "enum": ["Product Manager", "Engineer", "Designer", "Customer Success"] },
"deliverables": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
}
},
"required": ["title", "duration_days", "responsible_role"]
}
},
"estimated_total_days": { "type": "integer", "minimum": 1 }
},
"required": ["steps", "estimated_total_days"]
},
"strict": true,
"validation_rules": [
"All steps must have a unique title.",
"Duration days must be non-decreasing across steps.",
"At least one step must be assigned to Customer Success."
]
}
}Failure mode: the model returns estimated_total_days: 5.5 instead of an integer. The downstream service fails with:
Error: Invalid type for field estimated_total_days: expected integer, got floatThe strict flag catches this, but only if the model is explicitly told to validate. Without it, the model may return flexible JSON with duration_days: 4.2 or deliverables: ["docs", null, "setup"], which the parser rejects.
When multiple steps or roles are involved, use a template-driven prompt where each instruction is prefixed with a role tag and step marker.
The template uses ### ROLE: <name> blocks to define context, and > STEP <number> to sequence actions.
Example:
### ROLE: System Administrator
You are responsible for deploying and maintaining the production environment. Your tasks include monitoring, scaling, and troubleshooting.
> STEP 1: Analyze the provided logs and identify root causes of the recent 400ms latency spike.
> STEP 2: Propose a configuration change to reduce latency by at least 20%.
> STEP 3: Generate a deployment checklist for the proposed change.
### ROLE: Developer
You are responsible for implementing and reviewing code changes. You must ensure correctness, performance, and maintainability.
> STEP 1: Review the code changes in the pull request and list all issues.
> STEP 2: Suggest improvements to the code structure and naming.
> STEP 3: Write a summary of the changes for the release notes.
> STEP 4: Cross-check your findings with the system administrator’s analysis.This pattern ensures that the model processes each role’s responsibilities independently and integrates across roles. A common failure: the model omits the cross-check step, merging the two roles into a single narrative. The solution is to include a final_integration_step in the prompt:
"final_integration_step": {
"instruction": "Merge the findings of both roles into a single cohesive report.",
"format": "Use a two-column table: Role | Key Findings | Action Items | Dependencies",
"required": true
}Few-shot prompting uses a set of input-output pairs to teach the model the desired behavior. This pattern is essential when the output requires subtle reasoning.
Structure examples as a examples list in the prompt, each with input and output fields.
Example for a model that extracts metadata from technical documentation:
{
"prompt": "Extract metadata from the following documentation section.",
"examples": [
{
"input": "The system uses a microservices architecture. Services are deployed via Kubernetes and monitored with Prometheus. Data is stored in a PostgreSQL cluster with read replicas.",
"output": {
"architectures": ["microservices", "kubernetes", "prometheus", "postgresql"],
"deployment": "Kubernetes",
"monitoring": "Prometheus",
"storage": {
"type": "relational",
"engine": "PostgreSQL",
"replication": "read replicas"
}
}
},
{
"input": "Authentication is handled by OAuth 2.0 with JWT tokens. Sessions are stored in Redis and expire after 7 days. API endpoints are versioned using path-based versioning.",
"output": {
"auth": {
"method": "OAuth 2.0",
"token_type": "JWT",
"session_storage": "Redis",
"session_ttl": 7
},
"api_versioning": "path-based"
}
}
],
"output_format": {
"schema": {
"type": "object",
"properties": {
"architectures": { "type": "array", "items": { "type": "string" } },
"deployment": { "type": "string" },
"monitoring": { "type": "string" },
"storage": {
"type": "object",
"properties": {
"type": { "type": "string" },
"engine": { "type": "string" },
"replication": { "type": "string" }
}
},
"auth": {
"type": "object",
"properties": {
"method": { "type": "string" },
"token_type": { "type": "string" },
"session_storage": { "type": "string" },
"session_ttl": { "type": "integer" }
}
},
"api_versioning": { "type": "string" }
}
}
}
}Common failure: the model applies only the first example to all inputs, producing outputs that resemble the first case. To fix this, add a example_selection_strategy:
"example_selection_strategy": "closest_semantic_match",
"max_examples": 3,
"include_context": trueThe model now selects the most relevant examples based on semantic similarity to the input and includes the context of the selected examples in the prompt.
When the model must adhere to strict constraints—like word counts, formatting rules, or required keywords—use constraint-based prompting.
Each constraint is a rule with a type, value, and optional metadata.
Example:
{
"prompt": "Summarize the following technical article in 150 words.",
"constraints": [
{
"type": "word_count",
"value": 150,
"tolerance": 10,
"enforcement": "required"
},
{
"type": "format",
"value": "markdown",
"enforcement": "required"
},
{
"type": "keywords",
"value": ["performance", "scalability", "latency", "caching"],
"enforcement": "required",
"min_count": 3
},
{
"type": "tone",
"value": "professional",
"enforcement": "recommended"
}
]
}The model must return a Markdown-formatted summary with exactly 150 words (±10), and at least three of the keywords must appear. The enforcement field guides how strictly the model applies each rule:
required: failure if rule is not met.recommended: model should strive to meet, but can deviate.Failure mode: the model generates a summary with 148 words and only two keywords. The word_count rule is satisfied, but the keywords rule is not. The downstream service checks the output and fails with:
Validation failed: Required keywords missing.
Expected: performance, scalability, latency, caching.
Found: performance, scalability, caching.
Missing: latency.To prevent this, add a missing_keywords_strategy:
"missing_keywords_strategy": {
"insert_missing": true,
"add_contextual_sentence": true,
"add_in_conclusion": true
}The model now inserts a sentence introducing the missing keyword and adds it to the conclusion.
When the model must reason through multiple steps—e.g., debugging a system, designing a deployment strategy—use chain-of-thought (CoT) prompting.
The prompt includes a reasoning_steps block that defines the sequence of thought.
Example:
{
"prompt": "Diagnose the root cause of the application’s slow response time.",
"reasoning_steps": [
"1. Identify the components involved in the request lifecycle.",
"2. Analyze the latency breakdown per component.",
"3. Compare current metrics to historical baselines.",
"4. Hypothesize the most likely cause based on patterns.",
"5. Propose a solution and justify it with evidence."
],
"output_format": {
"schema": {
"type": "object",
"properties": {
"components": { "type": "array", "items": { "type": "string" } },
"latency_breakdown": {
"type": "object",
"properties": {
"backend": { "type": "number" },
"database": { "type": "number" },
"cache": { "type": "number" },
"frontend": { "type": "number" }
}
},
"hypothesis": { "type": "string" },
"solution": { "type": "string" },
"justification": { "type": "string" }
}
}
}
}The model returns a structured output, but the chain-of-thought is embedded in the hypothesis and justification fields. The system can extract the reasoning process and use it for debugging logs or documentation.
Common failure: the model skips a step or merges two steps. To enforce rigor, add a step_validation block:
"step_validation": {
"required": true,
"check_order": true,
"check_completeness": true,
"include_evidence": true
}The model now ensures that each reasoning step is complete, in order, and supported by evidence from the input data.
When the model’s output is used to train or refine the prompt itself, use a feedback-driven refinement loop.
The process starts with an initial prompt and a feedback object that captures errors and suggestions from a validation step.
Example feedback:
{
"prompt_id": "onboarding_workflow_v1",
"errors": [
{
"field": "steps.duration_days",
"actual": 4.2,
"expected": "integer",
"message": "Duration must be integer, but model returned float."
},
{
"field": "steps.responsible_role",
"actual": "engineer",
"expected": "one of: Product Manager, Engineer, Designer, Customer Success",
"message": "Role 'engineer' is not in the allowed list."
}
],
"suggestions": [
"Add 'onboarding' as a required keyword.",
"Include a 'next_steps' section in the output.",
"Use a bullet list for deliverables instead of JSON array."
]
}This feedback is used to generate a revised prompt:
{
"original_prompt": { ... },
"revision_reasons": [
"Fix type mismatch in duration_days",
"Add validation for role values",
"Incorporate feedback from 3 production runs"
],
"updated_prompt": {
"prompt": "Generate a customer onboarding workflow for a SaaS product...",
"output_format": {
"schema": {
"properties": {
"steps": {
"items": {
"properties": {
"responsible_role": {
"type": "string",
"enum": ["Product Manager", "Engineer", "Designer", "Customer Success"],
"description": "Must match exactly from the list."
},
"duration_days": {
"type": "integer",
"description": "Duration must be an integer (e.g., 3, 5, 7), not a float."
}
}
}
},
"next_steps": {
"type": "array",
"items": { "type": "string" },
"description": "List of actions to take after onboarding."
}
}
},
"format": "Use bullet points for deliverables."
},
"constraints": [
{
"type": "keywords",
"value": ["onboarding", "product manager", "engineer", "designer", "customer success"],
"min_count": 2
}
]
}
}This loop enables continuous improvement of prompts without requiring model retraining, and is especially effective in production systems with high-volume, low-latency requirements.
This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy