Write prompts that produce consistent, high-quality output from large language models in production systems. Covers prompt structure, few-shot learning, chain-of-thought, output formatting, guardrails, evaluation, and the patterns that turn unpredictable AI into reliable software components.
Prompt engineering is the disciplined practice of crafting input instructions for large language models to produce consistent, accurate, and actionable outputs—especially in software development workflows where precision and reliability are paramount. It matters when LLMs are not just tools for exploration but core components of production systems: code generation, documentation synthesis, test case creation, debugging, and API specification. This page details how developers can engineer prompts that yield predictable results across diverse environments, from local dev to CI/CD and deployment pipelines.
A well-structured prompt explicitly guides the model through a sequence of steps, reducing ambiguity and variability. The key is to break down the task into clear, sequential instructions, with each step building on the previous. The structure should include:
For example, when generating a REST API spec from a function signature, a prompt like this ensures consistency across runs:
You are an API design expert. Generate a detailed OpenAPI 3.0 specification from the following function signature and its context.
Function:
def create_user(email: str, name: str, age: int, active: bool) -> dict:
...
Context:
- The user service runs on a Kubernetes cluster with Redis caching.
- The email must be unique and validated using a 6-digit code sent via email.
- The response includes a `user_id` and a `created_at` timestamp.
Instructions:
1. Define the base path as `/v1/users`.
2. Create a POST endpoint at `/v1/users` with the following:
- Request body: a JSON object with `email`, `name`, `age`, and `active`.
- Validate `email` using a regex pattern: ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$.
- Ensure `age` is between 1 and 120.
- Return a 201 Created status on success.
3. Include the following response schema:
- `user_id` (string, required)
- `email` (string, required)
- `name` (string, required)
- `age` (integer, optional)
- `active` (boolean, optional)
- `created_at` (string, ISO 8601 format, required)
4. Add the following error responses:
- 400 Bad Request: if the request body is malformed.
- 409 Conflict: if the email already exists.
5. Return the full OpenAPI spec as a JSON object.This prompt produces consistent output across different models and runs, with predictable structure and content.
System prompts define the model’s role, tone, and expectations. They are fixed across multiple interactions and are critical for stabilizing behavior. Use a system prompt to set the model’s persona, such as “You are a senior backend engineer reviewing pull requests.”
In a production workflow, this system prompt is sent as part of the request body to the model API:
{
"model": "gpt-4-turbo",
"messages": [
{
"role": "system",
"content": "You are a senior backend engineer reviewing pull requests. Your reviews should be concise, technically precise, and include code suggestions, performance considerations, and potential edge cases. Prioritize clarity and actionable feedback."
},
{
"role": "user",
"content": "Here is a PR that adds rate-limiting to the user service.\n\nChanges:\n- Added Redis-backed rate limiter.\n- Implemented sliding window algorithm.\n- Added test suite with 12 test cases.\n- Updated API documentation.\n\nPlease review the PR."
}
],
"temperature": 0.4,
"max_tokens": 1024,
"top_p": 0.9,
"presence_penalty": 0.3,
"frequency_penalty": 0.3
}The temperature and top_p values are crucial: temperature=0.4 reduces randomness, while top_p=0.9 ensures the model selects from the most probable tokens, yielding more predictable outputs.
Few-shot prompting adds input-output examples to the prompt to guide the model’s behavior. This is effective when the task has subtle nuances or edge cases not captured by a single instruction.
For instance, when generating test cases from a function’s docstring, include three examples:
Generate test cases for the following function. Use the format:
- Test case: <description>
- Input: <JSON object>
- Expected output: <JSON object>
Example 1:
- Test case: Handles valid user creation
- Input: {"email": "[email protected]", "name": "Alice", "age": 30, "active": true}
- Expected output: {"user_id": "u123", "email": "[email protected]", "name": "Alice", "age": 30, "active": true, "created_at": "2023-09-15T10:00:00Z"}
Example 2:
- Test case: Fails for invalid email
- Input: {"email": "invalid-email", "name": "Bob", "age": 25, "active": false}
- Expected output: {"error": "email_invalid", "details": {"field": "email", "value": "invalid-email"}}
Example 3:
- Test case: Creates user with optional fields
- Input: {"email": "[email protected]", "name": "Charlie", "age": 35}
- Expected output: {"user_id": "u124", "email": "[email protected]", "name": "Charlie", "age": 35, "active": true, "created_at": "2023-09-15T10:00:00Z"}
Now generate test cases for:
def process_order(items: list, customer_id: str, shipping_method: str) -> dict:
"""Process an order and return confirmation details.
- Items must be non-empty.
- If shipping_method is 'express', add a 10% premium.
- Returns confirmation ID, total, and estimated delivery date."""The model now learns from the examples and adapts to the expected format, reducing variability in test case generation.
LLMs often return unstructured text, even when the downstream system expects a schema. To enforce structure, use explicit output specifications. For example, to extract metadata from a code file, specify the output format using JSON, YAML, or a well-defined template.
When generating a changelog from a Git log, use a prompt that specifies the output as a JSON array of changelog entries:
Extract a changelog from the following Git commit history. Return a JSON array of objects with the following schema:
{
"version": "string",
"date": "string (ISO 8601)",
"changes": [
{
"type": "string (feature|bugfix|documentation|performance|refactor|other)",
"title": "string",
"description": "string",
"author": "string",
"pr_url": "string"
}
]
}
Commit history:
- [feat] Add user authentication (PR #42)
- [bugfix] Fix login timeout (PR #41)
- [docs] Update README with setup guide (PR #40)
- [refactor] Simplify order processing logic (PR #39)This ensures that the output is machine-readable and can be consumed directly by a CI/CD pipeline or a release automation tool.
As prompts grow, they become harder to maintain, version, and debug. Use prompt templates with placeholders for dynamic values. For example, a Jinja2-style template in a Python script:
from jinja2 import Template
prompt_template = Template("""
Generate a documentation string for the function {{ function_name }} in the module {{ module_path }}.
Context:
- The function is part of a {{ service_name }} service.
- It handles {{ data_type }} processing with {{ performance_goal }} performance.
Instructions:
1. Write a docstring in Google style.
2. Include parameters, returns, and raises sections.
3. Add an example usage block with a code snippet.
Function signature:
def {{ function_name }}({{ parameters }}) -> {{ return_type }}:
...
Examples:
- Use the example values: {{ example_values }}
- Show how to call the function and handle the result.
""")
prompt = prompt_template.render(
function_name="process_batch",
module_path="data/processing.py",
service_name="data ingestion",
data_type="streaming events",
performance_goal="sub-second latency",
parameters="events: List[Event], batch_size: int",
return_type="Dict[str, Any]",
example_values={"events": [{"id": 1, "type": "click"}], "batch_size": 10}
)This approach allows developers to version prompts, reuse them across projects, and dynamically construct prompts based on context.
temperature=1.0 produces verbose, creative outputs but introduces inconsistency across runs. A temperature=0.4 or 0.6 is more predictable.top_p: top_p=1.0 is equivalent to greedy decoding. Setting top_p=0.9 without adjusting temperature leads to suboptimal token selection.presence_penalty and frequency_penalty: These control the model’s tendency to introduce new words (presence_penalty) and reuse existing ones (frequency_penalty). A common mistake is setting both to 0.5 without understanding their interaction.python instead of ``). This leads to misparsing in the model.max_tokens and context_window to monitor this. A common failure is a model that generates correct output but ignores the first few examples in a few-shot prompt.When output is unexpectedly inconsistent, inspect the prompt as it was sent to the model. Use curl to test a prompt directly:
curl -X POST https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4-turbo",
"messages": [
{
"role": "system",
"content": "You are a code reviewer. Be concise. Prioritize correctness over style."
},
{
"role": "user",
"content": "Review this function:\n\n```python\nasync def fetch_user(id: int) -> dict:\n return await db.query(\"SELECT * FROM users WHERE id = ?\", id)\n```\n\nWhat do you think?"
}
],
"temperature": 0.3,
"max_tokens": 512
}' | jq .Use jq to inspect the response, verify that the model output includes expected sections, and compare outputs across different runs.
temperature, top_p, presence_penalty, and frequency_penalty.With these practices, developers can transform LLM interactions from ad-hoc explorations into reliable, repeatable components of their software systems.
This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy