Parse structured data from LLM responses reliably using JSON mode, XML tags, Pydantic validators, and retry-with-feedback loops.
LLM Output Parsing is the process of transforming unstructured, free-form text responses from large language models into structured data suitable for downstream systems. It matters when LLMs generate answers, summaries, configurations, or code that must be consumed by other services—whether a database, a workflow engine, a dashboard, or a client application. A parser transforms a raw text output into json, csv, xml, or a database schema, enabling reliable integration of LLMs into production systems. This is especially critical when the LLM’s output is not just a human-readable string, but a machine-readable payload.
When an LLM returns a response like:
Task: Review the pull request #42.
Summary: The changes improve performance by 30% and fix a memory leak in the user session handler.
- Reviewed by: Alice Chen
- Priority: High
- Required actions:
- [ ] Add unit tests for the new cache layer
- [ ] Update documentation in docs/README.md
- [ ] Deploy to staging and verify
Labels: performance, bug, reviewThe consumer system must extract structured data: a task, summary, reviewer, priority, actions, and labels. The parser must handle variation in formatting, missing fields, and inconsistent capitalization.
The most common approach uses prompt templates that guide the LLM toward a predictable output format. A typical prompt includes:
Generate a response in the following format:
- Task: <title>
- Summary: <text>
- Reviewed by: <name>
- Priority: <High|Medium|Low>
- Required actions:
- [ ] <task>
- [ ] <task>
- Labels: <comma-separated list>
Return only the text in this format, no extra content.The parser then applies a format-specific extraction strategy based on the expected structure.
When the LLM is instructed to produce valid JSON, the parsing is more reliable. The LLM response may look like:
{
"task": "Review pull request #42",
"summary": "The changes improve performance by 30% and fix a memory leak in the user session handler.",
"reviewer": "Alice Chen",
"priority": "High",
"required_actions": [
"Add unit tests for the new cache layer",
"Update documentation in docs/README.md",
"Deploy to staging and verify"
],
"labels": [
"performance",
"bug",
"review"
]
}The consumer system parses this using standard JSON parsing libraries, with the following pattern:
import json
response = client.generate(
prompt="...prompt...",
temperature=0.3,
max_tokens=512,
response_format="json_object" # LLM-specific flag
)
try:
parsed = json.loads(response.text)
except json.JSONDecodeError as e:
logger.error(f"Failed to parse LLM output: {e}")
# Fall back to text-based parserKey failure mode: The LLM returns JSON that is not valid—missing commas, unquoted keys, or extra trailing commas. The parser must validate the structure and handle partial or malformed outputs.
The response_format="json_object" flag is critical. Without it, the LLM may return a JSON string wrapped in a code block:
```json
{
"task": "Review pull request #42"
}
The parser must detect and extract the inner JSON. A common error is assuming the output is pure JSON when it is actually a Markdown code block.
---
### Parsing Key-Value Blocks
When the LLM output is a key-value block (as in the example above), the parser uses **line-based extraction**. Each line starting with a hyphen (`-`) or a colon (`:`) is a field.
A typical parser in Python:
```python
def parse_key_value_block(text: str) -> dict:
result = {}
current_key = None
current_value = []
for line in text.splitlines():
line = line.strip()
if not line:
continue
if line.startswith("- "):
# New field
key, value = line[2:].split(":", 1) if ":" in line else (line[2:], "")
if current_key:
result[current_key] = "\n".join(current_value).strip()
current_key = key.strip()
current_value = [value.strip()]
elif line.startswith(" "):
# Indented continuation
current_value.append(line.strip())
else:
# Line not starting with - or indent
current_value.append(line)
if current_key:
result[current_key] = "\n".join(current_value).strip()
return resultThis parser handles:
- Label: bug and Labels: performance, bug)Common failure: The LLM omits a colon in a key-value pair, so Priority High becomes Priority as key and High as value. The parser must detect this and either infer the format or use a default key-value separator.
When the LLM returns a bulleted or numbered list, the parser must extract items and convert them into arrays.
For example:
- Add unit tests for the new cache layer
- Update documentation in docs/README.md
- Deploy to staging and verifyThe parser applies a list extraction function:
def parse_list(text: str) -> list[str]:
items = []
for line in text.splitlines():
line = line.strip()
if not line:
continue
# Match bullet types: -, •, *, [ ], or numbers
if line.startswith("- ") or line.startswith("• ") or line.startswith("* "):
items.append(line[2:].strip())
elif line.startswith("[ ] ") or line.startswith("[x] "):
items.append(line[5:].strip())
elif re.match(r"^\d+\.\s", line):
items.append(line.split(".", 1)[1].strip())
else:
# Fallback: treat as single item
items.append(line)
return itemsThis parser detects:
[ ] Task, [x] Done)1. First item)- Task, • Summary, * Action)Sharp edge: The LLM sometimes returns a list with inconsistent indentation or missing bullets. A list may span multiple paragraphs, and the parser must group lines into items correctly.
After parsing, the system must validate the structure and normalize the data. A typical workflow:
def process_llm_output(raw_text: str, expected_schema: dict) -> dict:
parsed = parse_key_value_block(raw_text)
# Normalize keys
normalized = {
k.lower().replace(" ", "_"): v
for k, v in parsed.items()
}
# Validate required fields
for field in expected_schema["required"]:
if field not in normalized:
logger.warning(f"Missing required field: {field}")
normalized[field] = None
# Convert types
if "priority" in normalized:
priority = normalized["priority"].lower()
if priority in ["high", "h", "1"]:
normalized["priority"] = "High"
elif priority in ["medium", "m", "2"]:
normalized["priority"] = "Medium"
elif priority in ["low", "l", "3"]:
normalized["priority"] = "Low"
else:
normalized["priority"] = "Unknown"
# Parse list fields
if "required_actions" in normalized:
normalized["required_actions"] = parse_list(normalized["required_actions"])
if "labels" in normalized:
normalized["labels"] = [l.strip() for l in normalized["labels"].split(",")]
return normalizedThis function ensures the output is consistent even when the LLM varies its wording.
Critical failure mode: The LLM returns labels: performance, bug, review but the consumer expects an array. The parser must split on comma, trim, and handle spaces. A common mistake is not normalizing case: Performance, performance, and PERFORMANCE are all valid but not equivalent.
When parsing fails, the system must fall back to alternative strategies. The most common fallback is text-based extraction using regular expressions.
For example, if the JSON parser fails due to malformed JSON, the system tries:
def extract_with_regex(text: str) -> dict:
patterns = {
"task": r"Task:\s+(.+)",
"summary": r"Summary:\s+(.+)",
"reviewer": r"Reviewed by:\s+(.+)",
"priority": r"Priority:\s+(High|Medium|Low)",
"required_actions": r"Required actions:\s+(.+)",
"labels": r"Labels:\s+(.+)",
}
result = {}
for key, pattern in patterns.items():
match = re.search(pattern, text, re.IGNORECASE | re.DOTALL)
if match:
value = match.group(1)
if key == "required_actions":
result[key] = parse_list(value)
elif key == "labels":
result[key] = [l.strip() for l in value.split(",")]
else:
result[key] = value.strip()
else:
result[key] = None
return resultThe system tries JSON parsing first, then falls back to regex parsing if JSON fails.
Silent failure: The LLM returns a valid JSON string, but it is wrapped in a json code block. The parser loads the outer text but fails to parse the inner JSON. The error is not caught until logs show a missing task field.
A production system may define a parsing pipeline in configuration:
parsing_pipelines:
pull_request_review:
format: key_value
expected_fields:
- task
- summary
- reviewer
- priority
- required_actions
- labels
field_mappings:
task: "task"
summary: "summary"
reviewer: "reviewer"
priority: "priority"
actions: "required_actions"
tags: "labels"
fallback: "regex"
normalization:
- key: "priority"
map:
high: "High"
h: "High"
1: "High"
medium: "Medium"
m: "Medium"
2: "Medium"
low: "Low"
l: "Low"
3: "Low"
default_priority: "Medium"This configuration allows multiple pipelines to be reused across different LLM calls.
The pipeline is executed as:
def run_pipeline(pipeline_name: str, raw_text: str) -> dict:
config = load_pipeline_config(pipeline_name)
parsed = parse_with_format(raw_text, config["format"])
# Apply mappings
mapped = {
config["field_mappings"].get(k, k): v
for k, v in parsed.items()
}
# Validate and normalize
normalized = normalize_fields(mapped, config["normalization"])
normalized = set_default_values(normalized, config["default_priority"])
return normalizedThis approach decouples parsing logic from application code, enabling teams to evolve the parsing rules without changing the LLM or API consumers.
json.loads with error handling and fallback to text parsing.The most common mistake is assuming the LLM output is perfect. In reality, the output varies across models, prompts, and environments. The parser must be resilient, not just accurate.
This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy