How to Structure System Prompts for LLM Applications

A production system prompt should make each responsibility easy to find and change. Start with the model's role and task, then separate relevant context, variable input, constraints, output requirements, and examples. Use Markdown headings for a readable default; use XML-style tags when explicit boundaries or nested content help. Treat this structure as a maintainability pattern, not a guarantee of better model output.

The right structure is the one your team can understand, test, version, and adapt to the model and task it actually uses. OpenAI documents Markdown and XML as complementary ways to mark logical boundaries, while Anthropic recommends descriptive XML tags for complex prompts. Neither recommendation establishes a universal format that works best for every model or task.

A practical system prompt template#

Use the following as a starting point, not a mandatory schema. Remove sections that do not change behavior, and add sections only when they solve a real ambiguity. Every bracketed phrase is a placeholder to replace with application-specific content. Do not put credentials, customer secrets, or sensitive production data in prompt examples.

system-prompt-template.txtPlain text

# Role
You are [ROLE] working in [APPLICATION CONTEXT].

# Task
[DESCRIBE THE USER-VISIBLE OUTCOME].

# Context
[PROVIDE RELEVANT DOMAIN FACTS OR BUSINESS RULES].

# Input
[DESCRIBE THE INPUT FIELDS OR INSERT UNTRUSTED CONTENT HERE].

# Constraints
- [STATE IMPORTANT REQUIREMENTS].
- [STATE PROHIBITED ACTIONS OR LIMITS].
- [STATE HOW TO HANDLE MISSING OR UNCERTAIN INFORMATION].

# Output format
[DEFINE THE EXPECTED STRUCTURE, FIELDS, LENGTH, OR STYLE].

# Examples
[ADD REPRESENTATIVE INPUT/OUTPUT EXAMPLES IF THEY CLARIFY EXPECTED BEHAVIOR].

Role answers what function the assistant performs; Task states the result; Context supplies relevant facts; Input identifies the data; Constraints define requirements and fallback behavior; Output format defines the response shape; Examples clarify representative cases. A small prompt may need only a task and an output requirement. Do not add headings merely to make a short prompt look formal.

Keep instructions separate from variable data#

A recurring source of ambiguity is mixing what the model must do with the content it must process. Give those parts visibly distinct boundaries, especially when user-provided text may contain instructions of its own.

summarization-prompt.mdMarkdown

## Task
Summarize the supplied document in three bullets.
Use only claims supported by the document.

## Document
[PASTE THE DOCUMENT TO SUMMARIZE]

The placeholder marks where runtime content belongs. In an application, insert the document as data in the appropriate message or structured input field; do not assume that a delimiter alone makes untrusted content safe. Prompt injection is an application security problem requiring layered design, least-privilege tool access, and validation—not just formatting.

Markdown or XML: choose a readable boundary#

Markdown headings and lists are usually easy for a team to scan in code review. XML-style tags can make the beginning and end of sections more explicit, and tags can be nested when the data has a natural hierarchy.

prompt-markdown-example.mdMarkdown

## Task
Summarize the document in three bullets.

## Context
The reader is a software engineering manager.

## Input
[DOCUMENT TEXT]

prompt-xml-example.xmlXML

<task>
Summarize the document in three bullets.
</task>

<context>
The reader is a software engineering manager.
</context>

<input>
[DOCUMENT TEXT]
</input>

These examples express the same intended sections; neither is evidence that one format will produce better answers in a particular application. OpenAI describes Markdown and XML as usable together. Anthropic specifically recommends descriptive XML tags for complex prompts that combine instructions, context, examples, and variable inputs. Test the structure with the model and task you deploy.

If you already have prompt wording and only need to arrange it into Markdown headings, XML tags, or plain labels, use the Letinix Prompt Formatter at https://letinix.com/tools/prompt-formatter. It organizes optional sections and offers whitespace and quotation-mark cleanup; it does not rewrite the prompt's wording or optimize its instructions.

Write constraints that can be checked#

Avoid broad directions such as “be accurate” when a concrete operational rule is possible. State what the model should do when information is missing, what sources it may rely on, and what the output must include.

constraints-example.txtPlain text

- If the input does not contain the requested fact, return "unknown".
- Do not infer missing values.
- Return no more than [MAX_ITEMS] items.
- Keep each explanation under [MAX_WORDS] words.

Concrete constraints make expected behavior easier to inspect, but do not guarantee compliance. Test representative cases and enforce important requirements in application code where possible.

Treat output format as a contract#

If downstream software consumes the model response, define required fields and value types explicitly. A prompt can ask for JSON, but prompt wording alone is not equivalent to an API feature that constrains output to a schema.

OpenAI distinguishes JSON mode from Structured Outputs: JSON mode targets valid JSON but does not enforce a particular schema, while Structured Outputs can constrain output to a supported JSON Schema. Refusals, incomplete responses, unsupported schema features, and other documented edge cases still require handling. Consult https://developers.openai.com/api/docs/guides/structured-outputs.

A robust application combines a clearly specified output contract, a supported structured-output API feature where available, parsing and application-side validation, explicit handling for refusals and truncation, and tests for normal and edge cases.

Make prompts maintainable as software#

A prompt used in production is part of the application's behavior. Treat meaningful changes as code changes, not copy edits hidden in a dashboard.

Store prompts in version control; keep runtime values as placeholders; use representative examples; create a small test set; evaluate after changes; and keep application checks outside the prompt when requirements must be enforced. OpenAI's prompting guidance recommends organizing prompts so teams can locate and update them, and running tests and evaluations when publishing changes.

A practical workflow for a new prompt#

1. Define the user-visible outcome. 2. Separate stable instructions from runtime inputs. 3. Start with Markdown for readability and introduce XML tags where explicit boundaries or nesting help. 4. Specify output requirements and API-level constraints when available. 5. Add examples only when useful. 6. Format and review with the Prompt Formatter if needed. 7. Test representative inputs and version the prompt.

What prompt structure cannot guarantee#

A well-organized prompt can make responsibilities and boundaries easier for people—and potentially the model—to interpret. It does not guarantee factual accuracy, consistent behavior, valid output, resistance to prompt injection, or successful tool use. Those outcomes depend on the model, task, available context, API features, application controls, and evaluation process. Use formatting to make a prompt legible and maintainable; use tests and system-level controls to establish whether it works for your requirements.