Prompt Engineering & Structured Output

In the previous article, we moved from running AI model locally with Ollama to communicating with it through Spring AI. Now that we can send prompts to an AI model, the next question is: how can we make those prompts more effective and get more predictable responses?

Prompt Fundamentals

A prompt is an input to an AI model. It doesn't just contain the question; it can also include context, instructions, constraints and examples.

For example:

You are an experienced Java architect.
 
Explain HashMap to a senior Java developer.
 
Focus on internal implementation, collision handling,
time complexity and Java 8+ improvements.
 
Keep the answer under 150 words.
 
What is a HashMap?

Here, the question is only one part of the prompt. The additional information helps the model understand the context and the type of response we expect.

A simple way to think about a prompt is:

Prompt
│
├── Context
├── Instructions
├── Constraints
├── Examples (optional)
└── User input

The 5-Part Prompt Structure

A useful way to structure a prompt is to think about five key elements:

ROLE
CONTEXT
TASK
CONSTRAINTS
OUTPUT FORMAT

For example:

## ROLE
You are a senior software architect and system design reviewer, acting
as an interviewer who assesses a candidate's system design skills.
 
## CONTEXT
The user is an experienced software developer who wants to improve
their practical system design and architectural decision-making skills.
 
## TASK
Present a realistic software or system design scenario, then evaluate
the user's proposed solution by assessing their ability to design
scalable, reliable, and maintainable systems.
 
## CONSTRAINTS
* Focus on scalability, reliability, performance, security, and maintainability.
* Prefer realistic scenarios over theoretical questions.
* Encourage the user to explain their architectural decisions and trade-offs.
* Do not immediately provide the ideal solution.
* Ask one scenario or follow-up question at a time.
* Adapt the difficulty based on the user's answers.
* Keep responses concise.
 
## OUTPUT FORMAT
For each scenario, provide:
 
Scenario:
Difficulty:
Focus Areas:

The five parts don't have to appear literally as headings in every prompt. They provide a useful structure for thinking about what information the model needs.

Prompt Templates

In a real application, prompts are often generated dynamically rather than hard-coded for a single input.

A prompt template can contain placeholders:

You are a senior software architect.
 
Present a <difficulty> system design scenario
about <topic>.
 
Focus particularly on <focus>.

The application can then provide:

topic      = URL shortener
difficulty = Medium
focus      = Scalability

Conceptually:

Implementing Prompt Templates with Spring AI

Add an endpoint:

@GetMapping("/design-review")
public String designReview(
        @RequestParam String topic,
        @RequestParam String difficulty,
        @RequestParam String focus) {
 
    return aiService.designReview(topic, difficulty, focus);
}

The service can create the template, provide the variables and send the resolved prompt to the model:

public String designReview(String topic, String difficulty, String focus) {
 
    String template = """
            You are a senior software architect and system design reviewer.
 
            Assess the user's understanding of software architecture.
 
            Present a realistic <difficulty> system design scenario
            about <topic>.
 
            Focus particularly on:
            <focus>
 
            Do not provide the ideal solution.
            Ask one question at a time.
            Keep the response concise.
 
            Output:
 
            Scenario:
            Difficulty:
            Focus Areas:
            """;
 
    PromptTemplate promptTemplate = PromptTemplate.builder()
            .template(template)
            .variables(Map.of(
                    "topic", topic,
                    "difficulty", difficulty,
                    "focus", focus))
            .build();
 
    return chatClient.prompt(promptTemplate.create())
            .call()
            .chatResponse()
            .getResult()
            .getOutput()
            .getText();
}

We can test it with:

GET http://localhost:8080/design-review?topic=URL%20shortener&difficulty=Medium&focus=Scalability

The main benefit is that the same prompt can now be reused with different inputs.

Few-Shot Prompting

With few-shot prompting, instead of only describing what we want, we provide examples.

The model can use these examples to infer the desired pattern.

For example:

Input: I absolutely loved this product.
Output: POSITIVE
 
Input: The product stopped working after one day.
Output: NEGATIVE
 
Input: The product arrived yesterday.
Output: NEUTRAL
 
Input: The customer service was fantastic.
Output:

There are three common forms:

Zero-shot — no examples
One-shot — one example
Few-shot — multiple examples

Few-Shot Implementation

Add an endpoint:

@GetMapping("/design-review-few-shots")
public String designReviewFewShots(
        @RequestParam String topic,
        @RequestParam String difficulty,
        @RequestParam String focus) {
 
    return aiService.designReviewFewShot(topic, difficulty, focus);
}

The service method can include examples directly in the prompt:

public String designReviewFewShot(
        String topic,
        String difficulty,
        String focus) {
 
    String fewShotTemplate = """
            You are a senior software architect and system design reviewer.
 
            Generate a realistic system design scenario.
 
            Here are examples of the expected style.
 
            Example 1:
 
            Topic: URL shortener
            Difficulty: Medium
            Focus: Scalability
 
            Scenario:
            Design a URL shortening service capable of handling high read traffic.
 
            Question:
            How would you generate unique short URLs while supporting horizontal scaling?
 
 
            Example 2:
 
            Topic: Payment service
            Difficulty: Hard
            Focus: Reliability
 
            Scenario:
            Design a payment processing service that remains reliable during
            network failures and duplicate requests.
 
            Question:
            How would you guarantee idempotency when processing payments?
 
 
            Now generate a scenario for:
 
            Topic: <topic>
            Difficulty: <difficulty>
            Focus: <focus>
 
            Do not provide the solution.
            Ask one question at a time.
            """;
 
    PromptTemplate promptTemplate = PromptTemplate.builder()
            .template(fewShotTemplate)
            .variables(Map.of(
                    "topic", topic,
                    "difficulty", difficulty,
                    "focus", focus))
            .build();
 
    return chatClient.prompt(promptTemplate.create())
            .call()
            .content();
}

Test it with:

GET http://localhost:8080/design-review-few-shots?topic=Real-time%20chat%20application&difficulty=Hard&focus=Reliability%20and%20message%20delivery

A sample response might be:

Question: What design principles should be applied to ensure a real-time
chat application remains reliable during high traffic and guarantees
message delivery accuracy in the face of potential disruptions?

The key difference is that we're not just telling the model what to produce — we're showing it examples of the expected style and structure.

Output Constraints

So far, we've focused on controlling the prompt.

We can also explicitly constrain the format of the model's response.

For example, suppose we want the response to be JSON:

@GetMapping("/design-review-json")
public String designReviewJson(
        @RequestParam String topic,
        @RequestParam String difficulty,
        @RequestParam String focus) {
 
    return aiService.designReviewJson(topic, difficulty, focus);
}

The service can instruct the model to return only the required JSON structure:

public String designReviewJson(
        String topic,
        String difficulty,
        String focus) {
 
    String template = """
           You are a senior software architect.
 
           Generate a system design scenario about a <topic>.
           Focus particularly on: <focus> and difficulty: <difficulty>
 
           Return ONLY valid JSON.
 
           The JSON must contain exactly these fields:
 
           {
             "scenario": "String",
             "difficulty": "String",
             "question": "String"
           }
 
           Do not include markdown.
           Do not include ```json.
           Do not include explanations.
           """;
 
    PromptTemplate promptTemplate = PromptTemplate.builder()
            .template(template)
            .variables(Map.of(
                    "topic", topic,
                    "difficulty", difficulty,
                    "focus", focus))
            .renderer(StTemplateRenderer.builder()
                    .startDelimiterToken('<')
                    .endDelimiterToken('>')
                    .build())
            .build();
 
    return chatClient.prompt(promptTemplate.create())
            .call()
            .content();
}

Calling:

http://localhost:8080/design-review-json?topic=URL%20shortener&difficulty=Medium&focus=Scalability

can produce:

{
  "scenario": "A real-time data processing system that prioritizes latency and scalability, with a focus on optimizing data pipeline throughput and caching strategies.",
  "difficulty": "medium",
  "question": "How does the system handle real-time data processing with fluctuating traffic demands and ensure optimal latency while minimizing operational complexity?"
}

Output constraints are useful, but asking the model to return JSON doesn't automatically make that JSON suitable for application business logic.

That's where structured output becomes useful.

Structured Output

Structured output allows us to transform an LLM response into a format that our application can work with directly.

Without structured output, the flow looks like:

Prompt → LLM → String → "Hopefully valid JSON"

With structured output:

Prompt → LLM → Structured Output → Java Object

Implementing Structured Output

First, create a Java model:

public record DesignReview(
        String scenario,
        String difficulty,
        String question
) { }

Then create an endpoint:

@GetMapping("/design-review-structured")
public DesignReview designReviewStructured(
        @RequestParam String topic,
        @RequestParam String difficulty,
        @RequestParam String focus) {
 
    return aiService.designReviewStructured(
            topic, difficulty, focus);
}

The service method can use Spring AI's entity() method:

public DesignReview designReviewStructured(
        String topic,
        String difficulty,
        String focus) {
 
    String template = """
            You are a senior software architect.
 
            Generate a system design scenario about <topic>.
 
            Focus particularly on <focus>.
            Difficulty: <difficulty>.
 
            Provide:
            - a realistic scenario
            - the difficulty
            - one architectural question
 
            Do not provide the solution.
            """;
 
    PromptTemplate promptTemplate = PromptTemplate.builder()
            .template(template)
            .variables(Map.of(
                    "topic", topic,
                    "difficulty", difficulty,
                    "focus", focus))
            .renderer(StTemplateRenderer.builder()
                    .startDelimiterToken('<')
                    .endDelimiterToken('>')
                    .build())
            .build();
 
    ChatClient.CallResponseSpec responseSpec =
            chatClient.prompt(promptTemplate.create())
                    .call();
 
    System.out.println("Response=" + responseSpec.content());
 
    return responseSpec.entity(DesignReview.class);
}

Now calling:

GET http://localhost:8080/design-review-structured?topic=URL%20shortener&difficulty=Medium&focus=Scalability

can return a Java object that is serialised as:

{
    "scenario": "A global e-commerce platform must handle 10 million daily requests while maintaining uptime across 50 data centers...",
    "difficulty": "Moderate",
    "question": "Design a redundant data center architecture to ensure business continuity during high-traffic events..."
}

One interesting thing to observe is the difference between the raw model response and the API response.

The raw response logged by the application may contain Markdown and additional text:

Response=**Scenario:**
A real-time data streaming platform must handle millions of concurrent requests...
 
**Difficulty:** Moderate
 
**Architectural Question:**
How can we design a system that scales horizontally...

Yet the API response is still returned as a structured DesignReview object.

Spring AI has handled the mapping for us.

Structured Output Does Not Guarantee Correctness

There is an important distinction here.

Suppose we send:

topic = URL shortener
difficulty = Medium
focus = Scalability

but the model returns something like:

scenario = global e-commerce platform
difficulty = Moderate

The response is successfully mapped into our DesignReview object.

So we have:

Valid Java object
        ≠
Correct AI response

This leads to an important principle:

Deserialization guarantees structure, not semantic correctness.

The application should therefore treat AI output as untrusted input, even when it has been successfully converted into a Java object.

Type Safety with Java Enums

Some fields have a finite set of valid values.

For example:

EASY
MEDIUM
HARD

Using a String allows any value to be returned.

We can improve this by using an enum:

public enum Difficulty {
    EASY,
    MEDIUM,
    HARD
}

and update our model:

public record DesignReview(
        String scenario,
        Difficulty difficulty,
        String question
) { }

Now the difficulty field is represented using a well-defined Java type.

Calling the same endpoint can produce:

{
    "scenario": "A microservices-based cloud platform must handle real-time user authentication...",
    "difficulty": "MEDIUM",
    "question": "How can a distributed system ensure real-time consensus over multiple nodes in a high-latency environment?"
}

The enum gives us type safety for fields with a finite set of allowed values.

Bean Validation

Enums are useful for finite values, but they don't help with fields such as:

scenario = ""
question = ""

For those fields, we can use Bean Validation.

First, add the validation dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-validation</artifactId>
</dependency>

Then add validation annotations to the model:

public record DesignReview(
 
        @NotBlank
        String scenario,
 
        @NotNull
        Difficulty difficulty,
 
        @NotBlank
        String question
 
) { }

Finally, validate the object after receiving the structured response:

DesignReview review =
        responseSpec.entity(DesignReview.class);
 
Set<ConstraintViolation<DesignReview>> validations =
        validator.validate(review);
 
if (!validations.isEmpty()) {
    throw new IllegalArgumentException(
            "Invalid AI response: " + validations);
}
 
return review;

The complete pipeline is now:

A detailed implementation is available in the /design-review-valid-structured endpoint in the source code.

Handling Invalid AI Output

What should happen when validation fails?

There are several approaches.

Reject

LLM → invalid → error

This is appropriate when invalid AI output must never reach business logic.

Retry

LLM → invalid
       ↓
     retry
       ↓
      LLM

Retrying can be useful, but it increases latency and model usage.

Ask the Model to Correct Itself

LLM
 ↓
invalid output
 ↓
"Your previous response was invalid because..."
 ↓
LLM
 ↓
corrected output

This can work, but the second response should also be validated.

It's also important to place a limit on retries:

Attempt 1
   ↓
Invalid
   ↓
Attempt 2
   ↓
Invalid
   ↓
Attempt 3
   ↓
Invalid
   ↓
FAIL

Fallback

LLM → invalid
       ↓
    fallback
       ↓
deterministic application behaviour

For critical functionality, a deterministic fallback can often be the safest option.

The overall flow becomes:

The source repository also contains a /design-review POST endpoint demonstrating the concepts covered in this article.

Conclusion

Prompt engineering helps us control what the model receives and how it responds.

Structured output takes this a step further by allowing the application to work with a Java object instead of raw text. But structure alone isn't enough — validation is still required before AI-generated data reaches business logic.

This is the foundation for building more reliable AI-powered applications with Spring AI.