Spring AI is an application framework for AI engineering on the JVM. It lets you talk to large language models with the same fluent, dependency-injected style you already use for the web and the database.
This is a beginner’s tour. By the end you will know what Spring AI is and how to make your first model call, then connect RAG and memory from a plain Spring Boot app.
Table of contents
What Spring AI is
Spring AI is, in the project’s own words, “an application framework for AI engineering. Its goal is to apply to the AI domain Spring ecosystem design principles such as portability and modular design.”
That tells you what to expect. Spring AI is not a new model. It does not compete with OpenAI or Anthropic. It is the integration layer between your Java application and whatever model you pick. The team frames the core problem it solves as “connecting your enterprise data and APIs with AI models.”
If you have heard of LangChain in the Python world, you already have a mental model. It brings the same idea (abstractions over models, prompts, retrieval, and memory) to the JVM, with Spring Boot auto-configuration doing the wiring you would otherwise write by hand.
Your first call: the ChatClient
The center of the beginner experience is the ChatClient. The docs describe it as a “fluent API for communicating with AI chat models, idiomatically similar to the WebClient and RestClient APIs.” If you have used either, this will feel familiar.
Spring Boot auto-configures a ChatClient.Builder for you. Inject it and build the client. In a real endpoint, keep the HTTP contract explicit with request and response DTOs:
@RestController
@RequestMapping("/api/chat")
public class ChatController {
private final ChatClient chatClient;
public ChatController(ChatClient.Builder chatClientBuilder) {
this.chatClient = chatClientBuilder.build();
}
@PostMapping
GenerationResponse generation(@RequestBody GenerationRequest request) {
var modelContent = chatClient.prompt(request.userMessage()).call().content();
return new GenerationResponse(modelContent);
}
}
record GenerationRequest(String userMessage) {}
record GenerationResponse(String modelContent) {}That is a complete, working AI endpoint. Four steps carry the whole interaction:
prompt(request.userMessage())starts the fluent chain with the user message.call()sends the request to the model.content()returns the response as aString.new GenerationResponse(modelContent)wraps the model output in your response DTO.
Send a POST request and you get the model’s reply. The same ChatClient shape works whether the model behind it comes from OpenAI, Anthropic, or runs locally. In this guide I will use a local model through LM Studio with an OpenAI-compatible API.
Setting up Spring AI
To get that controller running you need the Spring AI BOM, the provider starter, an API key, and a few properties to wire them together.
The starter artifacts follow a predictable pattern, spring-ai-starter-model-{provider}. For OpenAI:
plugins {
id 'java'
id 'org.springframework.boot' version '4.1.0'
id 'io.spring.dependency-management' version '1.1.7'
}
repositories {
mavenCentral()
}
ext {
set('springAiVersion', "2.0.0-RC2")
}
dependencies {
implementation 'org.springframework.ai:spring-ai-starter-model-openai'
}
dependencyManagement {
imports {
mavenBom "org.springframework.ai:spring-ai-bom:${springAiVersion}"
}
}The starter triggers the auto-configuration that hands you a ready ChatClient.Builder. Each provider reads its own properties. For an OpenAI-compatible API:
spring.ai.openai.api-key=${OPENAI_API_KEY:local-model}
spring.ai.openai.base-url=${OPENAI_BASE_URL:http://localhost:1234/v1}
spring.ai.openai.chat.model=${OPENAI_MODEL:nvidia/nemotron-3-nano-4b}Keep the real key in an environment variable rather than in application.properties; for local models the provided placeholder is enough.
For any Spring setup from scratch, I highly recommend using Spring Initializr.
Full demo project you can find here: spring-ai-demo-project
Useful building blocks in Spring AI
A single chat call is just the beginning. Teams adopt Spring AI for the building blocks that come after chat.
We will build one thing through them all: a personal assistant. It starts as a plain chat call. By the end of this section it answers from your own notes, remembers the conversation, reads your live calendar, returns a typed object, and exposes that calendar lookup to other AI agents. Each block adds a few lines to the same assistant.
Here is the starting point. A @Service that wraps a ChatClient:
@Service
public class Assistant {
private final ChatClient chatClient;
Assistant(ChatClient.Builder builder) {
this.chatClient = builder.build();
}
public String ask(String question) {
return chatClient.prompt()
.user(question)
.call()
.content();
}
}That works, but the model has no persona and no memory. Let me fix that one block at a time.
System prompts
A prompt is more than the user’s text. It carries two kinds of message: a system prompt that sets the rules and persona, and the user message that holds the question. Spring AI builds these as typed objects, so you set them with methods instead of gluing strings together.
Give the assistant a system prompt so it behaves like a personal assistant. You set it once, as a default, when you build the client:
Assistant(ChatClient.Builder builder) {
this.chatClient = builder
.defaultSystem("""
You are my personal assistant.
Answer in two or three sentences. Be direct and friendly.
If you do not know, say so instead of guessing.
""")
.build();
}Now every call carries that system prompt in front of your question. The model receives its role before it reads a single word you type. defaultSystem sets it for the whole client; you can also set a per-call .system(...) when one request needs different rules.
Advisors
An advisor is a component that intercepts data on the way to the model and on the way back, so you can attach cross-cutting behavior without touching your call code. The Advisors API encapsulates what the docs call “recurring Generative AI patterns.” The two features you will reach for first, memory and retrieval, are both advisors. That is the payoff: register an advisor and the behavior is there.
Start with memory. Out of the box, the assistant forgets everything between calls. Ask “what is on my calendar today?” then “move the first one to 17:00” and the second question has lost track of what “the first one” is. MessageChatMemoryAdvisor fixes that by replaying recent turns into each prompt.
Assistant(ChatClient.Builder builder) {
ChatMemory chatMemory = MessageWindowChatMemory.builder().build();
this.chatClient = builder
.defaultSystem("You are my personal assistant...")
.defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
.build();
}The memory advisor needs to know which conversation a question belongs to, so separate chats never bleed into each other. You pass that id per call with ChatMemory.CONVERSATION_ID:
String ask(String conversationId, String question) {
return chatClient.prompt()
.user(question)
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
.call()
.content();
}MessageWindowChatMemory keeps a rolling window of recent messages (20 by default), so the prompt does not grow without bound. The advisor requires the conversation id: leave it out and it throws at runtime. Now the assistant remembers, but it still only knows the base model’s training data. It has never read a single note about you. That is the next block.
Vector stores and embeddings
An embedding is a list of numbers that captures what a piece of text is about, so a model can search by meaning rather than exact words. A vector store holds those embeddings and finds the nearest ones to a query.
To answer from your own content, the assistant first has to store that content as embeddings. Two notes about travel end up close together in that number space, even if they share no words.
Spring AI gives you one VectorStore interface across more than twenty backends, including PostgreSQL with PGvector and Redis. The same code runs against any of them.
For embeddings and storage, I will use PgVector with the text-embedding-nomic-embed-text-v1.5 local model. A real database makes the demo match what you would deploy.
spring.ai.openai.embedding.api-key=${OPENAI_API_KEY:local-model}
spring.ai.openai.embedding.base-url=${OPENAI_EMBEDDING_BASE_URL:http://localhost:1234/v1}
spring.ai.openai.embedding.model=${OPENAI_EMBEDDING_MODEL:text-embedding-nomic-embed-text-v1.5}
spring.ai.vectorstore.pgvector.initialize-schema=truePgVector comes in through its own starter, spring-ai-starter-vector-store-pgvector, plus a running PostgreSQL; initialize-schema=true creates the vector table at startup.
Load your notes once at startup. Each note becomes a Document, and vectorStore.add(...) turns it into an embedding and stores it:
@Component
class NotesLoader {
NotesLoader(VectorStore vectorStore) {
vectorStore.add(List.of(
new Document("My name is Dima, a short name for Dmitry."),
new Document("I love jelly beans, so make sure I have them in the fridge."),
new Document("I prefer window seats on flights with extra leg room."),
new Document("My wedding anniversary is on 30 April."),
new Document("I go to the gym on Monday and Thursday.")
));
}
}You never call an embedding model by hand here. add uses the auto-configured EmbeddingModel to vectorize each note for you. The store now holds five searchable facts about you. The next block puts them in front of the model.
Using RAG with Spring AI
RAG is how you get a model to answer from your data instead of guessing. The flow is: take the question, search the vector store for the most relevant notes, paste them into the prompt, and let the model answer grounded in that text. The model can still draw on its own training when the retrieved notes do not cover the question, so the system prompt you set earlier matters: “If you do not know, say so instead of guessing” is what steers it away from invention.
Spring AI ships this as an advisor too: QuestionAnswerAdvisor. Hand it the vector store and add it to the client. It runs the search and injects the matches on every call, so your ask method does not change:
Assistant(ChatClient.Builder builder, VectorStore vectorStore) {
var chatMemory = MessageWindowChatMemory.builder().build();
var memoryAdvisor = MessageChatMemoryAdvisor.builder(chatMemory).build();
var questionAnswerAdvisor = QuestionAnswerAdvisor.builder(vectorStore).build();
this.chatClient = builder
.defaultSystem(systemPrompt)
.defaultAdvisors(memoryAdvisor, questionAnswerAdvisor)
.build();
}Ask “what seat should you book me on the next flight?” and the advisor finds the window-seat note and injects it. The model then answers from your preference rather than guessing. You can tune the search with a SearchRequest, for example to set a similarity floor and cap the number of notes:
QuestionAnswerAdvisor.builder(vectorStore)
.searchRequest(SearchRequest.builder()
.similarityThreshold(0.7)
.topK(4)
.build())
.build();topK is how many notes to pull, and similarityThreshold drops weak matches to keep unrelated notes out of the prompt. The assistant now has your notes in context for each answer. But “what is on my calendar today?” is not in any note. That answer changes by the hour, and the model needs to fetch it live.
Found this useful? Follow me on social media to stay updated.
Tool calling
Tool calling (also called function calling) lets the model request the execution of Java methods you register, so it can reach live data beyond its training. You write a normal method, mark it with @Tool, and the model decides when to call it. The model never runs your code itself; it asks Spring AI to run the method and hands back the result. Use it when the answer changes by the hour, like a calendar or a price feed.
Give the assistant a calendar lookup. In real code the method hits your calendar API; here it returns a canned answer:
@Component
class CalendarTools {
@Tool(description = "Get the agenda for a given day from the user's calendar")
public String getAgenda(
@ToolParam(description = "The day in ISO format, for example 2026-06-11") String day) {
// real code would query your calendar API
return "On " + day + ": standup at 10:00, dentist at 16:00.";
}
@Tool(description = "Fetch current (today) date in ISO format")
public String getCurrentDate() {
return LocalDate.now().toString();
}
}Register the tool on the call with .tools(...). The model reads the @Tool description and, if it decides the question calls for it, requests execution with the day it parsed from the question:
String ask(String conversationId, String question, CalendarTools calendarTools) {
return chatClient.prompt()
.user(question)
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
.tools(calendarTools)
.call()
.content();
}A question like “what is on my calendar today?” can lead the model to call getCurrentDate for today’s date and then getAgenda. It reads the result and replies in plain language. @ToolParam descriptions tell the model what to put in each argument. The assistant can answer from your notes and from live data. The last gap is the shape of the answer.
Structured output
Structured output maps the model’s response straight onto a Java record, so you get a typed object instead of a free-text string. Free text reads well to a human and badly to code. If another part of your system needs to route the answer or flag it, parsing a paragraph is fragile.
Declare the shape you want as a record, then ask for it with .entity(...) instead of .content():
record AssistantReply(String answer, String category, boolean needsFollowUp) {}
AssistantReply reply(String conversationId, String question, CalendarTools calendarTools) {
return chatClient.prompt()
.user(question)
.advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
.tools(calendarTools)
.call()
.entity(AssistantReply.class);
}Spring AI sends a format instruction that requests JSON shaped like AssistantReply, then converts the response back into the record. Now you can act on the result in code: route by category, or schedule a reminder when needsFollowUp is true. The same assistant that answers in prose for a chat window returns a typed object for the rest of your app.
Model Context Protocol (MCP)
MCP is the newer, fast-rising piece, an open protocol that connects AI applications to external tools and data sources in a standard way. Spring AI provides both client and server Boot starters. With the server starter, a Spring app exposes its own tools so any MCP-aware agent (an IDE assistant, a desktop AI app, another service) can call them over a standard protocol.
You already wrote a tool worth sharing: getAgenda. Expose it as an MCP server and an outside agent can read your calendar without ever touching your REST API. Add the server starter:
implementation 'org.springframework.ai:spring-ai-starter-mcp-server-webmvc'Then publish the same CalendarTools as MCP tools with a ToolCallbackProvider bean. No new tool code; you reuse the class from the tool-calling block:
@Configuration
public class McpCalendarToolsConfiguration {
@Bean
public ToolCallbackProvider toolCallbackProvider(CalendarTools calendarTools) {
return MethodToolCallbackProvider.builder()
.toolObjects(calendarTools)
.build();
}
}Optionally, give the server a name in application.properties:
spring.ai.mcp.server.name=mcp-server-demo-ai
spring.ai.mcp.server.version=1.0.0The same @Tool method now serves two callers: your own assistant through .tools(...), and any external agent through MCP. That reuse is the point of the protocol. If you file one term away from this post, make it this one.
The starter speaks the SSE protocol by default. Switch it to streamable HTTP, and the server listens on the /mcp path; override the path with the second property:
spring.ai.mcp.server.protocol=STREAMABLE
spring.ai.mcp.server.streamable-http.mcp-endpoint=/mcpThe supporting cast
A few more pieces round out the framework, and you will meet them once the assistant goes past a demo. An ETL pipeline reads real documents (PDFs, web pages) and chunks them before they reach the vector store, so you are not hand-typing notes. Model evaluation utilities check generated content and help guard against hallucinated responses.
That is the whole assistant. It started as one chat call and grew, a few lines at a time, into something that knows your notes, remembers the conversation, reads live data, and returns structured output.
Where to go next with Spring AI
You now have the map. Spring AI is the Spring-native layer between your Java app and AI models, built on portability and the same dependency-injection style you already use. The ChatClient gets you a first response in minutes. Prompts, advisors, RAG, vector stores, tool calling, structured output, and MCP are the rooms you explore after that.
A learning path that works: get the /api/chat endpoint returning a reply, point it at a local model, then add structured output so the response comes back as a typed object. After that, try a small RAG example over a handful of your own documents. Each step adds a dependency and a few lines of Spring AI code.
Do you use AI in your development workflow? Read my latest: AI Coding Workflow: TDD, Reviews, and Guardrails
Found this useful? Follow me on social media to stay updated.
