When thinking about starting a Generative AI (GenAI) project, you might assume that Python is required to get started in this new space. However, if you\u2019re already a Java developer, there\u2019s no need to learn a new language. The Java ecosystem offers robust tools and libraries that make building GenAI applications both accessible and productive. <\/p>\n
In this blog, you\u2019ll learn how to build a GenAI app using Java. We\u2019ll do a step-by-step demo to show you how RAG enhances the model response, using Spring AI and Docker tools. Spring AI integrates with many model providers (for both chat and embeddings), vector databases, and more. In our example, we\u2019ll use the OpenAI and Qdrant modules provided by the Spring AI project to take advantage of built-in support for these integrations. Additionally, we\u2019ll use Docker Model Runner<\/a> (instead of a cloud-hosted OpenAI model), which offers an OpenAI-compatible API, making it easy to run AI models locally<\/a>. We\u2019ll automate the testing process using Testcontainers and Spring AI\u2019s tools to ensure the LLM\u2019s answers are contextually grounded in the documents we\u2019ve provided. Last, we\u2019ll show you how to use Grafana for observability and ensure our app behaves as designed.\u00a0<\/p>\n Let\u2019s start building a sample application by going to Spring Initializr<\/a> and choosing the following dependencies: Web, OpenAI, Qdrant Vector Database, and Testcontainers.<\/p>\n It\u2019ll have two endpoints: a \u201c\/chat\u201d endpoint that interacts directly with the model and a \u00a0 \u201c\/rag\u201d endpoint that provides the model with additional context from documents stored in the vector database.<\/p>\n Enable Docker Model Runner in your Docker Desktop or Docker Engine as described in the official documentation<\/a>.<\/p>\n Then pull the following two models:<\/p>\n\n ai\/llama3.1 \u2013 chat model<\/p>\n ai\/mxbai-embed-large \u2013 embedding model<\/p>\n Both models are hosted at Docker Hub under the ai<\/strong><\/a> namespace. You can also pick specific tags for the model, which usually provide different quantization of the model. If you don\u2019t know which tag to pick, the default one is a good starting point.<\/p>\n\n Let\u2019s create a ChatController<\/strong> under \/src\/main\/java\/com\/example<\/strong>, which will be our entry point to interact with the chat model:<\/p>\n\n \tprivate final ChatClient chatClient;<\/p>\n \tpublic ChatController(ChatModel chatModel) { \t@GetMapping(“\/chat”) }<\/p>\n<\/div>\n ChatClient is the interface that provides the available operations to interact with the model. We\u2019ll be injecting the actual model value (which model to use) via configuration properties.<\/p>\n If no message query param is provided, then we\u2019ll ask the model to tell a joke (as seen in the defaultValue).<\/p>\n Let\u2019s configure our application to point to Docker Model Runner and use the \u201cai\/llama3.1\u201d model by adding the following properties to \/src\/test\/resources\/application.properties<\/strong><\/p>\n\n spring.ai<\/a>.openai.api-key is required by the framework, but we can use any value here since it is not needed for Docker Model Runner.<\/p>\n Let\u2019s start our application by running .\/mvnw spring-boot:test-run<\/strong> or .\/gradlew bootTestRun <\/strong>and ask it about Testcontainers:<\/p>\n Below, we can find the answer provided by the LLM (ai\/llama3.1)<\/p>\n **What is Testcontainers?**<\/p>\n Testcontainers provides a way to run real, fully functional containerized services (like databases, message queues, web servers, etc.) directly within your tests. Instead of relying on mocked or stubbed versions of these services, you’re using the *actual* services, which leads to much more realistic and reliable test results.<\/p>\n **Why Use Testcontainers?**<\/p>\n * **Realistic Testing:** This is the biggest benefit. Mocking databases or message queues can be brittle and difficult to maintain. Testcontainers provides a service that behaves exactly like the real thing, leading to tests that more accurately reflect how your application will perform in production. **How Does It Work?**<\/p>\n 1. **Client Library:** Testcontainers provides client libraries for various programming languages (Java, Python, JavaScript, Ruby, Go, .NET, and more). **Example (Conceptual – Python):**<\/p>\n “`python # Create a PostgreSQL environment # Start the container # Connect to the database # Perform database operations in your test # Stop the container (cleanup) **Key Concepts:**<\/p>\n * **Environment:** A Testcontainers environment is a configuration that defines which containers to run and how they should be configured. **Resources to Learn More:**<\/p>\n * **Official Website:** [https:\/\/testcontainers.io\/](https:\/\/testcontainers.io\/) – This is the best place to start. **In short, Testcontainers is a powerful tool that dramatically improves the quality and reliability of your local tests by allowing you to test against real, running containerized services.**<\/p>\n Do you want me to delve deeper into a specific aspect of Testcontainers, such as:<\/p>\n * A specific language implementation (e.g., Python)? We can see that the answer provided by the LLM has some mistakes, for example, PostgreSQLEnvironment doesn\u2019t exist in testcontainers-python. Another one is the links to the docs, testcontainers.io doesn\u2019t exist. So, we can see some hallucinations in the answer.<\/p>\n Of course, LLM responses are non-deterministic, and since each model is trained until a certain cutoff date, the information may be outdated, and the answers might not be accurate.<\/p>\n To improve this situation, let\u2019s provide the model with some curated context about Testcontainers!<\/p>\n We\u2019ll create another controller, RagController<\/strong>, which will retrieve documents from a vector search database.<\/p>\n \tprivate final ChatClient chatClient;<\/p>\n \tprivate final VectorStore vectorStore;<\/p>\n \tpublic RagController(ChatModel chatModel, VectorStore vectorStore) { \t@GetMapping(“\/rag”) \tstatic ChatClient.CallResponseSpec callResponseSpec(ChatClient chatClient, VectorStore vectorStore, }<\/p>\n<\/div>\n Spring AI provides many advisors. In this example, we are going to use the QuestionAnswerAdvisor to perform the query against the vector search database. It takes care of all the individual integrations with the vector database.<\/p>\n First, we need to load the relevant documents into the vector database. Under src\/test\/java\/com\/example<\/strong>, let\u2019s create an IngestionConfiguration class:<\/p>\n \t@Value(“classpath:\/docs\/testcontainers.txt”) \t@Bean \t\t\tvar tokenTextSplitter = new TokenTextSplitter(); \t\t\tvectorStore.add(testcontainersDocuments); }<\/p>\n<\/div>\n testcontainers.txt under \/src\/test\/resources\/docs<\/strong> directory will have the following content\u00a0 specific information. For a real-world use case, you would probably have a more extensive collection of documents.<\/p>\n\n Testcontainers provides modules for a wide range of commonly used infrastructure dependencies including relational databases, NoSQL datastores, search engines, message brokers, etc. See https:\/\/testcontainers.com\/modules\/ for a complete list.<\/p>\n Technology-specific modules are a higher-level abstraction on top of GenericContainer which help configure and run these technologies without any boilerplate, and make it easy to access their relevant parameters.<\/p>\n Official website: https:\/\/testcontainers.com\/ Now, let\u2019s add an additional property to the src\/test\/resources\/application.properties<\/strong> file.<\/p>\n ai\/mxbai-embed-large<\/a> is an embedding model that will be used to create the embeddings of the documents. They will be stored in the vector search database, in our case, Qdrant. Spring AI will initialize the Qdrant schema and use the collection named test.<\/p>\n\n Let\u2019s update our TestDemoApplication<\/strong> Java class and add the IngestionConfiguration.class<\/p>\n\n \tpublic static void main(String[] args) { }<\/p>\n<\/div>\n Now we start our application by running .\/mvnw spring-boot:test-run <\/strong>or .\/gradlew bootTestRun<\/strong> and ask it again about Testcontainers:<\/p>\n\n This time, the answer contains references from the docs we have provided and is more accurate.<\/p>\n It provides modules for commonly used infrastructure dependencies such as relational databases, NoSQL datastores, search engines, and message brokers.<\/p>\n If you have any specific questions about how to use Testcontainers or its features, I’d be happy to help.<\/p>\n<\/div>\n Testing is a key part of software development. Fortunately, Testcontainers and Spring AI\u2019s utilities support testing of GenAI applications. So far, we\u2019ve been testing the application manually, starting the application and performing requests to the given endpoints, verifying the correctness of the response ourselves. Now, we\u2019re going to automate it by writing an integration test to check if the answer provided by the LLM is more contextual, augmented by the information provided in the documents.<\/p>\n\n \t@LocalServerPort \t@Autowired \t@Autowired \t@Test \t\tassertFactCheck(question, answer); \tprivate String retrieveAnswer(String question) { \tprivate void assertFactCheck(String question, String answer) { \tprivate List<Document> docs(String question) { }<\/p>\n<\/div>\n Importing ContainerConfiguration, Qdrant will be provided.<\/p>\n Importing IngestionConfiguration, will load the documents into the vector database.<\/p>\n We\u2019re going to use FactCheckingEvaluator<\/a> to tell the chat model (ai\/llama3.1) to check the answer provided by the LLM and verify it with the documents stored in the vector database.<\/p>\n Note: The integration test is using the same model we have declared in the previous steps. But we can definitely use a different model.<\/p>\n Automating your tests ensures consistency and reduces the risk of errors that often come with manual execution.\u00a0<\/p>\n Finally, let\u2019s introduce some observability into our application. By introducing metrics and tracing, we can understand if our application is behaving as designed during development and in production.<\/p>\n Add the following dependencies to the pom.xml<\/p>\n Now, let\u2019s create GrafanaContainerConfiguration <\/strong>under src\/test\/java\/com\/example.<\/p>\n \t@Bean }<\/p>\n<\/div>\n Grafana provides the grafana\/otel-lgtm<\/a> image, which will start Prometheus, Tempo, and OpenTelemetry Collector, and other related services, all combined into a single convenient Docker image.<\/p>\n For the sake of our demo, let\u2019s add a couple of properties at \/src\/test\/resources\/application.properties<\/strong> to sample 100% of requests.<\/p>\n\n Update the TestDemoApplication<\/strong> class to include GrafanaContainerConfiguration.class<\/strong><\/p>\n\n \tpublic static void main(String[] args) { }<\/p>\n<\/div>\n Now, run .\/mvnw spring-boot:test-run<\/strong> or .\/gradlew bootTestRun <\/strong>one more time, and perform a request.<\/p>\n Then, look for the following text in logs.<\/p>\n The port can be different for you, but clicking on it should open the Grafana dashboard. This is where you can query for metrics related to the model or vector search, and also see the traces.<\/p>\n Figure 1: Grafana dashboard showing model metrics, vector search performance, and traces<\/strong><\/p>\n We can also display the token usage metric used by the chat endpoint.<\/p>\n Figure 2: Grafana dashboard panel displaying token usage metrics for the chat endpoint<\/strong><\/p>\n List traces for the service with name \u201cdemo\u201d, and we can see a list of operations executed as part of this trace. You can use the trace ID with the name http get \/rag<\/strong> to see the full control flow within the same HTTP request.<\/p>\n Figure 3:\u00a0 Grafana dashboard showing trace details for a \/rag endpoint in a Java GenAI application<\/strong><\/p>\n\n Docker offers powerful capabilities that complement the Spring AI project, allowing developers to build GenAI applications efficiently with Docker tools that they know and trust. It simplifies the startup of service dependencies, including the Docker Model Runner, which exposes an OpenAI-compatible API for running local models. Testcontainers help to quickly spin out integration testing to evaluate your app by providing lightweight containers for your services and dependencies.\u00a0 From development to testing, Docker and Spring AI have proven to be a reliable and productive combination for building modern AI-driven applications.<\/p>\n\n Get an inside look at the design architecture of the Docker Model Runner<\/a>.\u00a0<\/p>\n Explore the story<\/a> behind our model distribution specification<\/p>\n Read our quickstart guide to Docker Model Runner<\/a>.<\/p>\n Find documentation for Model Runner<\/a>.<\/p>\nGetting started\u00a0<\/h2>\n
Configuring Docker Model Runner<\/h2>\n
\ndocker model pull ai\/mxbai-embed-large\n<\/div>\nBuilding the GenAI app<\/h2>\n
\npublic class ChatController {\n
\n\t\tthis.chatClient = ChatClient.builder(chatModel).build();
\n\t}<\/p>\n
\n\tpublic String generate(@RequestParam(value = “message”, defaultValue = “Tell me a joke”) String message) {
\n\t\treturn this.chatClient.prompt().user(message).call().content();
\n\t}<\/p>\n
\nspring.ai.openai.api-key=test
\nspring.ai.openai.chat.options.model=ai\/llama3.1\n<\/div>\n
\n* **Simplified Test Setup:** Forget about manually setting up and configuring databases or other services on your test machine. Testcontainers automatically handles the container creation, configuration, and cleanup for you.
\n* **Faster Tests:** Because the services are running locally, there\u2019s no network latency involved, resulting in significantly faster test execution times.
\n* **Consistent Environments:** You eliminate the “it works on my machine” problem. Everyone running the tests will be using the same, pre-configured environment.
\n* **Supports Many Services:** Testcontainers supports a huge range of services, including:
\n * **Databases:** PostgreSQL, MySQL, MongoDB, Redis, Cassandra, MariaDB
\n * **Message Queues:** RabbitMQ, Kafka, ActiveMQ
\n * **Web Servers:** Tomcat, Jetty, H2 (for in-memory databases)
\n * **And many more!** The list is constantly growing.<\/p>\n
\n2. **Container Run:** When you use the Testcontainers client library in your test, it automatically starts the specified container (e.g., a PostgreSQL database) in the background.
\n3. **Connection:** Your test code then connects to the running container using standard protocols (e.g., JDBC for PostgreSQL, HTTP for a web server).
\n4. **Test Execution:** You execute your tests as usual.
\n5. **Cleanup:** When the tests are finished, Testcontainers automatically shuts down the container, ensuring a clean state for the next test run.<\/p>\n
\nfrom testcontainers.postgresql import PostgreSQLEnvironment<\/p>\n
\nenv = PostgreSQLEnvironment()<\/p>\n
\nenv.start()<\/p>\n
\ndb = env.db() # This creates a connection object to the running PostgreSQL container<\/p>\n
\n# …<\/p>\n
\nenv.shutdown()
\n“`<\/p>\n
\n* **Container:** A running containerized service (e.g., a database instance).
\n* **Connection:** An object that represents a connection to a specific container.<\/p>\n
\n* **GitHub Repository:** [https:\/\/github.com\/testcontainers\/testcontainers](https:\/\/github.com\/testcontainers\/testcontainers) – See the source code and contribute.
\n* **Documentation:** [https:\/\/testcontainers.io\/docs\/](https:\/\/testcontainers.io\/docs\/) – Comprehensive documentation with examples for various languages.<\/p>\n
\n* A particular service it supports (e.g., PostgreSQL)?
\n* How to integrate it with a specific testing framework (e.g., JUnit, pytest)?<\/p>\n<\/div>\n
\npublic class RagController {\n
\n\t\tthis.chatClient = ChatClient.builder(chatModel).build();
\n\t\tthis.vectorStore = vectorStore;
\n\t}<\/p>\n
\n\tpublic String generate(@RequestParam(value = “message”, defaultValue = “What’s Testcontainers?”) String message) {
\n\t\treturn callResponseSpec(this.chatClient, this.vectorStore, message).content();
\n\t}<\/p>\n
\n\t\t\tString question) {
\n\t\tQuestionAnswerAdvisor questionAnswerAdvisor = QuestionAnswerAdvisor.builder(vectorStore)
\n\t\t\t.searchRequest(SearchRequest.builder().topK(1).build())
\n\t\t\t.build();
\n\t\treturn chatClient.prompt().advisors(questionAnswerAdvisor).user(question).call();
\n\t}<\/p>\nIngesting documents into the vector database<\/h2>\n
\npublic class IngestionConfiguration {\n
\n\tprivate Resource testcontainersDoc;<\/p>\n
\n\tApplicationRunner init(VectorStore vectorStore) {
\n\t\treturn args -> {
\n\t\t\tvar javaTextReader = new TextReader(this.testcontainersDoc);
\n\t\t\tjavaTextReader.getCustomMetadata().put(“language”, “java”);<\/p>\n
\n\t\t\tvar testcontainersDocuments = tokenTextSplitter.apply(javaTextReader.get());<\/p>\n
\n\t\t};
\n\t}<\/p>\n
\nGetting Started: https:\/\/testcontainers.com\/getting-started\/
\nModule Catalog: https:\/\/testcontainers.com\/modules\/<\/p>\n<\/div>\n
\nspring.ai.vectorstore.qdrant.initialize-schema=true
\nspring.ai.vectorstore.qdrant.collection-name=test\n<\/div>\n
\n\t\tSpringApplication.from(DemoApplication::main)
\n\t\t\t.with(TestcontainersConfiguration.class, IngestionConfiguration.class)
\n\t\t\t.run(args);
\n\t}<\/p>\nIntegration testing<\/h2>\n
\n\t\twebEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
\nclass RagControllerTest {\n
\n\tprivate int port;<\/p>\n
\n\tprivate VectorStore vectorStore;<\/p>\n
\n\tprivate ChatClient.Builder chatClientBuilder;<\/p>\n
\n\tvoid verifyTestcontainersAnswer() {
\n\t\tvar question = “Tell me about Testcontainers”;
\n\t\tvar answer = retrieveAnswer(question);<\/p>\n
\n\t}<\/p>\n
\n\t\tRestClient restClient = RestClient.builder().baseUrl(“http:\/\/localhost:%d”.formatted(this.port)).build();
\n\t\treturn restClient.get().uri(“\/rag?message={question}”, question).retrieve().body(String.class);
\n\t}<\/p>\n
\n\t\tFactCheckingEvaluator factCheckingEvaluator = new FactCheckingEvaluator(this.chatClientBuilder);
\n\t\tEvaluationResponse evaluate = factCheckingEvaluator.evaluate(new EvaluationRequest(docs(question), answer));
\n\t\tassertThat(evaluate.isPass()).isTrue();
\n\t}<\/p>\n
\n\t\tvar response = RagController
\n\t\t\t.callResponseSpec(this.chatClientBuilder.build(), this.vectorStore, question)
\n\t\t\t.chatResponse();
\n\t\treturn response.getMetadata().get(QuestionAnswerAdvisor.RETRIEVED_DOCUMENTS);
\n\t}<\/p>\nObservability with the Grafana LGTM Stack<\/h2>\n
\n\t<groupId>org.springframework.boot<\/groupId>
\n\t<artifactId>spring-boot-starter-actuator<\/artifactId>
\n<\/dependency>
\n<dependency>
\n\t<groupId>io.micrometer<\/groupId>
\n\t<artifactId>micrometer-registry-otlp<\/artifactId>
\n<\/dependency>
\n<dependency>
\n\t<groupId>io.micrometer<\/groupId>
\n\t<artifactId>micrometer-tracing-bridge-otel<\/artifactId>
\n<\/dependency>
\n<dependency>
\n\t<groupId>io.opentelemetry<\/groupId>
\n\t<artifactId>opentelemetry-exporter-otlp<\/artifactId>
\n<\/dependency>
\n<dependency>
\n\t<groupId>org.testcontainers<\/groupId>
\n\t<artifactId>grafana<\/artifactId>
\n <scope>test<\/scope>
\n<\/dependency>\n<\/div>\n
\npublic class GrafanaContainerConfiguration {\n
\n\t@ServiceConnection
\n\tLgtmStackContainer lgtmContainer() {
\n\t\treturn new LgtmStackContainer(“grafana\/otel-lgtm:0.11.4”);
\n\t}<\/p>\n
\nmanagement.tracing.sampling.probability=1\n<\/div>\n
\n\t\tSpringApplication.from(DemoApplication::main)
\n\t\t\t.with(TestcontainersConfiguration.class, IngestionConfiguration.class, GrafanaContainerConfiguration.class)
\n\t\t\t.run(args);
\n\t}<\/p>\n
\nhttp:\/\/localhost:64908\n<\/div>\nConclusion<\/h3>\n
Learn more<\/h3>\n