Hi, I\u2019m Philippe Charriere, a Principal Solutions Architect at Docker. I like to test new tools and see how they fit into real-world workflows. Recently, I set out to see if JetBrains\u2019 Koog framework could run with Docker Model Runner, and what started as a quick test turned into something a lot more interesting than I expected. In this new blog post, we\u2019ll explore how to create a small Koog agent specializing in ratatouille recipes using popular Docker AI tools (disclaimer: I\u2019m French). We\u2019ll be using:<\/p>\n
Koog<\/a>: a framework for building AI Agents in Kotlin<\/p>\n Docker Model Runner<\/a>: a Docker feature that allows deploying AI models locally, based on Llama.cpp<\/a><\/p>\n Agentic Compose<\/a>: a Docker Compose feature to easily integrate AI models into your applications<\/p>\n Docker MCP Gateway<\/a>: a gateway to access MCP (Model Context Protocol) servers from the Docker MCP Catalog<\/a>\u00a0<\/p>\n I use IntelliJ IDEA Community Edition<\/a> to initialize the Kotlin project.<\/p>\n I use OpenJDK 23 and Gradle Kotlin DSL for project configuration.<\/p>\n Here\u2019s my build configuration: build.gradle.kts<\/p>\n group = “kitchen.ratatouille” repositories { dependencies { }<\/p>\n application { tasks.test { tasks.jar { kotlin { The new \u201cagentic\u201d feature of Docker Compose allows defining the models to be used by Docker Compose services.<\/p>\n With the content below, I define that I will use the hf.co\/menlo\/lucy-128k-gguf:q4_k_m model from Hugging Face<\/a> for my \u201cKoog agent\u201d.<\/p>\n And I make the \u201clink\u201d between the koog-app service and the app_model model and the Koog agent as follows at the service level:<\/p>\n Docker Compose will automatically inject the MODEL_RUNNER_BASE_URL and MODEL_RUNNER_CHAT_MODEL environment variables into the koog-app service, which allows the Koog agent to connect to the model.<\/p>\n If you entered interactive mode in the koog-app container, you could verify that the environment variables are properly defined with the command:<\/p>\n And you would get something like:<\/p>\n It\u2019s entirely possible to define multiple models.<\/p>\n The complete compose.yaml file looks like this:<\/p>\n koog-app: models: Next, we\u2019ll need a Dockerfile to build the Docker image of our Koog application. The Dockerfile uses multi-stage build to optimize the final image size, so it\u2019s divided into two parts\/stages: one for building the application (build) and one for execution (runtime). Here\u2019s the content of the Dockerfile:<\/p>\n WORKDIR \/app<\/p>\n COPY gradlew . RUN chmod +x .\/gradlew<\/p>\n COPY src\/ src\/<\/p>\n # Build # Stage 2: Runtime WORKDIR \/app<\/p>\n COPY –from=build \/app\/build\/libs\/ratatouille-1.0-SNAPSHOT.jar app.jar Now, here\u2019s the source code of our application, in the src\/main\/kotlin\/Main.kt file to be able to use Docker Model Runner. The API exposed by Docker Model Runner is compatible with the OpenAI API, so we\u2019ll use Koog\u2019s OpenAI client to interact with our model:<\/p>\n import ai.koog.prompt.executor.clients.openai.OpenAIClientSettings suspend fun main() {<\/p>\n val apiKey = “nothing” val client = OpenAILLMClient( Creating an agent with Koog is relatively simple as you can see in the code below. We\u2019ll need:<\/p>\n a SingleLLMPromptExecutor that will use the OpenAI client we created previously to execute requests to the model.<\/p>\n an LLModel that will define the model we\u2019re going to use.<\/p>\n an AIAgent that will encapsulate the model and the prompt executor to execute requests.<\/p>\n Regarding the prompt, I use the SYSTEM_PROMPT environment variable to define the agent\u2019s system prompt, and AGENT_INPUT to define the agent\u2019s input (the \u201cuser message\u201d). These variables were defined in the compose.yaml file previously:<\/p>\n And here\u2019s the complete code of the Koog agent in the src\/main\/kotlin\/Main.kt file:<\/p>\n import ai.koog.agents.core.agent.AIAgent suspend fun main() {<\/p>\n val apiKey = “nothing” val client = OpenAILLMClient( val promptExecutor = SingleLLMPromptExecutor(client)<\/p>\n val llmModel = LLModel( val agent = AIAgent( val recipe = agent.run(System.getenv(“AGENT_INPUT”))<\/p>\n println(“Recipe:n $recipe”)<\/p>\n }<\/p>\n<\/div>\n All that\u2019s left is to launch the project with the following command:<\/p>\n Then wait a moment, depending on your machine, the build and completion times will be more or less long. I nevertheless chose Lucy 128k<\/a> because it can run on small configurations, even without a GPU. This model also has the advantage of being quite good at \u201cfunction calling\u201d detection despite its small size (however, it doesn\u2019t support parallel tool calls). And you should finally get something like this in the console:<\/p>\n —<\/p>\n ### **Ingredients** —<\/p>\n ### **Steps** 2. **Add Vegetables & Flavor** 3. **Slow Cook** 4. **Finish & Serve** —<\/p>\n ### **Tips** Enjoy your meal! <\/p>\n<\/div>\n As you can see, it\u2019s quite simple to create an agent with Koog and Docker Model Runner!\u00a0<\/p>\n But we have a problem, I told you I was French and the ratatouille recipe proposed by Lucy 128k<\/a> doesn\u2019t really suit me: there\u2019s no rabbit, chicken, or duck in a ratatouille!!!. But let\u2019s see how to fix that.<\/p>\n What I\u2019d like to do now is have my application first search for information about ratatouille ingredients, and then have the Koog agent use this information to improve the recipe. For this, I\u2019d like to use the DuckDuckGo MCP server that\u2019s available on the Docker MCP Hub<\/a>. And to make my life easier, I\u2019m going to use the Docker MCP Gateway<\/a> to access this MCP server.<\/p>\n To use the Docker MCP Gateway, I\u2019ll first modify the compose.yml file to add the gateway configuration.<\/p>\n Here\u2019s the configuration I added for the gateway in the compose.yaml file:<\/p>\n This configuration will create an mcp-gateway service that will listen on port 8811 and use the sse (Server-Sent Events) transport to communicate with MCP servers.<\/p>\n Important<\/strong>:<\/p>\n with \u2013servers=duckduckgo I can filter the available MCP servers to only use the DuckDuckGo server.<\/p>\n the MCP Gateway will automatically pull the available MCP servers from the Docker MCP Hub.<\/p>\n The MCP Gateway is an open-source project that you can find here<\/a>:\u00a0<\/p>\n Next, I\u2019ll modify the koog-app service so it can communicate with the gateway by adding the MCP_HOST environment variable that will point to the gateway URL, as well as the dependency on the mcp-gateway service:<\/p>\n I\u2019ll also modify the system prompt and user message:<\/p>\n So here\u2019s the complete compose.yml file with the MCP Gateway configuration and the modifications made to the koog-app service:<\/p>\n\n koog-app: mcp-gateway: models: Now, let\u2019s modify the Kotlin code to use the MCP Gateway and search for ratatouille ingredients.<\/p>\n The modification is extremely simple; you just need to:<\/p>\n define the MCP transport (SseClientTransport) with the gateway URL: val transport = McpToolRegistryProvider.defaultSseTransport(System.getenv(\u201cMCP_HOST\u201d))<\/p>\n create the MCP tools registry with the gateway: val toolRegistry = McpToolRegistryProvider.fromTransport(transport = transport, name = \u201csse-client\u201d, version = \u201c1.0.0\u201d)<\/p>\n and finally, add the tools registry to the Koog agent constructor: toolRegistry = toolRegistry<\/p>\n Extremely important<\/strong>: I added capabilities = listOf(LLMCapability.Completion, LLMCapability.Tools) for the LLM model, because we\u2019re going to use its \u201cfunction calling\u201d capabilities (the tools are defined and provided by the MCP server).<\/p>\n Here\u2019s the complete code of the Koog agent modified to use the MCP Gateway in the src\/main\/kotlin\/Main.kt file:<\/p>\n import ai.koog.agents.core.agent.AIAgent suspend fun main() {<\/p>\n val transport = McpToolRegistryProvider.defaultSseTransport(System.getenv(“MCP_HOST”)) val apiKey = “nothing” val client = OpenAILLMClient( val promptExecutor = SingleLLMPromptExecutor(client)<\/p>\n val llmModel = LLModel( val agent = AIAgent( val recipe = agent.run(System.getenv(“AGENT_INPUT”))<\/p>\n println(“Recipe:n $recipe”)<\/p>\n }<\/p>\n<\/div>\n Let\u2019s launch the project again with the command:<\/p>\n And after a while, you should get a new ratatouille recipe, but the LLM will have relied on the search results performed by the DuckDuckGo MCP server (via the MCP Gateway) to improve the recipe. The response time will be a bit longer because the LLM will first query the MCP server to get the ratatouille ingredients, then it will generate the recipe. And the DuckDuckGo MCP server will search for links and then retrieve the content of those links (indeed, the DuckDuckGo MCP server exposes 2 tools: search and fetch_content).<\/p>\n Here\u2019s an example of what you might get with an improved and more \u201cauthentic\u201d ratatouille recipe:<\/p>\n —<\/p>\n ### **What Is Ratatouille?** —<\/p>\n ### **Ingredients** (for 4 servings): —<\/p>\n ### **How to Make Ratatouille** 1. **Prepare the dish**: Coat a 1\u00bd-quart casserole dish with 1 tbsp olive oil. **Cook’s Note**: —<\/p>\n ### **How to Serve** —<\/p>\n<\/div>\n This blog post perfectly illustrates the modern containerized AI ecosystem that Docker is building. By combining Docker Model Runner<\/strong>, Agentic Compose<\/strong>, Docker MCP Gateway,<\/strong> and the Koog<\/strong> framework (but we could of course use other frameworks), we were able to create an \u201cintelligent\u201d agent quite simply.<\/p>\n Docker Model Runner<\/strong> allowed us to use an AI model locally.<\/p>\n Agentic Compose<\/strong> simplified the integration of the model into our application by automatically injecting the necessary environment variables.<\/p>\n The Docker MCP Gateway<\/strong> transformed our little agent into a system capable of interacting with the outside world.<\/p>\n The Koog<\/strong> framework allowed us to orchestrate these components in Kotlin.<\/p>\n Soon, I\u2019ll go deeper into the MCP Gateway and how to use it with your own MCP servers, and not just with Koog. And I continue my explorations with Koog and Docker Model Runner. Check out the entire source code of this project is available here<\/a>\u00a0<\/p>\n If you need more GPUs to experiment with different models, sign up for Docker Offload beta program<\/a> and get 300 minutes for free.\u00a0<\/p>\n Discover hundreds of curated MCP servers on the Docker MCP Catalog<\/a><\/p>\n Learn more about Docker MCP Toolkit<\/a><\/p>\n Explore Docker MCP Gateway<\/a> on GitHub<\/p>\n Get started with Docker Model Runner<\/a><\/p>\nPrerequisites: Kotlin project initialization<\/h2>\n
Step 1: Gradle Configuration<\/h3>\n
\n kotlin(“jvm”) version “2.1.21”
\n application
\n}\n
\nversion = “1.0-SNAPSHOT”<\/p>\n
\n mavenCentral()
\n}<\/p>\n
\n testImplementation(kotlin(“test”))
\n implementation(“ai.koog:koog-agents:0.3.0”)
\n implementation(“org.slf4j:slf4j-simple:2.0.9”)<\/p>\n
\n mainClass.set(“kitchen.ratatouille.MainKt”)
\n}<\/p>\n
\n useJUnitPlatform()
\n}<\/p>\n
\n duplicatesStrategy = DuplicatesStrategy.EXCLUDE
\n manifest {
\n attributes(“Main-Class” to “kitchen.ratatouille.MainKt”)
\n }
\n from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) })
\n}<\/p>\n
\n jvmToolchain(23)
\n}<\/p>\n<\/div>\nStep 2: Docker Compose Project Configuration<\/h3>\n
\n app_model:
\n model: hf.co\/menlo\/lucy-128k-gguf:q4_k_m\n<\/div>\n
\n app_model:
\n endpoint_var: MODEL_RUNNER_BASE_URL
\n model_var: MODEL_RUNNER_CHAT_MODEL\n<\/div>\n
\nMODEL_RUNNER_CHAT_MODEL=hf.co\/menlo\/lucy-128k-gguf:q4_k_m\n<\/div>\n
\n build:
\n context: .
\n dockerfile: Dockerfile
\n environment:
\n SYSTEM_PROMPT: You are a helpful cooking assistant.
\n AGENT_INPUT: How to cook a ratatouille?
\n models:
\n app_model:
\n endpoint_var: MODEL_RUNNER_BASE_URL
\n model_var: MODEL_RUNNER_CHAT_MODEL<\/p>\n
\n app_model:
\n model: hf.co\/menlo\/lucy-128k-gguf:q4_k_m<\/p>\n<\/div>\nStep 3: Dockerfile<\/h3>\n
\nFROM eclipse-temurin:23-jdk-noble AS build\n
\nCOPY gradle\/ gradle\/
\nCOPY build.gradle.kts .
\nCOPY settings.gradle.kts .<\/p>\n
\nRUN .\/gradlew clean build<\/p>\n
\nFROM eclipse-temurin:23-jre-noble AS runtime<\/p>\n
\nCMD [“java”, “-jar”, “app.jar”]<\/p>\n<\/div>\nStep 4: Kotlin side:<\/h3>\n
Connecting to Docker Model Runner<\/h4>\n
\nimport ai.koog.prompt.executor.clients.openai.OpenAILLMClient<\/p>\n
\n val customEndpoint = System.getenv(“MODEL_RUNNER_BASE_URL”).removeSuffix(“\/”)
\n val model = System.getenv(“MODEL_RUNNER_CHAT_MODEL”)<\/p>\n
\n apiKey=apiKey,
\n settings = OpenAIClientSettings(customEndpoint)
\n )
\n}<\/p>\n<\/div>\nFirst Koog Agent<\/h4>\n
\n SYSTEM_PROMPT: You are a helpful cooking assistant.
\n AGENT_INPUT: How to cook a ratatouille?\n<\/div>\n
\nimport ai.koog.prompt.executor.clients.openai.OpenAIClientSettings
\nimport ai.koog.prompt.executor.clients.openai.OpenAILLMClient
\nimport ai.koog.prompt.executor.llms.SingleLLMPromptExecutor
\nimport ai.koog.prompt.llm.LLMCapability
\nimport ai.koog.prompt.llm.LLMProvider
\nimport ai.koog.prompt.llm.LLModel<\/p>\n
\n val customEndpoint = System.getenv(“MODEL_RUNNER_BASE_URL”).removeSuffix(“\/”)
\n val model = System.getenv(“MODEL_RUNNER_CHAT_MODEL”)<\/p>\n
\n apiKey=apiKey,
\n settings = OpenAIClientSettings(customEndpoint)
\n )<\/p>\n
\n provider = LLMProvider.OpenAI,
\n id = model,
\n capabilities = listOf(LLMCapability.Completion)
\n )<\/p>\n
\n executor = promptExecutor,
\n systemPrompt = System.getenv(“SYSTEM_PROMPT”),
\n llmModel = llmModel,
\n temperature = 0.0
\n )<\/p>\nRunning the project<\/h2>\n
\n Sure! Here’s a step-by-step guide to cooking a classic ratatouille:\n
\n– 2 boneless chicken thighs or 1-2 lbs rabbit (chicken is common, but rabbit is traditional)
\n– 1 small onion (diced)
\n– 2 garlic cloves (minced)
\n– 1 cup tomatoes (diced)
\n– 1 zucchini (sliced)
\n– 1 yellow squash or eggplant (sliced)
\n– 1 bell pepper (sliced)
\n– 2 medium potatoes (chopped)
\n– 1 red onion (minced)
\n– 2 tbsp olive oil
\n– 1 tbsp thyme (or rosemary)
\n– Salt and pepper (to taste)
\n– Optional: 1\/4 cup wine (white or red) to deglaze the pan <\/p>\n
\n1. **Prep the Ingredients**
\n – Dice the onion, garlic, tomatoes, zucchini, squash, bell pepper, potatoes.
\n – Saut\u00e9 the chicken in olive oil until browned (about 10\u201315 minutes).
\n – Add the onion and garlic, saut\u00e9 for 2\u20133 minutes. <\/p>\n
\n – Pour in the tomatoes, zucchini, squash, bell pepper, red onion, and potatoes.
\n – Add thyme, salt, pepper, and wine (if using). Stir to combine.
\n – Add about 1 cup water or stock to fill the pot, if needed. <\/p>\n
\n – Place the pot in a large pot of simmering water (or use a Dutch oven) and cook on low heat (around 200\u00b0F\/90\u00b0C) for about 30\u201340 minutes, or until the chicken is tender.
\n – Alternatively, use a stovetop pot with a lid to cook the meat and vegetables together, simmering until the meat is cooked through. <\/p>\n
\n – Remove the pot from heat and let it rest for 10\u201315 minutes to allow flavors to meld.
\n – Stir in fresh herbs (like rosemary or parsley) if desired.
\n – Serve warm with crusty bread or on the plate as is. <\/p>\n
\n– **Meat Variations**: Use duck or other meats if you don’t have chicken.
\n– **Vegetables**: Feel free to swap out any vegetables (e.g., mushrooms, leeks).
\n– **Liquid**: If the mixture is too dry, add a splash of water or stock.
\n– **Serving**: Ratatouille is often eaten with bread, so enjoy it with a side of crusty bread or a simple salad. <\/p>\nLet\u2019s add superpowers to our Koog agent with the Docker MCP Gateway<\/h2>\n
Configuring the Docker MCP Gateway in Docker Compose<\/h3>\n
Configuring the gateway in the compose.yaml file<\/h4>\n
\n image: docker\/mcp-gateway:latest
\n command:
\n – –port=8811
\n – –transport=sse
\n – –servers=duckduckgo
\n – –verbose
\n volumes:
\n – \/var\/run\/docker.sock:\/var\/run\/docker.sock\n<\/div>\n
\n MCP_HOST: http:\/\/mcp-gateway:8811\/sse
\n depends_on:
\n – mcp-gateway\n<\/div>\n
\n SYSTEM_PROMPT: |
\n You are a helpful cooking assistant.
\n Your job is to understand the user prompt and decide if you need to use tools to run external commands.
\n AGENT_INPUT: |
\n Search for the ingredients to cook a ratatouille, max result 1
\n Then, from these found ingredients, generate a yummy ratatouille recipe
\n Do it only once\n<\/div>\n
\n build:
\n context: .
\n dockerfile: Dockerfile
\n environment:
\n SYSTEM_PROMPT: |
\n You are a helpful cooking assistant.
\n Your job is to understand the user prompt and decide if you need to use tools to run external commands.
\n AGENT_INPUT: |
\n Search for the ingredients to cook a ratatouille, max result 1
\n Then, from these found ingredients, generate a yummy ratatouille recipe
\n Do it only once
\n MCP_HOST: http:\/\/mcp-gateway:8811\/sse
\n depends_on:
\n – mcp-gateway
\n models:
\n app_model:
\n # NOTE: populate the environment variables with the model runner endpoint and model name
\n endpoint_var: MODEL_RUNNER_BASE_URL
\n model_var: MODEL_RUNNER_CHAT_MODEL<\/p>\n
\n image: docker\/mcp-gateway:latest
\n command:
\n – –port=8811
\n – –transport=sse
\n – –servers=duckduckgo
\n – –verbose
\n volumes:
\n – \/var\/run\/docker.sock:\/var\/run\/docker.sock<\/p>\n
\n app_model:
\n model: hf.co\/menlo\/lucy-128k-gguf:q4_k_m\n<\/p><\/div>\nModifying the Kotlin code to use the MCP Gateway<\/h3>\n
\nimport ai.koog.agents.mcp.McpToolRegistryProvider
\nimport ai.koog.prompt.executor.clients.openai.OpenAIClientSettings
\nimport ai.koog.prompt.executor.clients.openai.OpenAILLMClient
\nimport ai.koog.prompt.executor.llms.SingleLLMPromptExecutor
\nimport ai.koog.prompt.llm.LLMCapability
\nimport ai.koog.prompt.llm.LLMProvider
\nimport ai.koog.prompt.llm.LLModel<\/p>\n
\n \/\/ Create a tool registry with tools from the MCP server
\n val toolRegistry = McpToolRegistryProvider.fromTransport(
\n transport = transport,
\n name = “sse-client”,
\n version = “1.0.0”
\n )
\n println(toolRegistry.tools)<\/p>\n
\n val customEndpoint = System.getenv(“MODEL_RUNNER_BASE_URL”).removeSuffix(“\/”)
\n val model = System.getenv(“MODEL_RUNNER_CHAT_MODEL”)<\/p>\n
\n apiKey=apiKey,
\n settings = OpenAIClientSettings(customEndpoint)
\n )<\/p>\n
\n provider = LLMProvider.OpenAI,
\n id = model,
\n capabilities = listOf(LLMCapability.Completion, LLMCapability.Tools)
\n )<\/p>\n
\n executor = promptExecutor,
\n systemPrompt = System.getenv(“SYSTEM_PROMPT”),
\n llmModel = llmModel,
\n temperature = 0.0,
\n toolRegistry = toolRegistry
\n )<\/p>\nLaunching the project with the MCP Gateway<\/h3>\n
\n Here’s a **complete and easy-to-follow version** of **Ratatouille**, based on the recipe you provided, with tips and variations to suit your preferences:\n
\nA classic French vegetable stew, traditionally made with eggplant, tomatoes, zucchini, bell peppers, onions, and mushrooms. It’s often seasoned with herbs like parsley, thyme, or basil and paired with crusty bread or a light sauce.<\/p>\n
\n– **1\/2 cup olive oil** (divided)
\n– **2 tbsp olive oil** (for the skillet)
\n– **3 cloves garlic**, minced
\n– **1 eggplant**, cubed
\n– **2 zucchinis**, sliced
\n– **2 large tomatoes**, chopped
\n– **2 cups fresh mushrooms**, sliced
\n– **1 large onion**, sliced
\n– **1 green or red bell pepper**, sliced
\n– **1\/2 tsp dried parsley**
\n– **Salt to taste**
\n– **1\/2 cup grated Parmesan cheese** (or pecorino, as you mentioned) <\/p>\n
\n**Preheat oven** to 350\u00b0F (175\u00b0C). <\/p>\n
\n2. **Cook the base**: In a skillet, saut\u00e9 garlic until fragrant (about 1\u20132 minutes). Add eggplant, parsley, and salt; cook for 10 minutes until tender.
\n3. **Layer the vegetables**: Spread the eggplant mixture in the dish, then add zucchini, tomatoes, mushrooms, onion, and bell pepper. Top with Parmesan.
\n4. **Bake**: Cover and bake for 45 minutes. Check for tenderness; adjust time if needed. <\/p>\n
\n– Add mushrooms (optional) or omit for a traditional flavor.
\n– Use fresh herbs like thyme or basil if preferred.
\n– Substitute zucchini with yellow squash or yellow bell pepper for color. <\/p>\n
\n– **Main dish**: Serve with crusty French bread or rice.
\n– **Side**: Pair with grilled chicken or fish.
\n– **Guilt-free twist**: Add black olives or a sprinkle of basil\/others for a lighter version. <\/p>\nConclusion<\/h3>\n
Learn more<\/h3>\n