{"id":3238,"date":"2026-01-15T14:11:39","date_gmt":"2026-01-15T14:11:39","guid":{"rendered":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2026\/01\/15\/permission-aware-rag-end-to-end-testing-with-the-spicedb-testcontainer\/"},"modified":"2026-01-15T14:11:39","modified_gmt":"2026-01-15T14:11:39","slug":"permission-aware-rag-end-to-end-testing-with-the-spicedb-testcontainer","status":"publish","type":"post","link":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2026\/01\/15\/permission-aware-rag-end-to-end-testing-with-the-spicedb-testcontainer\/","title":{"rendered":"Permission-Aware RAG: End-to-End Testing with the SpiceDB Testcontainer"},"content":{"rendered":"<p>We use GenAI in every facet of technology now \u2013 internal knowledge bases, customer support systems, and code review bots, to name just a few use cases. And in nearly every one of these, someone eventually asks:<\/p>\n<p><span class=\"text-highlight\"><em>What stops the model from returning something the user shouldn\u2019t see?<\/em>\u201d<\/span><\/p>\n<p>This is a roadblock that companies building RAG features or AI Agents eventually hit \u2013 the moment where an LLM returns data from a document that the user was not authorized to access, introducing potential legal, financial, and reputational risk to all parties. Unfortunately, traditional methods of authorization are not suited for the hierarchical, dynamic nature of access control in RAG. This is exactly where modern authorization <s>permissioning<\/s> systems such as <strong>SpiceDB<\/strong> shine: in building fine-grained authorization for filtering content in your AI-powered applications.<\/p>\n<p>In fact, <a href=\"https:\/\/authzed.com\/customers\/openai\" rel=\"nofollow noopener\" target=\"_blank\">OpenAI uses SpiceDB<\/a> to secure 37 Billion documents for 5 Million users who use ChatGPT Connectors \u2013 a feature where you bring your data from different sources such as Google Drive, Dropbox, GitHub etc. into ChatGPT.<\/p>\n<p>This blog post shows how you can pair SpiceDB with <a href=\"https:\/\/testcontainers.com\/\" rel=\"nofollow noopener\" target=\"_blank\"><strong>Testcontainers<\/strong><\/a> to give you the ability to test your permission logic inside your RAG pipeline, end-to-end, automatically, with zero infrastructure dependencies.The example repo can be <a href=\"https:\/\/github.com\/sohanmaheshwar\/spicedb-testcontainer-rag\" rel=\"nofollow noopener\" target=\"_blank\">found here<\/a><em>.<\/em><\/p>\n<h2 class=\"wp-block-heading\">Quick Primer on Authorization<\/h2>\n<p>Before diving into implementation, let\u2019s clarify two foundational concepts: <strong>Authentication<\/strong> (verifying <em>who<\/em> a user is) and <strong>Authorization<\/strong> (deciding <em>what<\/em> they can access).<\/p>\n<p>Authorization is commonly implemented via techniques such as:<\/p>\n<ul class=\"wp-block-list\">\n<li>Access Control Lists (ACLs)<\/li>\n<li>Role-Based Access Control (RBAC)<\/li>\n<li>Attribute-Based Access Control (ABAC)<\/li>\n<\/ul>\n<p>However, for complex, dynamic, and context-rich applications like RAG pipelines, traditional methods such as RBAC or ABAC fall short. The new kid on the block \u2013 <strong>ReBAC<\/strong> (Relationship-Based Access Control) is ideal as it models <a href=\"https:\/\/authzed.com\/blog\/check-it-out\" rel=\"nofollow noopener\" target=\"_blank\">access as a graph of relationships<\/a> rather than fixed rules, providing the necessary flexibility and scalability required.<\/p>\n<p>ReBAC was popularized in <a href=\"https:\/\/zanzibar.tech\/\" rel=\"nofollow noopener\" target=\"_blank\">Google Zanzibar<\/a>, the internal authorization system Google built to manage permissions across all its products (e.g., Google Docs, Drive). Zanzibar systems are optimized for low-latency, high-throughput authorization checks, and global consistency \u2013 requirements that are well-suited for RAG systems.<\/p>\n<p><a href=\"https:\/\/github.com\/authzed\/spicedb\/\" rel=\"nofollow noopener\" target=\"_blank\"><strong>SpiceDB<\/strong><\/a> is the most scalable open-source implementation of Google\u2019s Zanzibar authorization model. It stores access as a relationship graph, where the fundamental check reduces to:\u00a0<\/p>\n<p><em><span class=\"text-highlight\">Is this <\/span><\/em><strong><span class=\"text-highlight\"><em>actor<\/em><\/span><\/strong><em><span class=\"text-highlight\"> allowed to perform this <\/span><\/em><strong><span class=\"text-highlight\"><em>action<\/em><\/span><\/strong><em><span class=\"text-highlight\"> on this <\/span><\/em><strong><span class=\"text-highlight\"><em>resource<\/em><\/span><\/strong><em><span class=\"text-highlight\">?<\/span><\/em><\/p>\n<p>For a Google Docs-style example:<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: plain; gutter: false; title: ; notranslate\">\ndefinition user {}\ndefinition document {\n  relation reader: user\n  relation writer: user\n\n  permission read = reader + writer\n  permission write = writer\n}\n\n<\/pre>\n<\/div>\n<p>This schema defines object types (<code>user<\/code> and <code>document<\/code>), explicit <strong>Relations<\/strong> between the objects (<code>reader<\/code>, <code>writer<\/code>), and derived <strong>Permissions<\/strong> (<code>read<\/code>, <code>write<\/code>). SpiceDB evaluates the relationship graph in microseconds, enabling real-time authorization checks at massive scale.<\/p>\n<h2 class=\"wp-block-heading\">Access Control for RAG\u00a0<\/h2>\n<p>RAG (Retrieval-Augmented Generation) is an architectural pattern that enhances Large Language Models (LLMs) by letting them consult an external knowledge base, typically involving a <strong>Retriever<\/strong> component finding document chunks and the LLM generating an informed response.<\/p>\n<p>This pattern is now used by businesses and enterprises for apps like chatbots that query sensitive data such as customer playbooks or PII \u2013 all stored in a vector database for performance. However, the fundamental risk in this flow is <strong>data leakage<\/strong>: the Retriever component ignores permissions, and the LLM will happily summarize unauthorized data. In fact, OWASP has a <a href=\"https:\/\/owasp.org\/www-project-top-10-for-large-language-model-applications\/\" rel=\"nofollow noopener\" target=\"_blank\">Top 10 Risks for Large Language Model Applications<\/a> list which includes Sensitive Information Disclosure, Excessive Agency &amp; Vector and Embedding Weaknesses. The consequences of this leakage can be severe, ranging from loss of customer trust to massive financial and reputational damage from compliance violations.<\/p>\n<p>This setup desperately needs fine-grained authorization, and that\u2019s where <strong>SpiceDB<\/strong> comes in. SpiceDB can post-filter retrieved documents by performing real-time authorization checks, ensuring the model only uses data the querying user is permitted to see. The only requirement is that the documents have <strong>metadata<\/strong> that indicates where the information came from.But testing this critical permission logic without mocks, manual Docker setup, or flaky Continuous Integration (CI) environments is tricky. <strong>Testcontainers<\/strong> provides the perfect solution, allowing you to spin up a real, production-grade, and disposable SpiceDB instance inside your unit tests to deterministically verify that your RAG pipeline respects permissions end-to-end.<\/p>\n<h2 class=\"wp-block-heading\">Spin Up Real Authorization for Every Test<\/h2>\n<p>Instead of mocking your authorization system or manually running it on your workstation, you can add this line of code in your test:<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: go; gutter: false; title: ; notranslate\">\ncontainer, _ := spicedbcontainer.Run(ctx, \"authzed\/spicedb:v1.47.1\")\n<\/pre>\n<\/div>\n<p>And Testcontainers will:<\/p>\n<ul class=\"wp-block-list\">\n<li>Pull the real SpiceDB image<\/li>\n<li>Start it in a clean, isolated environment<\/li>\n<li>Assign it dynamic ports<\/li>\n<li>Wait for it to be ready<\/li>\n<li>Hand you the gRPC endpoint<\/li>\n<li>Clean up afterwards<\/li>\n<\/ul>\n<p>Because Testcontainers handles the full lifecycle \u2013 from pulling the container, exposing dynamic ports, and tearing it down automatically, you eliminate manual processes such as running Docker commands, and writing cleanup scripts. This isolation ensures that every single test runs with a fresh, clean authorization graph, preventing data conflicts, and making your permission tests <strong>completely reproducible<\/strong> in your IDE and across parallel Continuous Integration (CI) builds.<\/p>\n<p>Suddenly you have a real, production-grade, Zanzibar-style permissions engine inside your unit test.\u00a0<\/p>\n<h2 class=\"wp-block-heading\">Using SpiceDB &amp; Testcontainers<\/h2>\n<p>Here\u2019s a walkthrough of how you can achieve end-to-end permissions testing using SpiceDB and Testcontainers.<\/p>\n<h3 class=\"wp-block-heading\">1. Testing Our RAG\u00a0<\/h3>\n<ol class=\"wp-block-list\">\n<li>\n<\/li>\n<\/ol>\n<p>For the sake of simplicity, we have a minimal RAG and the retrieval mechanism is trivial too.\u00a0<\/p>\n<p>We\u2019re going to test three documents which have doc_ids (<code>doc1<\/code> <code>doc2<\/code> ..) that act as metadata.\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li><code>doc1<\/code>: Internal roadmap<\/li>\n<li><code>doc2<\/code>: Customer playbook<\/li>\n<li><code>doc3<\/code>: Public FAQ<\/li>\n<\/ul>\n<p>And three users:<\/p>\n<ul class=\"wp-block-list\">\n<li>Emilia owns <code>doc1<\/code><\/li>\n<li>Beatrice can view <code>doc2<\/code><\/li>\n<li>Charlie (or anyone) can view <code>doc3<\/code><\/li>\n<\/ul>\n<p>This SpiceDB schema defines a <code>user<\/code> and a <code>document<\/code> object type. A user has <code>read<\/code> permission on a document if they are the direct <code>viewer<\/code> or the <code>owner<\/code> of the document.<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: plain; gutter: false; title: ; notranslate\">\ndefinition user {}\n\ndefinition document {\n  relation owner: user\n  relation viewer: user | owner\n  permission read = owner + viewer\n}\n<\/pre>\n<\/div>\n<h3 class=\"wp-block-heading\">2. Starting the Testcontainer\u00a0<\/h3>\n<p>Here\u2019s how a line of code can start a test to launch the disposable SpiceDB instance:<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: go; gutter: false; title: ; notranslate\">\ncontainer, err := spicedbcontainer.Run(ctx, \"authzed\/spicedb:v1.47.1\")\nrequire.NoError(t, err)\n<\/pre>\n<\/div>\n<p>Next, we connect to the running containerized service:<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: go; gutter: false; title: ; notranslate\">\nhost, _ := container.Host(ctx)\nport, _ := container.MappedPort(ctx, \"50051\/tcp\")\nendpoint := fmt.Sprintf(\"%s:%s\", host, port.Port())\n\nclient, err := authzed.NewClient(\n    endpoint,\n    grpc.WithTransportCredentials(insecure.NewCredentials()),\n    grpcutil.WithInsecureBearerToken(\"somepresharedkey\"),\n)\n\n<\/pre>\n<\/div>\n<p>This is now a fully-functional SpiceDB instance running inside your test runner.<\/p>\n<h3 class=\"wp-block-heading\">3. Load the Schema + Test Data<\/h3>\n<p>The test seeds data the same way your application would:<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: go; gutter: false; title: ; notranslate\">\n_, err := client.WriteSchema(ctx, &amp;apiv1.WriteSchemaRequest{Schema: schema})\nrequire.NoError(t, err)\n<\/pre>\n<\/div>\n<p>Then:<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: go; gutter: false; title: ; notranslate\">\nrel(\"document\", \"doc1\", \"owner\", \"user\", \"emilia\")\nrel(\"document\", \"doc2\", \"viewer\", \"user\", \"beatrice\")\nrel(\"document\", \"doc3\", \"viewer\", \"user\", \"emilia\")\nrel(\"document\", \"doc3\", \"viewer\", \"user\", \"beatrice\")\nrel(\"document\", \"doc3\", \"viewer\", \"user\", \"charlie\")\n<\/pre>\n<\/div>\n<p>We now have a predictable, reproducible authorization graph for every test run.<\/p>\n<h3 class=\"wp-block-heading\">4. Post-Filtering With SpiceDB<\/h3>\n<p>Before the LLM sees anything, we check permissions with SpiceDB which acts as the source of truth of the permissions in the documents.<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: go; gutter: false; title: ; notranslate\">\nresp, err := r.spiceClient.CheckPermission(ctx, &amp;apiv1.CheckPermissionRequest{\n    Resource:   docObject,\n    Permission: \"read\",\n    Subject:    userSubject,\n})\n<\/pre>\n<\/div>\n<p>If SpiceDB says <strong>no<\/strong>, the doc is never fed into the LLM, thereby ensuring the user gets an answer to their query only based on what they have permissions to read.<\/p>\n<p>This avoids:<\/p>\n<ul class=\"wp-block-list\">\n<li>Accidental data leakage<\/li>\n<li>Overly permissive vector search<\/li>\n<li>Compliance problems<\/li>\n<\/ul>\n<p>Traditional access controls break down when data becomes embeddings hence having guardrails prevents this from happening.\u00a0<\/p>\n<h2 class=\"wp-block-heading\">End-to-End Permission Checks in a Single Test<\/h2>\n<p>Here\u2019s what the full test asserts:<\/p>\n<p><strong>Emilia queries \u201croadmap\u201d \u2192 gets <code>doc1<\/code><br \/> <\/strong>Because they\u2019re the owner.<\/p>\n<p><strong>Beatrice queries \u201cplaybook\u201d \u2192 gets <code>doc2<\/code><br \/> <\/strong>Because she\u2019s a viewer.<\/p>\n<p><strong>Charlie queries \u201cpublic\u201d \u2192 gets <code>doc3<\/code><br \/> <\/strong>Because it\u2019s the only doc he can read, as it\u2019s a public doc<\/p>\n<p>If there is a single failing permission rule, the end-to-end test will immediately fail, which is critical given the constant changes in RAG pipelines (such as new retrieval modes, embeddings, document types, or permission rules).\u00a0<\/p>\n<h2 class=\"wp-block-heading\">What If Your RAG Pipeline Isn\u2019t in Go?<\/h2>\n<p>First, a shoutout to <a href=\"https:\/\/github.com\/Mariscal6\/\" rel=\"nofollow noopener\" target=\"_blank\">Guillermo Mariscal<\/a> for his original contribution to the <a href=\"https:\/\/github.com\/Mariscal6\/testcontainers-spicedb-go\/tree\/main\/testdata\" rel=\"nofollow noopener\" target=\"_blank\">SpiceDB Go Testcontainers module<\/a>.\u00a0<\/p>\n<p>What if your RAG pipeline is written in a different language such as Python? Not to worry, there\u2019s also a community Testcontainers module written in Python that you can use similarly. The module can be <a href=\"https:\/\/pypi.org\/project\/testcontainers-spicedb\/\" rel=\"nofollow noopener\" target=\"_blank\">found here<\/a>.<\/p>\n<p>Typically, you would integrate it in your integration tests like this:<\/p>\n<div class=\"wp-block-syntaxhighlighter-code \">\n<pre class=\"brush: python; gutter: false; title: ; notranslate\">\n # Your RAG pipeline test\n  def test_rag_pipeline_respects_permissions():\n      with SpiceDBContainer() as spicedb:\n          # Set up permissions schema\n          client = create_spicedb_client(\n              spicedb.get_endpoint(),\n              spicedb.get_secret_key()\n          )\n\n          # Load your permissions model\n          client.WriteSchema(your_document_permission_schema)\n\n          # Write test relationships\n          # User A can access Doc 1\n          # User B can access Doc 2\n\n          # Test RAG pipeline with User A\n          results = rag_pipeline.search(query=\"...\", user=\"A\")\n          assert \"Doc 1\" in results\n          assert \"Doc 2\" not in results  # Should be filtered out!\n\n<\/pre>\n<\/div>\n<p>Similar to the Go module, this container gives you a clean, isolated SpiceDB instance for every test run.<\/p>\n<h2 class=\"wp-block-heading\">Why This Approach Matters<\/h2>\n<p>Authorization testing in RAG pipelines can be tricky, given the scale and latency requirement and it can get trickier in systems handling sensitive data. By integrating the flexibility and scale of SpiceDB with the automated, isolated environments of Testcontainers, you shift to a completely reliable, <strong>deterministic<\/strong> approach to authorization.\u00a0<\/p>\n<p>Every time your code ships, a fresh, production-grade authorization engine is spun up, loaded with test data, and torn down cleanly, guaranteeing <strong>zero drift between your development machine and CI<\/strong>. This pattern can ensure that your RAG system is safe, correct, and permission-aware as it scales from three documents to millions.<\/p>\n<h2 class=\"wp-block-heading\">Try It Yourself<\/h2>\n<p>The complete working example in Go along with a sample RAG pipeline is here:<br \/><a href=\"https:\/\/github.com\/sohanmaheshwar\/spicedb-testcontainer-rag\" rel=\"nofollow noopener\" target=\"_blank\"><strong>https:\/\/github.com\/sohanmaheshwar\/spicedb-testcontainer-rag<\/strong><\/a><br \/>Clone it.<br \/>Run <code>go test -v<\/code>.<br \/>Watch it spin up a fresh SpiceDB instance, load permissions, and assert RAG behavior.<br \/>Also, find the community modules for the SpiceDB testcontainer in <a href=\"https:\/\/testcontainers.com\/modules\/spicedb\/?language=go\" rel=\"nofollow noopener\" target=\"_blank\">Go<\/a> and <a href=\"https:\/\/testcontainers.com\/modules\/spicedb\/?language=python\" rel=\"nofollow noopener\" target=\"_blank\">Python<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>We use GenAI in every facet of technology now \u2013 internal knowledge bases, customer support systems, and code review bots, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":94,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[4],"tags":[],"class_list":["post-3238","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-docker"],"_links":{"self":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/3238","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/comments?post=3238"}],"version-history":[{"count":0,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/3238\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media\/94"}],"wp:attachment":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media?parent=3238"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/categories?post=3238"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/tags?post=3238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}