{"id":492,"date":"2026-09-24T10:15:00","date_gmt":"2026-09-24T08:15:00","guid":{"rendered":"https:\/\/www.lukaswojcik.com\/?p=492"},"modified":"2026-09-10T10:52:48","modified_gmt":"2026-09-10T08:52:48","slug":"building-a-rag-knowledge-base-with-google-cloud-vertex-ai-bigquery","status":"publish","type":"post","link":"https:\/\/www.lukaswojcik.com\/blog\/en\/cloud-ai\/tutorials-en-cloud-ai\/building-a-rag-knowledge-base-with-google-cloud-vertex-ai-bigquery\/","title":{"rendered":"Building a RAG Knowledge Base with Google Cloud Vertex AI & BigQuery"},"content":{"rendered":"\r\n
Standard Large Language Models (LLMs) frequently suffer from hallucinations and outdated knowledge cutoffs when queried about internal corporate manuals, proprietary databases, or highly specialized technical documentation. Retrieval-Augmented Generation (RAG) overcomes these limitations by anchoring the generation process in a verified, enterprise-specific knowledge repository. Implementing a serverless RAG architecture using Google Cloud Vertex AI Embeddings and BigQuery Vector Search enables scalable semantic search across millions of document chunks without managing dedicated vector database infrastructure.<\/p>\r\n\r\n\r\n\r\n
1. Architectural Framework: Serverless RAG on Google Cloud<\/h2>\r\n\r\n\r\n\r\n
An enterprise-grade RAG pipeline on Google Cloud decouples document processing, semantic indexing, and answer generation across three managed services:<\/p>\r\n\r\n\r\n\r\n
\r\n
Data Ingestion & Chunking:<\/strong> Raw PDF manuals, intranet wikis, and structured database tables are ingested into Google Cloud Storage (GCS) and split into semantic chunks of 500\u20131000 tokens with a 10% overlap.<\/li>\r\n
BigQuery ML Vector Indexing:<\/strong> Chunks are stored in BigQuery tables, where Vertex AI embedding models (e.g., text-embedding-004<\/code> or text-multilingual-embedding-002<\/code>) generate high-dimensional vector representations directly via SQL.<\/li>\r\n
Grounding with Vertex AI Gemini:<\/strong> When a query arrives, BigQuery Vector Search executes an approximate nearest neighbor (ANN) search to retrieve the top-k most relevant text chunks, which are then injected as factual context into the prompt of a Gemini model.<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n\n\nIndexing and answering are two separate jobs joined by one artefact: the vector index is built once and read on every question, which is what keeps the answers grounded and the cost predictable.<\/figcaption>\n<\/figure>\r\n\r\n\r\n\r\n
2. Step-by-Step Embedding Generation in BigQuery ML<\/h2>\r\n\r\n\r\n\r\n
To generate vector embeddings without exporting datasets to external Python scripts, a remote Vertex AI model connection must be established within BigQuery. The following SQL script connects the embedding model and transforms a table of raw document chunks into a vector-indexed knowledge repository:<\/p>\r\n\r\n\r\n\r\n
-- 1. Create a remote connection to Vertex AI Embedding endpoint\nCREATE OR REPLACE MODEL `enterprise_kb.vertex_embed_model`\nREMOTE WITH CONNECTION `us-central1.vertex_ai_connection`\nOPTIONS (\n ENDPOINT = 'text-embedding-004'\n);\n\n-- 2. Generate vector embeddings for raw corporate document chunks\nCREATE OR REPLACE TABLE `enterprise_kb.document_embeddings` AS\nSELECT\n doc_id,\n chunk_id,\n content_text,\n source_url,\n ml_generate_embedding_result AS content_embedding\nFROM\n ML.GENERATE_EMBEDDING(\n MODEL `enterprise_kb.vertex_embed_model`,\n TABLE `enterprise_kb.raw_document_chunks`,\n STRUCT(TRUE AS flatten_json_output)\n )\nWHERE\n content_text IS NOT NULL;<\/code><\/pre>\r\n\r\n\r\n\r\n
To ensure sub-second query performance across large document repositories, an Inverted File (IVF) vector index must be applied to the embedding column before running semantic similarity searches:<\/p>\r\n\r\n\r\n\r\n
-- 1. Create an Inverted File (IVF) Approximate Nearest Neighbor index\nCREATE VECTOR INDEX `kb_semantic_index`\nON `enterprise_kb.document_embeddings`(content_embedding)\nOPTIONS (\n index_type = 'IVF',\n distance_type = 'COSINE',\n ivf_options = '{\"num_lists\": 1000}'\n);\n\n-- 2. Execute a vector similarity search to retrieve top-3 relevant chunks\nSELECT\n base.doc_id,\n base.chunk_id,\n base.content_text,\n base.source_url,\n distance AS cosine_distance\nFROM\n VECTOR_SEARCH(\n TABLE `enterprise_kb.document_embeddings`,\n 'content_embedding',\n (\n SELECT ml_generate_embedding_result AS content_embedding\n FROM ML.GENERATE_EMBEDDING(\n MODEL `enterprise_kb.vertex_embed_model`,\n (SELECT 'What are the SLA escalation rules for critical server outages?' AS content_text),\n STRUCT(TRUE AS flatten_json_output)\n )\n ),\n top_k => 3,\n distance_type => 'COSINE'\n )\nORDER BY distance ASC;<\/code><\/pre>\r\n\r\n\r\n\r\n
4. Step-by-Step RAG Grounding Pipeline with Gemini<\/h2>\r\n\r\n\r\n\r\n
Once the relevant context chunks are retrieved via BigQuery Vector Search, a structured grounding prompt is assembled and transmitted to the Vertex AI Gemini API via Python:<\/p>\r\n\r\n\r\n\r\n
from google.cloud import bigquery\nimport vertexai\nfrom vertexai.generative_models import GenerativeModel\n\n# Initialize Google Cloud clients\nbq_client = bigquery.Client(project='enterprise-ai-project')\nvertexai.init(project='enterprise-ai-project', location='us-central1')\n\ndef answer_query_with_rag(user_question):\n \"\"\"Executes a BigQuery vector search and grounds Gemini responses.\"\"\"\n \n # Query BigQuery Vector Search for top-3 semantic chunks\n search_query = f\"\"\"\n SELECT base.content_text, base.source_url\n FROM VECTOR_SEARCH(\n TABLE `enterprise_kb.document_embeddings`,\n 'content_embedding',\n (\n SELECT ml_generate_embedding_result AS content_embedding\n FROM ML.GENERATE_EMBEDDING(\n MODEL `enterprise_kb.vertex_embed_model`,\n (SELECT @question AS content_text),\n STRUCT(TRUE AS flatten_json_output)\n )\n ),\n top_k => 3,\n distance_type => 'COSINE'\n )\n \"\"\"\n \n job_config = bigquery.QueryJobConfig(\n query_parameters=[bigquery.ScalarQueryParameter(\"question\", \"STRING\", user_question)]\n )\n results = bq_client.query(search_query, job_config=job_config).result()\n \n # Format retrieved chunks into a factual context block\n context_block = \"\\n---\\n\".join([f\"Source ({row.source_url}): {row.content_text}\" for row in results])\n \n # Construct strict RAG prompt\n rag_prompt = f\"\"\"You are an authoritative enterprise technical assistant.\nAnswer the user's question ONLY using the facts provided in the Context below.\nIf the answer cannot be determined from the Context, state explicitly that information is unavailable.\n\nContext:\n{context_block}\n\nQuestion:\n{user_question}\n\"\"\"\n\n model = GenerativeModel(\"gemini-1.5-pro\")\n response = model.generate_content(rag_prompt)\n return response.text\n\nif __name__ == '__main__':\n output = answer_query_with_rag(\"What are the SLA escalation rules for critical server outages?\")\n print(output)<\/code><\/pre>\r\n\r\n\r\n\r\n
What this tutorial achieves:<\/strong> The deployment of a serverless, highly scalable RAG knowledge base utilizing Google Cloud Vertex AI embeddings, BigQuery ML vector indexing, and Gemini generative grounding.<\/p>\r\n\r\n\r\n\r\n
Resulting value:<\/strong> Domain-specific questions regarding internal corporate documentation, manuals, and databases are answered with high precision and verifiable source attribution. AI hallucinations are eliminated by strictly bounding answers to retrieved organizational facts. Furthermore, executing semantic indexing and vector searches directly within BigQuery removes the need for expensive, dedicated vector database clusters while maintaining enterprise-grade data security and IAM governance.<\/p>\r\n\n\n
![\"Two](\"https:\/\/www.lukaswojcik.com\/blog\/wp-content\/uploads\/diagrams\/hand-rag-en.png\")
\n