Retrieval-Augmented Generation (RAG) for Commerce

Retrieval-Augmented Generation (RAG) is an AI architecture that pairs a large language model with an external knowledge base. Instead of relying only on what the model learned during training, the system retrieves relevant documents at query time and passes them to the model as context. The model then generates its response grounded in those retrieved documents rather than in its frozen training data.

The approach was introduced by Facebook AI Research in the 2020 paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (Lewis et al., NeurIPS 2020). It has since become the default architecture for enterprise LLM deployments because it solves three of the biggest problems with standalone LLMs: outdated training data, hallucination on specific facts, and the inability to access private or proprietary information.

In commerce, RAG is what makes it possible for an AI shopping agent to answer "is this jacket in stock in my size?" with actual inventory data rather than a guess based on two-year-old training data. Every major AI shopping surface - ChatGPT Shopping, Google AI Mode, Perplexity Shopping, Amazon Rufus - uses some form of RAG against live retailer data.

A commerce RAG pipeline runs in four stages:

1. Ingestion. Product catalog, inventory, reviews, policies, and support docs are chunked, embedded (converted into vector representations), and stored in a vector database or hybrid retrieval index. For commerce, structured product attributes (SKU, price, availability, variants) are typically indexed alongside unstructured text (descriptions, reviews, Q&A).

2. Query embedding. When a shopper asks a question ("waterproof hiking boots for flat feet under $200"), the system embeds the query into the same vector space as the catalog and retrieves the top N semantically relevant products and documents.

3. Context assembly. Retrieved products, policies, and any relevant session context are assembled into a prompt. For shopping, this usually includes product cards, current prices, stock levels, shipping info, and any policies that affect the answer (return policy, warranty).

4. Grounded generation. The LLM generates its response using the assembled context. Because the response is conditioned on retrieved data, the model can cite specific SKUs, correct prices, and current availability - and can say "that's out of stock in your size, but here are two alternatives" with authority.

The quality of the grounded answer is strictly limited by the quality of the retrieval step. Poor retrieval - caused by missing attributes, thin descriptions, or stale feeds - produces confident-sounding but incorrect recommendations. This is why product data enrichment and structured product data matter so much for AI commerce: they are the inputs the retrieval layer has to work with.

Three architectural choices come up in commerce AI:

Plain LLM. Ask ChatGPT "what laptops does Best Buy have in stock?" and a model without retrieval will hallucinate. It will produce plausible-sounding SKUs and prices that are simply wrong. This is why every serious AI shopping feature uses retrieval - plain LLMs cannot answer inventory, price, or availability questions reliably.

Fine-tuning. Training a model on a retailer's specific data can improve tone and domain-specific reasoning. It does not solve the freshness problem - inventory changes minute-to-minute and a fine-tuned model is stuck with whatever data was in its training set. Fine-tuning is expensive, slow to update, and gets worse over time unless constantly retrained.

RAG. The knowledge base can be updated in real time. A retailer can re-embed product descriptions nightly and reflect inventory every few minutes. The model itself never needs to be retrained. This is why RAG is the dominant pattern for enterprise LLM deployments in 2026.

The practical answer for most commerce teams: use RAG as the primary architecture, layer a small amount of fine-tuning (or system prompts) on top if you need a specific voice or reasoning style. Do not bet on a plain LLM for anything that touches real product data.