RAG Retrieval Engineering

Translation note: The Simplified Chinese source was translated from the English report RAG Retrieval Engineering. To preserve technical accuracy and executability, product names, paper titles, model names, abbreviations, API names, and code blocks remain in their original form wherever possible; Mermaid labels in the Chinese source were translated into Chinese.

Executive summary

The “Knowledge recall strategies (such as hybrid retrieval, Rerank, etc.)” in the job requirements most directly corresponds to RAG retrieval Retrieval Engineering (RAG Retrieval Engineering). This direction is sometimes also called retrieval Engineering (Retrieval Engineering), Search Relevance Engineering, or Retri retrieval eval Layer in AI Application Engineering.

In real systems, this work lies between the enterprise’s raw data and large language models (LLMs): it determines how content is chunking, indexed, filtered, retrieval, fusion, reranking, assembled into context, and then evaluation, observed, and governed in production environments. Therefore, retrieval engineering is not just about “vector search,” but an engineering discipline that systematically optimizes recall rate, accuracy, latency, security, and cost efficiency around well-founded AI applications.

For most teams, the most robust default architecture is a Two-stage or three-stage assembly lines:

1. Using inverted index-based lexical retrieval (BM25) with Embedding and ANN-based dense retrieval as the first stage recall layer;
2. Use RRF (Reciprocal Rank Fusion, last fusion) or normalized weighted score fusion to merge multi-path retrieval results;
3. Before generating the model assembly context, candidate results are fine-tuned using Cross-Encoder reranking Model (Cross-Encoder Reranker).

This pattern repeatedly appears in mainstream platform documentation and retrieval research because the two approaches are clearly complementary: lexical retrieval still excel at handling precise terminology, IDs, codes, version numbers, and low-frequency words; dense retrieval is better at semantic matching and synonym paraphrasing; reranking is used to restore the sorting accuracy of the final candidate set.

If you are preparing for an AI application engineering position, a solid learning sequence is:

First, learn Lucene, BM25, and participles; then learn Embedding and ANN; Then they learn hybrid retrieval and reranking; Finally, we learn evaluation, observability, ACL filtering, and governance.

Teams that skip the first and last parts often produce a decent demo system but fail under real enterprise traffic, permission boundaries, or cost constraints.

Background

retrieval Retrieval-Augmented Generation (RAG) connects LLMs with external knowledge, enabling models to generate answers based on corpora beyond pre-training data. Organizations like AWS and Google publicly describe RAG as “retrieval first, then generate.” But in production systems, this simple “retrieval” box expands into a large subsystem that includes: index building, retrieval orchestration, filtering, sorting, and context loading.

When job descriptions explicitly mention hybrid retrieval（Hybrid Retrieval） and reranking（Rerank）, what companies refer to as “knowledge recall strategy” usually refers to this subsystem.

Architecturally, retrieval engineering lies between Data Ingestion and Generation / Orchestration:

• Upstream dependencies document parsing, metadata extraction, ACL inheritance, text chunking, embedding generation, and index construction;
• Downstream determines which evidence enters the model context, in what order, whether redundancy exists, and what security constraints apply.

Since retrieval quality strongly affects answer quality, many problems that appear to belong to “LLM quality” in real production environments are essentially retrieval problems.

flowchart LR
    A[原始数据源] --> B[解析与规范化]
    B --> C[分块与元数据]
    C --> D1[词法索引]
    C --> D2[向量索引]
    U[用户查询] --> Q[查询理解]
    Q --> R1[BM25 检索]
    Q --> R2[稠密检索]
    R1 --> F[混合融合]
    R2 --> F
    F --> RR[重排器]
    RR --> CA[上下文组装]
    CA --> G[LLM 生成]
    G --> O[日志、评测与反馈]
    O --> Q
    O --> C

retrieval can be understood as undertaking four types of responsibilities:

1. Representation Engineering: Word segmentation, analyzer, embedding, and metadata;
2. candidate Generation (Candidate Generation): BM25, sparse vector, dense vector, ANN, and hybrid retrieval;
3. candidate Refinement (Candidate Refinement): reranking, filtering, deduplication, diversification, and context construction;
4. Production Control: Latency budgeting, caching, observability, evaluation, A/B testing, and governance.

This splitting method also aligns with the API design of modern search engines and vector systems: Elasticsearch Retriever, OpenSearch hybrid retrieval pipeline, Pinecone mixed search and reranking, Qdrant mixed query fusion, as well as Milvus’s filtering and multi-index search. Broadly, they distinguish between “broad recall” and “refined sorting” into different stages.

Core technology

Lexical Retrieval

lexical retrieval remains the foundation of enterprise-grade RAG. A system dependencies built on Lucene-style full-text search inverted index（Inverted Index）: it maps terms to documents containing them. OpenSearch’s concept document clearly defines inverted index in this way; Elasticsearch and OpenSearch analyzers further split text into word elements through a pipeline of character filters, tokenizers, and token filters.

This is important because retrieval quality is significantly influenced by analyzer design. Identifiers, camelCase, source code, legal terms, product SKUs, and multilingual text often require different segmentation strategies.

In Elasticsearch, BM25 is the default similarity model used for text fields. Elastic’s documentation and BM25 technical analysis point out that the BM25 comprehensively considers the following factors:

• Term Frequency;
• Inverse Document Frequency;
• Document-Length Normalization.

Here, parameter k1 controls word frequency saturation, and parameter b controls document length normalization strength.

In practice, when queries contain precise business terms, low-frequency terms, version numbers, chapter numbers, error messages, or product names, BM25 is usually very strong. This is why, even in zero-shot neural retrieval scenarios, BEIR still regards BM25 as a robust baseline.

An important engineering conclusion is: excellent lexical retrieval begins before choosing any Embedding model; it first depends on Field design and Analyzer design.

• Long narrative texts can use standard analyzers;
• ID and keywords may require precise matching fields;
• Code or logs usually require custom segmentation;
• Multilingual corpora often require language-specific analyzers or the creation of multiple index fields for the same content.

If a retrieval system performs poorly on precise matching intents, the problem often lies with the analyzer and field mapping, not necessarily with the Embedding model.

Dense Retrieval and ANN

dense retrieval represent text as Embedding and search based on vector similarity, rather than relying on perfectly identical word overlaps. Pinecone’s semantic search introduction describes dense vector as points in higher-dimensional space, where semantically similar texts come close to each other; Elasticsearch’s dense vector documents use the same pattern, performing kNN searches on dense_vector fields.

The core value of dense retrieval is to bridge the differences in synonymous paraphrasing and expression that BM25 is not good at.

For large-scale data, dense retrieval is usually an approximate search rather than an exhaustive, precise search. As corpus size increases, the cost of precise nearest neighbor search rises rapidly, so production systems typically use ANN (Approximate Nearest Neighbor) indexes.

HNSW’s foundational paper proposed a schematic ANN index with a controllable hierarchical structure. Today, HNSW remains one of the most widely deployed ANN algorithms in modern vector databases. Milvus describes HNSW as a multi-level graph structure that supports rapid traversal from rough to thin; Qdrant’s search documentation also uses HNSW as its core vector search structure.

FAISS is the most representative underlying toolkit in dense similarity search research and systems engineering. Meta and Faiss define it as a library for efficient and dense vector similarity search and clustering, with the following capabilities:

• CPU and GPU implementation;
• Multiple index structures;
• vector compression;
• Parameter tuning;
• Supports large-scale collections that may not be fully loaded into memory.

A recent Faiss paper emphasizes that Faiss itself is not a distributed enterprise database; It is a retrieval toolkit and often serves as the underlying engine for higher-level systems.

Different ANN families correspond to different engineering trade-offs:

• HNSW: Usually has excellent recall-delay balance, with relatively direct parameter tuning, but may consume more memory;
• IVF / PQ class index: Memory and cost are reduced through coarse quantization and compression, but more detailed parameter tuning is usually required; incorrect parameters may cause retrieval quality loss.

OpenSearch’s Faiss product quantification document states that PQ can be used with Faiss-based HNSW or IVF ANN. Elasticsearch’s dense vector documentation also offers built-in quantization capabilities for floating-point vector, reflecting the industry-wide trend of reducing vector search memory costs through quantization.

Hybrid Retrieval and Result Fusion

hybrid retrieval combine lexical and semantic signals, because the failure patterns of the two types of methods are complementary. Pinecone’s hybrid search guide defines hybrid search as combining both dense vector and sparse vector in a single request; Qdrant and OpenSearch support combining dense retrieval and lexical retrieval before fusion results.

In the corporate corpus, hybrid retrieval is usually the first baseline worth serious investment. For example:

• A user might search: SOX scope control 2.3 revision;
• Another user might search: “How do we restrict financial approval permissions?” ”

The former heavily depends on precise word items, while the latter relies more on semantic matching, both requiring different retrieval signals.

The simplest and most robust fusion method is RRF (Reciprocal Rank Fusion, last fusion). Elasticsearch’s RRF documentation describes it as a method to merge multiple result sets without tuning different correlation score scales. The standard format is: each result contributes to the final score based on its ranking in the results set:

1 / (k + rank)

OpenSearch’s Score Ranker Processor also uses RRF to combine the results of different query clauses.

RRF is popular because it avoids the fragile scale calibration between BM25 scores and vector similarity scores. As long as all retrievers can produce basically reasonable sorting, RRF often achieves surprisingly robust results.

When fraction calibration is feasible, Weighted fusion (Weighted Fusion) may outperform simple RRF in stable domains. OpenSearch’s Normalization Processor and Hybrid Search Guide describe the following process:

1. Normalize the scores of each path;
2. Weighted arithmetic means, harmonic means, or geometric averages are used for combinations.

OpenSearch’s hybrid search technology article also demonstrates the explicit weighted arithmetic fusion between lexical and semantic signals. Qdrant also supports ranking fusion based on RRF and DBSF, and allows you to overlay business logic such as timeliness and popularity on top of retrieval scores through custom scoring formulas.

Therefore, a clear default approach is:

1. Start with BM25 + dense retrieval + RRF;
2. When you already have domain annotation data and the score distribution is relatively stable, upgrade to Normalized weighted fusion;
3. If you need to combine two or more signals, such as BM25, dense retrieval, sparse retrieval, time attenuation, and authority, a phased design should be prioritized: first fusionretrieval channel, then execute reranking, and finally apply business rules.

The table above summarizes typical behaviors reported in BEIR, mixed search documents by various vendors, and later interactive retrieval literature.

Reranking

reranking is the refined stage that transforms the broad candidate pool into an accurate final candidate set. Pinceone’s reranking documentation describes reranking as a two-stage process: the system first retrieval candidate collection, then uses a reranking model to reorder queries and documents based on correlation between queries and candidate documents.

This is one of the most standard patterns in RAG, because the optimization goals for the first phase of search are usually speed and high recall rate, rather than final sorting accuracy.

The most common reranking device is the Cross-Encoder. The Sentence Transformers documentation clearly explains the differences between dual encoders and cross-encoders:

• Bi-Encoder: Encoding queries and documents separately;
• Cross-Encoder: Query-Document pairs the federated input model to directly output the correlation score.

Cross encoders are usually more accurate, but much slower when the candidate set is large. Elasticsearch’s semantic reranking documentation makes the same distinction and points out that its semantics reranking support for cross-encoders. Therefore, cross-encoders are usually only applied to the first 20~100 candidate results, rather than the entire corpus.

The so-called “Dual encoder refinement similar to reranking” is essentially still a stronger first stage dense retrieval: because document vector can be calculated in advance, it can scale to large corpora; However, when scoring, it cannot fully model token-level interactions between a specific query and a particular paragraph.

A practical rule of understanding is:

• Dual encoders are used to extend to large-scale corpora;
• Cross encoders are used for refining candidate shortlists;
• Late-stage interaction models like ColBERT sit between the two, exchanging higher storage overhead for stronger token-level matching.

If you need to explain that “semantic search is not a single technology,” ColBERTv2 and the LoTTE evaluation system are representative references.

LLM-based reranking is also becoming increasingly practical, especially with hosted reranking APIs. Cohere’s official documentation describes the reranking model as: sorting a set of text by semantic relevance based on queries; Pinecone also provides managed reranking endpoints that receive queries and candidate document lists and return new sorting.

Compared to training reranking tools yourself, these solutions are simpler in terms of operations but introduce API costs, network latency, and data governance issues, making them more suitable for small candidate sets, prototype validation, or high-value workflows.

Distillation is important when systems need to maintain reranking quality under massive traffic conditions. Recent studies like PairDistill show that stronger pairwise sorting models or LLM reranking can supervise faster retrievers, preserving a significant portion of reranking quality at lower online service costs.

From an industrial perspective, distillation is a bridge that transforms “good results but too slow” into “good enough and acceptable cost.”

Query understanding, filtering, chunking, and context assembly

Query understanding is where retrieval project begins to intersect with the Agent project.

Query Rewriting can transform insufficient information, context dependencies, or conversational user input into forms better suited for search. The Rewrite-Retrieve-Read paper shows that rewriting queries themselves can improve retrieval enhance LLM performance. Newer studies such as RaFe and DMQR-RAG further use sorting feedback or diversified multi-query rewriting.

In production environments, query rewriting is especially suitable for the following situations:

• User questions omit key information;
• Current issues depend on previous dialogues;
• User input contains a large number of task instructions, but very few keywords are truly suitable for retrieval;
• The original expression differs significantly from the language of the document in the knowledge base.

query expansion（Query Expansion） Increase coverage by adding relevant terms or pseudo-documents. Query2doc shows that pseudo-documents generated by LLMs can improve sparse retrieval and dense retrieval. HyDE is slightly different: it becomes a “hypothetical related document,” embedding the document, and then using this vector as the retrieval query.

Both approaches may be effective when “relevance” is more easily expressed in document form rather than through users’ original short queries. HyDE is especially suitable for zero-sample or label-scarce environments.

In enterprise-level RAG, Metadata filtering and ACL filtering are non-negotiable. Pinecone, Qdrant, Milvus, Elasticsearch, and OpenSearch all provide metadata or payload filtering, but their support for true access control differs:

• Pinecone and Qdrant support narrowing retrieval scope through metadata filtering;
• Milvus supports filtering search and scalar predicates;
• Elasticsearch and OpenSearch also provide document-level and field-level security controls for role-based read isolation.

If the requirement is simply to “display product documentation for 2025,” metadata filtering may be sufficient; If the requirement is “financial users absolutely cannot retrieval the HR-only paragraph,” then native search engine access control becomes much more important.

chunking（Chunking） is often underestimated. Recent studies show that chunking strategies, block size, overlap length, and context construction methods all measurably affect RAG reliability. A 2025 study compared Late Chunking with Contextual Retrieval; The 2026 study examined block types, block sizes, overlaps, and context lengths under industrial configuration.

A robust conclusion is not “there is a block size that works for all scenarios,” but rather: chunking should match Answer: Granularity and Retriever represents capability boundaries.

For narrative enterprise documents, a common starting point is:

• Each piece is about 300～800 Token;
• Use moderate overlap;
• Further optimization based on evaluation clusters.

For code, tables, or structured institutional documents, structure-aware chunking is usually better than simple fixed windows. This recommendation comes partly from related research and partly from engineering inferences based on research findings.

Even if retrieval result is “correct,” context assembly is still important. Lost in the Middle Research suggests that language models may not fully utilize evidence placed in the middle of the long context. Therefore, retrieval engineers should not simply cram the first 10 chunks into the prompt according to the original score order.

Better assembly strategies usually include:

• Removing duplicates;
• Grouping by document;
• Placing the highest-value evidence at the forefront;
• Interwoven arrangements of evidence from different sources to maintain diversity;
• When the marginal utility of new context decreases, actively truncate.

This is a typical example of the overlap between retrieval engineering and prompt engineering.

flowchart TD
    Q[用户查询] --> U1[改写或扩展]
    Q --> U2[实体与元数据提取]
    U1 --> L1[词法检索]
    U1 --> L2[稠密检索]
    U2 --> F1[元数据或 ACL 过滤]
    L1 --> H[混合融合]
    L2 --> H
    F1 --> H
    H --> R[交叉编码器或 LLM 重排]
    R --> D[去重、多样化与分组]
    D --> P[Prompt 上下文打包器]
    P --> M[LLM]

Vector Systems and Product Selection

Although FAISS is the most valuable underlying ANN library to understand, production retrieval engineers often need to choose between an integrated search engine and a vector database.

• Elasticsearch / OpenSearch: Usually the greatest advantage when full-text search, hybrid search, and security controls need to be unified on the same platform;
• Qdrant: Outstanding performance in modern hybrid retrieval, filtered vectorretrieval, and flexible query fusion;
• Milvus: Suitable for teams that need open-source distributed vector databases and want to support multiple types of ANN indexes;
• Pinecone: Suitable for teams that value managed operations, serverless deployment, and managed inference services.

The table above summarizes official documents from various manufacturers as well as basic research literature from FAISS.

Implementation guide

A pragmatic way to implement this is to preserve Two retrieval Surfaces (Retrieval Surfaces):

• One lexical retrieval surface, running on Elasticsearch or OpenSearch;
• A vectorretrieval can run on the same engine or use dedicated vector backends like Qdrant, Milvus, or Pinecone.

If an organization already uses Lucene-based search systems extensively, starting with Elasticsearch or OpenSearch to handle both lexical retrieval and vectorretrieval is usually the path with least resistance.

If the main pain point is large-scale semantic retrieval and you want to experiment quickly, you can combine Qdrant or Pinecone with Lucene-based engines: the former handles vector semantic recall, while the latter handles lexical recall and access control.

The following example uses the following:

• Python；
• LangChain-style application combination method;
• Qdrant As vector storage;
• OpenSearch / Elasticsearch As a lexical layer;
• Cross encoder As a reranking device.

Package names and secondary APIs will continue to evolve, especially between different versions of LangChain. Before going live, you should lock the dependent version and use the current LangChain documentation instead of the archived v0.x page. The migration guide for LangChain v1 also clearly states that some retrieval auxiliary components have been relocated or changed the import path.

Example: Build an Ingestion Pipeline with Chunking, Embeddings, and Qdrant

from typing import List
from dataclasses import dataclass

from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_core.documents import Document

from qdrant_client import QdrantClient, models

@dataclass
class ChunkedDoc:
    id: str
    text: str
    metadata: dict

def chunk_documents(raw_docs: List[Document]) -> List[ChunkedDoc]:
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=600,
        chunk_overlap=80,
        separators=["\n\n", "\n", ". ", " ", ""],
    )
    out: List[ChunkedDoc] = []
    for i, doc in enumerate(raw_docs):
        pieces = splitter.split_text(doc.page_content)
        for j, piece in enumerate(pieces):
            meta = dict(doc.metadata)
            meta["chunk_id"] = j
            out.append(
                ChunkedDoc(
                    id=f"{meta.get('doc_id', i)}:{j}",
                    text=piece,
                    metadata=meta,
                )
            )
    return out

emb = OpenAIEmbeddings(model="text-embedding-3-large")
client = QdrantClient(url="http://localhost:6333")

collection_name = "kb_chunks"
dim = len(emb.embed_query("dimension probe"))

client.recreate_collection(
    collection_name=collection_name,
    vectors_config=models.VectorParams(size=dim, distance=models.Distance.COSINE),
)

def upsert_chunks(chunks: List[ChunkedDoc]) -> None:
    texts = [c.text for c in chunks]
    vecs = emb.embed_documents(texts)

    client.upsert(
        collection_name=collection_name,
        points=[
            models.PointStruct(
                id=chunk.id,
                vector=vec,
                payload={"text": chunk.text, **chunk.metadata},
            )
            for chunk, vec in zip(chunks, vecs)
        ],
    )

This is a common dense data ingestion process: chunking, generating Embedding, attaching metadata, and then Upsert. The Qdrant documentation emphasizes payload indexing and filtering capabilities, which is why metadata design should be completed early in the project rather than added only after the system is established.

Example: Build a Lexical Index with Elasticsearch or OpenSearch

from opensearchpy import OpenSearch

os_client = OpenSearch(
    hosts=[{"host": "localhost", "port": 9200}],
    http_compress=True,
)

index_name = "kb_lexical"

mapping = {
    "settings": {
        "analysis": {
            "analyzer": {
                "default": {"type": "standard"}
            }
        }
    },
    "mappings": {
        "properties": {
            "title": {"type": "text"},
            "body": {"type": "text"},
            "doc_id": {"type": "keyword"},
            "tenant_id": {"type": "keyword"},
            "created_at": {"type": "date"},
            "acl": {"type": "keyword"},
        }
    },
}

if not os_client.indices.exists(index=index_name):
    os_client.indices.create(index=index_name, body=mapping)

def index_lexical(doc_id: str, title: str, body: str, tenant_id: str, acl: list[str]):
    os_client.index(
        index=index_name,
        id=doc_id,
        body={
            "title": title,
            "body": body,
            "doc_id": doc_id,
            "tenant_id": tenant_id,
            "acl": acl,
        },
        refresh=True,
    )

This code creates a lexical retrieval that supports BM25 and explicitly defines the metadata fields required for tenant isolation and ACL-aware filtering. If the system requires document-level or field-level security, native search engine control should be prioritized rather than relying solely on application-layer filtering.

Example: Hybrid Retrieval with Client-Side RRF

from collections import defaultdict
from typing import Any

def lexical_search(query: str, tenant_id: str, k: int = 20) -> list[dict[str, Any]]:
    resp = os_client.search(
        index=index_name,
        body={
            "size": k,
            "query": {
                "bool": {
                    "must": [{"multi_match": {"query": query, "fields": ["title^2", "body"]}}],
                    "filter": [{"term": {"tenant_id": tenant_id}}],
                }
            },
        },
    )
    return [
        {
            "id": hit["_id"],
            "text": hit["_source"]["body"],
            "score": hit["_score"],
            "source": "bm25",
        }
        for hit in resp["hits"]["hits"]
    ]

def dense_search(query: str, tenant_id: str, k: int = 20) -> list[dict[str, Any]]:
    qvec = emb.embed_query(query)
    points = client.query_points(
        collection_name=collection_name,
        query=qvec,
        limit=k,
        query_filter=models.Filter(
            must=[models.FieldCondition(
                key="tenant_id",
                match=models.MatchValue(value=tenant_id)
            )]
        ),
        with_payload=True,
    ).points
    return [
        {
            "id": str(p.id),
            "text": p.payload["text"],
            "score": p.score,
            "source": "dense",
        }
        for p in points
    ]

def rrf_fuse(rankings: list[list[dict[str, Any]]], rank_constant: int = 60) -> list[dict[str, Any]]:
    scores = defaultdict(float)
    doc_store = {}
    for ranking in rankings:
        for rank, item in enumerate(ranking, start=1):
            scores[item["id"]] += 1.0 / (rank_constant + rank)
            doc_store[item["id"]] = item
    fused = sorted(scores.items(), key=lambda x: x[1], reverse=True)
    return [{**doc_store[doc_id], "rrf_score": score} for doc_id, score in fused]

def hybrid_search(query: str, tenant_id: str, k: int = 30) -> list[dict[str, Any]]:
    bm25_hits = lexical_search(query, tenant_id, k)
    dense_hits = dense_search(query, tenant_id, k)
    return rrf_fuse([bm25_hits, dense_hits])[:k]

If the backend already natively supports RRF or hybrid retrieval pipelines, native capabilities should be prioritized. Elasticsearch provides the RRF Retriever, while OpenSearch offers both score-normalized weighted fusion and rank-based fusion pipelines. The above client implementations remain suitable as conceptual baselines and local experimental implementations.

Example: Rerank with a Cross-Encoder

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

def rerank(query: str, candidates: list[dict[str, Any]], top_n: int = 8) -> list[dict[str, Any]]:
    pairs = [(query, c["text"]) for c in candidates]
    scores = reranker.predict(pairs)
    rescored = [{**c, "rerank_score": float(s)} for c, s in zip(candidates, scores)]
    rescored.sort(key=lambda x: x["rerank_score"], reverse=True)
    return rescored[:top_n]

The Sentence Transformers documentation specifically explains how cross-encoders are used in reranking. Pinecone and Cohere also offer equivalent managed reranking APIs; If you don’t want to host the model yourself, you can place reranking capabilities behind the service boundary.

Example: HyDE-Style Query Expansion

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4.1-mini", temperature=0)

HYDE_PROMPT = """
Write a concise hypothetical passage that would answer the question below.
Do not explain what you are doing.

Question: {question}
"""

def hyde_expand(question: str) -> str:
    return llm.invoke(HYDE_PROMPT.format(question=question)).content

def hyde_search(question: str, tenant_id: str, k: int = 20):
    hypothetical_doc = hyde_expand(question)
    qvec = emb.embed_query(hypothetical_doc)
    # then use dense_search-like logic with qvec

This is just a simplified demonstration of HyDE’s approach, not a complete production implementation. Its core concept comes from the HyDE paper: Mr. becomes a hypothetical related document, embedding it, and then retrieval adjacent real documentation.

Recommended technology stack

Evaluation and Test Plan

Offline evaluation is fundamental. For retrieval systems, the most important metrics include:

• Recall@K；
• Precision@K；
• MRR；
• nDCG。

Pinecone’s offline evaluation guide and Evidently’s ranking metrics guide summarize the significance of these metrics:

• Recall@K Measure the coverage of the first K results across all relevant documents;
• Precision@K Measure the proportion of relevant documents among the top K candidate results;
• MRR Reward the first relevant result that appears earlier;
• nDCG Measure the quality of sorting with hierarchical correlation tags while considering ranking positions.

In practice, the core indicator retrieval the first stage is usually Recall@K; When the system joins the reranking, nDCG@10 and MRR@10 often become especially valuable.

Benchmark selection should match the domain and language environment:

• BEIR: Still the most important broad zero-shot retrieval benchmark, suitable for comparing sparse, dense, late-stage interaction, and reranking methods in heterogeneous tasks;
• MIRACL: Multilingual monolingual retrieval suitable for 18 languages;
• MTEB: Provides a broad Embedding evaluation framework covering multiple tasks and languages;
• LoTTE: Compared to many old benchmarks, it is closer to the long-tail theme retrieval;
• TREC RAG: Provides end-to-end retrieval—public benchmarks for generating systems.

Online testing should not start with “randomly switching production traffic to the new Retriever.” A more reliable launch sequence is:

1. Offline retrievalevaluation；
2. Shadow Traffic；
3. Small flow slicing A/B testing；
4. Expanded the scope of launches。

observability platforms like LangSmith are used to track and evaluation Agent and RAG behaviors; OpenTelemetry provides standardized Trace, Metric, and Log infrastructure. Together, they provide the minimum observational capabilities needed to compare different retrieval versions under real traffic.

A practical test matrix should at least cover:

• Single-round and multi-turn queries;
• Precise word queries and taut-intensive rewriting queries;
• Queries with tenant filtering versus without filtering;
• Short documents and long documents;
• Ordinary queries and adversarial queries.

Each slice should track not only the quality of the answers but also the following:

• Empty-Hit Rate；
• Filter Hit Rate；
• Reranker Fall-Through Rate (reranking Fall-Through Rate；
• P95 delay；
• Cost per request。

retrieval design often exposes problems first in these operational metrics. This suggestion is an engineering inference based on the above observability and evaluation tools.

Deployment Checklist

Before going live, first verify Data paths:

• Whether the parser preserves document boundaries and important structures;
• Whether the Chunk ID is stable;
• Whether metadata is normalized;
• Whether attributes related to ACLs are propagated to retrieval layers;
• Whether each Chunk can be traced back to the source documentation and specific chapters.

If retrieval results lose traceability of their source, the system will be difficult to debug.

Then verify retrieval path:

• Before enabling reranking, the BM25, dense retrieval, and hybrid retrieval baselines were measured separately;
• Tune ANN parameters using a clear recall rate—delay target;
• If metadata filtering is used, indexes for the filtered fields should be created as early as possible; Both Qdrant and Pinecone emphasize that filtering performance depends on how fields are modeled and indexed;
• If using OpenSearch weighted fusion, normalized and combined configurations should be saved in the versioned configuration to ensure experiments are reproducible.

Next, verify Precision path:

• reranking only for the amount of candidate allowed by the delayed budget;
• Cross-encoders and managed reranking APIs are generally not suitable for ultra-large candidate sets, which is why two-stage retrieval exist;
• If higher throughput is required, distillation or compression should be considered;
• If quality on complex paragraphs is insufficient, post-production interaction models or more reasonable chunking should be tested first, rather than recklessly scaling reranking.

Then verify Run path:

• Increase query cache;
• Increased Embedding cache;
• Increase response buffers in suitable scenarios;
• Using OpenTelemetry to collect traces, metrics, and logs for end-to-end paths;
• Create dedicated dashboards retrieval track empty result rate, reranking queue time, average number of retrieval tokens, and cost per request.

Redis positions itself as a high-performance caching and data system that supports vector capabilities, making it a practical way to achieve popular query caching.

Final verification Security and governance pathways:

• For enterprise systems, simple metadata filtering is usually insufficient;
• Use document-level and field-level permission control when needed;
• Carefully test multi-role combinations;
• Suppose the attacker attempts to inject prompts through retrieval chains;
• The OWASP LLM Top 10 lists prompt injection, sensitive information leaks, supply chain issues, and data poisoning as core risks;
• NIST’s Privacy and Risk Management Framework provides broader guidance for privacy and risk governance in AI systems.

A concise list of releases is as follows:

• At least a tagged evaluation set is established for the main query type;
• BM25, dense retrieval, hybrid retrieval, and reranking baselines were measured respectively;
• Adversarial use cases were used to test ACL or tenant filtering;
• P50, P95, and overtime budgets are set for retrieval and reranking;
• observability system collects trace, metric, log, retrieval documentation and costs;
• Retriever, fusion mode, and Reranker all have rollback switches;
• Each deployment records chunking version and index version;
• Documented data retention, privacy, and incident response rules.

These aspects either come directly from the relevant platform documentation or are conservative production engineering inferences based on the capabilities and risks of these platforms.

Recommended Learning Path

Step one: Before specializing in RAG, learn Classical Information retrieval Retrieval (Classical Information Retrieval):

• inverted index；
• Participle;
• Analyzer;
• BM25；
• Lucene’s basic mental model.

OpenSearch’s concept guide, Elasticsearch’s analyzer and segmentation documentation, and Elastic’s explanation of BM25 are all excellent practical starting points, as they connect the information retrieval concepts from textbooks to production search engines.

Step two: Learn dense retrieval and ANN:

• Read the HNSW paper;
• Read Faiss papers and current Faiss documents;
• Study how Milvus, Qdrant, Pinecone, and Elasticsearch implement ANN, filtering, quantization, and hybrid search.

This sequence can simultaneously establish both the algorithmic and system perspectives.

Step three: Learn hybrid retrieval and reranking:

1. Achieving a pure BM25 baseline;
2. Achieving a pure dense retrieval baseline;
3. Implement BM25 + dense retrieval + RRF;
4. Adding cross encoder reranking;
5. Finally, weighted fusion, DBSF, post-interaction analysis, and distillation were studied.

This sequence aligns with the maturity ladder in real systems and helps avoid premature optimization.

Step 4: Study Inquiry, understanding, and chunking:

• Query2doc；
• HyDE；
• Rewrite-Retrieve-Read；
• Recent chunkingevaluation research.

In interviews, candidate who can explain “when to prioritize improving query rewriting and when to adjust chunking” usually stand out more than those who only list terminology, because the former are analyzing failure patterns.

Finally, learn evaluation. observability and governance:

• Using BEIR, MIRACL, MTEB, and TREC RAG;
• Using LangSmith or similar platforms in conjunction with OpenTelemetry to enhance retrieval stack observation capabilities;
• Using OWASP and NIST guidelines to map system risks.

It is this part that transforms a search demo system into an enterprise-level AI application.

Conclusion

The requirements highlighted in the job description clearly belong to the broader RAG retrieval engineering of AI application engineering.

It specifically covers the entire retrieval layer:

• lexical retrieval；
• Embedding；
• ANN；
• Mixed fusion;
• reranking；
• Query rewriting and extension;
• Metadata and ACL filtering;
• chunking；
• context assembly;
• evaluation；
• observability；
• Governance.

If the job description mentions hybrid retrieval and Rerank, it is looking not for an engineer who only calls vector database APIs, but someone who can professionally design, evaluation, and operate the entire retrieval layer.

By 2026, the most reliable default architecture will still be phased systems:

Achieve high recall rates with BM25+ dense retrieval, perform hybrid sorting with RRF or weighted fusion, improve accuracy reranking crossencoders, enforce strict metadata/ACL filtering, and establish a disciplined system of offline and online evaluation.

On this basis, teams can further advance into post-production interaction, distillation, richer query understanding, and domain-specific optimization. But if you can explain why this baseline works, how to measure it, and how to safely operate it in production, then you already have core knowledge directly relevant to the job requirements.