r/LangChain
RAG ReRanking Pipeline & Comparison
- upvotes
- 6
- comments
- 7
Post
I recently built a complete RAG reranking pipeline to test how different reranking strategies actually impact the final LLM output. I compared FOUR rerankers (Cross-Encoder, LLM-based, Cohere, and Jina) on the same questions over 5 payment-industry PDFs (payment regulations, PCI DSS, and central bank reports - 1,507 chunks total). For each one, I tracked the ranking before and after reranking to see exactly WHY the order changed. What the project covers: The core weakness of standard vector search (embeddings & cosine similarity) Why we retrieve Top 20 but send only Top 5 to the LLM (tokens, latency, noise) Bi-encoder vs Cross-encoder: reading the question and chunk separately vs together LLM reranking strategies: Pointwise vs Pairwise vs Listwise How to use dedicated reranking APIs (Cohere and Jina) How to design one shared helper so all four rerankers plug into the same pipeline How to evaluate rerankers using before/after positions, Hit@5, and average latency KEY TAKEAWAYS: Similarity ≠ Relevance: A chunk can sit close to the query in vector space and still not answer it. Vector search finds chunks about the topic. A strong reranker picks the chunk that actually answers it. Small cross-encoders can make things WORSE: My local cross-encoder rewarded word matching, penalized scrambled PDF text, and actually pushed both correct answer chunks out of the Top...
Keep reading with a free account
The rest of this post, and every signal for Cohere, is in your free account.
Extracted from these lines
Cohere was a lifesaver: It lifted an answer chunk from 15th to 1st, turning a completely missed answer into a correct, cited one.
From the post
The Verdict: For this specific project, Cohere and Jina gave the best balance of accuracy and speed.
From the post