抽取式压缩示意
抽取式压缩示意
from sentence_transformers import CrossEncoder
scorer = CrossEncoder("BAAI/bge-reranker-base")
sentences = split_sentences(chunk.text)
pairs = [(query, s) for s in sentences]
scores = scorer.predict(pairs)
keep = [s for s, sc in zip(sentences, scores) if sc >= THRESHOLD]
compressed = " ".join(keep)压缩后再让生成模型引用原文页码/段落时，要保留片段 id 与顺序，否则引用无法回溯。