Context Similarity

class ContextSimilarityEvaluator

Measures how much context has contributed to the answer’s. A higher value suggests a greater proportion of the context is present in the LLM’s response.

Parameters:

  • embed_model (BaseEmbedding) – The embedding model used to compute vector representations.

  • similarity_mode (str, optional) – Similarity strategy to use. Supported options are “cosine”, “dot_product”, and “euclidean”. Defaults to “cosine”.

  • similarity_threshold (float, optional) – Embedding similarity threshold for determining whether a context segment “passes”. Defaults to 0.8.

Example

from pineflow.core.evaluation import ContextSimilarityEvaluator
from pineflow.embeddings.huggingface import HuggingFaceEmbedding

embedding = HuggingFaceEmbedding()
ctx_sim_evaluator = ContextSimilarityEvaluator(embed_model=embedding)
evaluate(contexts, generated_text)
Parameters:

  • contexts (List[str]) – List contexts used to generate LLM response.

  • generated_text (str) – LLM response based on given context.

Example

evaluation_result = ctx_sim_evaluator.evaluate(
    contexts=[], generated_text="<candidate>"
)