Skip to Content

Features

Citations and confidence

How each answer is checked against your sources, and how to use citations and confidence in your app.

A model can sound sure about something it never read. Docklee checks every answer against the chunks the engine actually used before it returns it. Citations show where an answer came from, and confidence shows how well your sources support it.

How an answer is checked

  1. Keep only real citations. A citation that doesn't point to one of the chunks used for the answer is removed.
  2. Compare the answer with the sources. Docklee measures how much of the answer's wording appears in those chunks.
  3. Adjust confidence. The confidence the model reported is then adjusted:
The check findsConfidence becomes
No valid citation0
Little overlap between the answer and its sourcesHalf of the reported value
Strong overlap between the answer and its sourcesThe reported value plus 0.1, up to 1
Anything in betweenThe reported value

Citations

PythonTypeScriptHTTPDescription
chunk_idchunkIdchunk_idThe chunk the answer drew on. Matches chunk_id from retrieve.
page_numberpageNumberpage_numberThe page in the source, when the source has pages.
section_titleThe section heading in the source, when there is one.

Use page_number and section_title to label a citation, such as "Refund policy, page 4".

Use confidence in your app

result = lynk.query_sync(question)

if result.confidence == 0:
    reply = "I couldn't find that in our documentation."
elif result.confidence < 0.5:
    reply = f"{result.answer}\n\nPlease double-check this with the source."
else:
    reply = result.answer

Pick thresholds by testing real questions against your own engine. A cutoff that suits one set of sources can be too strict for another.

Answers without citations

A Precision Mode answer is prepared ahead of time, so it comes back with its confidence and no citations.

Next steps

You can tell how much to trust an answer. See how an engine searches both kinds of source next.

On this page