Features
Citations and confidence
How each answer is checked against your sources, and how to use citations and confidence in your app.
A model can sound sure about something it never read. Docklee checks every answer against the chunks the engine actually used before it returns it. Citations show where an answer came from, and confidence shows how well your sources support it.
How an answer is checked
- Keep only real citations. A citation that doesn't point to one of the chunks used for the answer is removed.
- Compare the answer with the sources. Docklee measures how much of the answer's wording appears in those chunks.
- Adjust confidence. The confidence the model reported is then adjusted:
| The check finds | Confidence becomes |
|---|---|
| No valid citation | 0 |
| Little overlap between the answer and its sources | Half of the reported value |
| Strong overlap between the answer and its sources | The reported value plus 0.1, up to 1 |
| Anything in between | The reported value |
Citations
| Python | TypeScript | HTTP | Description |
|---|---|---|---|
chunk_id | chunkId | chunk_id | The chunk the answer drew on. Matches chunk_id from retrieve. |
page_number | pageNumber | page_number | The page in the source, when the source has pages. |
section_title | The section heading in the source, when there is one. |
Use page_number and section_title to label a citation, such as "Refund policy, page 4".
Use confidence in your app
result = lynk.query_sync(question)
if result.confidence == 0:
reply = "I couldn't find that in our documentation."
elif result.confidence < 0.5:
reply = f"{result.answer}\n\nPlease double-check this with the source."
else:
reply = result.answerPick thresholds by testing real questions against your own engine. A cutoff that suits one set of sources can be too strict for another.
Answers without citations
A Precision Mode answer is prepared ahead of time, so it comes back with its confidence and no citations.
Next steps
You can tell how much to trust an answer. See how an engine searches both kinds of source next.