Features
Answer cache
How a repeat of a recent question is answered in milliseconds from an encrypted cache.
Assistants are often asked the same thing many times in a few minutes, such as during an incident or a launch. Running the full search and answer steps again each time is slow and costs credits. The answer cache returns a recent answer straight away when the same question comes back.
How it works
- Answer. A question to
lynk.queryruns the full search and answer steps, and the answer is stored in the cache. - Match. A later question is compared by meaning with recent questions to the same engine. Only a near-identical question counts as a match.
- Return. A match returns the cached answer in a few milliseconds, with no search and no model call.
| Rule | Value |
|---|---|
| How close a question must be | Near-identical in meaning. Rewording that changes the meaning isn't a match. |
| How long an answer is kept | 5 minutes |
| When the cache is cleared | When any source linked to the engine is added, updated, or removed |
| Applies to | lynk.query without stream |
Isolation
- Cached answers are kept separately for each account and engine, and never shared between them.
- Each cached answer is encrypted.
- Cached answers can only be written by authenticated calls, so nothing outside your account can place an answer in your cache.
A cached answer comes back with its answer and confidence, and without citations. When citations matter on every reply, stream the answer instead. Streams aren't cached.
Next steps
Repeated questions are fast. See how Docklee prepares answers ahead of time next.