Features
Chunking
How a vector container splits documents into chunks that keep sections, paragraphs, and tables intact.
How a document is split decides what an engine can find. Splitting every few hundred characters cuts sentences in half and mixes unrelated sections. A vector container splits by the document's own structure instead.
How it works
- Read the structure. Docklee reads the document's sections, headings, paragraphs, and tables, along with page numbers.
- Group by section. Content is grouped by the section it belongs to, so a chunk never mixes two unrelated sections.
- Pack whole paragraphs. Paragraphs are added to a chunk until it reaches its size range. A chunk is always made of complete paragraphs, never part of a sentence.
- Keep tables whole. A table becomes its own chunk, so its rows stay together.
- Split only what's too large. A single paragraph too long for one chunk is split between sentences, and the last two sentences are repeated at the start of the next piece, so nothing loses its context.
- Keep the location. Each chunk keeps its page number and section title, which is what citations point to.
Sizes
| Setting | Value |
|---|---|
| Smallest chunk | About 300 tokens |
| Largest chunk | About 1,200 tokens |
| Overlap when a long paragraph is split | 2 sentences |
| Tables | Always one chunk each |
Why structure first
Research on chunking keeps reaching the same result: splitting at natural boundaries, such as sections and paragraphs, is the most reliable choice across real documents. Splitting by meaning with a model is costly and often produces fragments too small to answer from.
Give your documents real headings. A heading starts a new section, and sections decide where chunks begin and end.
Next steps
You know how documents are split. See how content is checked before it's stored next.