Skip to Content

Features

Chunking

How a vector container splits documents into chunks that keep sections, paragraphs, and tables intact.

How a document is split decides what an engine can find. Splitting every few hundred characters cuts sentences in half and mixes unrelated sections. A vector container splits by the document's own structure instead.

How it works

  1. Read the structure. Docklee reads the document's sections, headings, paragraphs, and tables, along with page numbers.
  2. Group by section. Content is grouped by the section it belongs to, so a chunk never mixes two unrelated sections.
  3. Pack whole paragraphs. Paragraphs are added to a chunk until it reaches its size range. A chunk is always made of complete paragraphs, never part of a sentence.
  4. Keep tables whole. A table becomes its own chunk, so its rows stay together.
  5. Split only what's too large. A single paragraph too long for one chunk is split between sentences, and the last two sentences are repeated at the start of the next piece, so nothing loses its context.
  6. Keep the location. Each chunk keeps its page number and section title, which is what citations point to.

Sizes

SettingValue
Smallest chunkAbout 300 tokens
Largest chunkAbout 1,200 tokens
Overlap when a long paragraph is split2 sentences
TablesAlways one chunk each

Why structure first

Research on chunking keeps reaching the same result: splitting at natural boundaries, such as sections and paragraphs, is the most reliable choice across real documents. Splitting by meaning with a model is costly and often produces fragments too small to answer from.

Give your documents real headings. A heading starts a new section, and sections decide where chunks begin and end.

Next steps

You know how documents are split. See how content is checked before it's stored next.

On this page