Chunking that respects structure — don't shred your own documents (step 9/9) · context and retrieval

Checkpoint

One last thing before we move on. pass this to mark the lesson done, or skip and keep moving. hop to the next when you're ready.

Final drill. Build a structure-aware chunker WITH overlap. Write chunk_document(text, max_chars, overlap, separators) that:

First runs the recursive split (algorithm from step 8) to get chunks ≤ max_chars.
Then adds overlap: for each chunk after the first, prepend the LAST overlap characters of the previous chunk.
Returns the overlap-augmented chunk list.

Three cases run for you. Expected output:

3 chunks, lengths: [22, 30, 27]
first chunk: 'First short paragraph.'
second chunk starts with: 'ragraph.Second paragrap'

Checkpoint

One last thing before we move on. pass this to mark the lesson done, or skip and keep moving. hop to the next when you're ready.

Final drill. Build a structure-aware chunker WITH overlap. Write chunk_document(text, max_chars, overlap, separators) that:

First runs the recursive split (algorithm from step 8) to get chunks ≤ max_chars.
Then adds overlap: for each chunk after the first, prepend the LAST overlap characters of the previous chunk.
Returns the overlap-augmented chunk list.

Three cases run for you. Expected output:

3 chunks, lengths: [22, 30, 27]
first chunk: 'First short paragraph.'
second chunk starts with: 'ragraph.Second paragrap'

full-screen editor opens — close anytime to keep reading.

Chunking that respects structure — don't shred your own documents — step 9 of 9