Chunking that respects structure — don't shred your own documents — step 8 of 9
Write recursive_split(text, max_chars, separators) that returns
a list of chunks, each ≤ max_chars. Logic:
- Base case: if
len(text) <= max_chars, return[text]. - Otherwise iterate
separatorsin order. For the first separator that exists intext:- Split
texton that separator intoparts. - Reassemble parts greedily: accumulate parts into
current(joined with the separator); when adding the next part would exceedmax_chars, pushcurrentintochunksand start a new accumulator. - For any chunk still bigger than
max_chars, recurse withseparatorsadvanced past the current one. - Return the result.
- Split
- If no separator fits, fall through and hard-cut every
max_charscharacters.
The separators are ["\n\n", ". ", " ", ""]. Use them in order.
Two cases run for you. Expected output:
paragraph case: 3 chunks
long-sentence case: 3 chunks
⌘↵ runs the editor.read, then continue.
Write recursive_split(text, max_chars, separators) that returns
a list of chunks, each ≤ max_chars. Logic:
- Base case: if
len(text) <= max_chars, return[text]. - Otherwise iterate
separatorsin order. For the first separator that exists intext:- Split
texton that separator intoparts. - Reassemble parts greedily: accumulate parts into
current(joined with the separator); when adding the next part would exceedmax_chars, pushcurrentintochunksand start a new accumulator. - For any chunk still bigger than
max_chars, recurse withseparatorsadvanced past the current one. - Return the result.
- Split
- If no separator fits, fall through and hard-cut every
max_charscharacters.
The separators are ["\n\n", ". ", " ", ""]. Use them in order.
Two cases run for you. Expected output:
paragraph case: 3 chunks
long-sentence case: 3 chunks
this step needs the editor
on desktop today; in the app (coming soon). save your spot and we'll bring you back here when you're ready.