INNER CODE UNIT · Python

split_document_by_splitter_type

Azure/logicapps · ai-sample/TokenizeDocFunction/function_app.py:81

def split_document_by_splitter_type(
    documents,
    document_type,
    splitter="RECURSIVE",
    secondary_splitter="RECURSIVE",
    headers_to_split_on=None,
    chunk_size=4000,
    chunk_overlap=200,
    length_function=len,
):

    MARKUP_HEADERS = [
        ("#", "Header 1"),
        ("##", "Header 2"),
        ("###", "Header 3"),
        ("####", "Header 4"),
    ]

View source record →