US patent US12801860
Database systems and automated conversational interaction methods using boundary coalescing chunks
Abstract
Database systems and methods are provided for managing usage of large language models (LLMs). One method involves dividing text data into primary chunks using input criteria associated with an LLM service, generating secondary chunks by merging respective pairs of adjacent primary chunks, and inputting a respective secondary chunk to the LLM service when a semantic similarity between a conversational input to a user interface and the respective secondary chunk of the one or more secondary chunks is greater than a threshold. The LLM service generates response data responsive to the conversational input based at least in part on a subset of the text data associated with the respective secondary chunk, and a response is provided to the conversational input at the user interface based at least in part on the response data generated by the LLM service.