apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-05-30 01:35:14 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

align chunker + googleaistudio fixes with release/v2.4

Browse source 
Master had a parallel sibling fix for issue #821 (PR #828) using
self.RecursiveCharacterTextSplitter / self.TokenTextSplitter; release
branches converged on the bare module-level form.  Adopt release/v2.4's
version so downstream branches don't drift further.
This commit is contained in:
Cyber MacGeddon 2026-04-29 18:00:55 +01:00
parent f3434307c5
commit c112af0ab0
3 changed files with 7 additions and 19 deletions
Download patch file Download diff file Expand all files Collapse all files

4
trustgraph-flow/trustgraph/chunking/recursive/chunker.py
View file

@ -58,7 +58,7 @@ class Processor(ChunkingService):
2500, 4000, 6400, 10000, 16000]
)
self.text_splitter = self.RecursiveCharacterTextSplitter(
self.text_splitter = RecursiveCharacterTextSplitter(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
length_function=len,
@ -111,7 +111,7 @@ class Processor(ChunkingService):
chunk_overlap = int(chunk_overlap)
# Create text splitter with effective parameters
text_splitter = self.RecursiveCharacterTextSplitter(
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
length_function=len,

4
trustgraph-flow/trustgraph/chunking/token/chunker.py
View file

@ -56,7 +56,7 @@ class Processor(ChunkingService):
2500, 4000, 6400, 10000, 16000]
)
self.text_splitter = self.TokenTextSplitter(
self.text_splitter = TokenTextSplitter(
encoding_name="cl100k_base",
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
@ -108,7 +108,7 @@ class Processor(ChunkingService):
chunk_overlap = int(chunk_overlap)
# Create text splitter with effective parameters
text_splitter = self.TokenTextSplitter(
text_splitter = TokenTextSplitter(
encoding_name="cl100k_base",
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,