trustgraph/templates/patterns/override-recursive-chunker.jsonnet
cybermaggedon f7a30006ad
Make templating work more flexibly (#44)
* Restructure directory
* Config loading
* Variable override points in JSONNET templates, separate pulsar-manager template
* Bump version
* Tidy chunking
* Simplified prompt overrides
* Update config loader
* Fix recursive chunker template
2024-08-30 17:47:35 +01:00

27 lines
758 B
Jsonnet
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
pattern: {
name: "override-recursive-chunker",
icon: "🪚",
title: "Replace default chunker with recursive chunker",
description: "The default chunker used in Trustgraph core is a token-based chunker. This pattern replaces that with a recursive chunker, and allows ou to configure the chunking parameters.",
requires: ["pulsar", "trustgraph"],
features: [],
args: [
{
name: "chunk-size",
type: "int",
description: "Chunk size value",
default: 2000,
required: true,
},
{
name: "chunk-overlap",
type: "int",
description: "Overlap size value",
default: 100,
required: true,
}
]
},
module: "components/cassandra.jsonnet",
}