Jack Colquitt
9612a11581
Added basic Llamafile integration ( #63 )
...
* Added basic Llamafile integration
* Added llamafile template support
* New templates following llamafile addition
---------
Co-authored-by: Cyber MacGeddon <cybermaggedon@gmail.com>
2024-09-16 16:18:01 +01:00
cybermaggedon
6af86fa09f
Topic templates for extractor ( #62 )
...
* Add topic prompt to templates
* Bump version
* Updated templates
2024-09-15 23:40:37 +01:00
cybermaggedon
ddd8cc90e5
Fix templates ( #55 )
...
* Fix template import problem
* Bump version
2024-09-05 18:24:51 +01:00
cybermaggedon
f7429f2f9b
Fix remaining templates ( #54 )
2024-09-05 18:17:47 +01:00
cybermaggedon
65d7f6d261
Refactor templates ( #52 )
...
* Switching from docker compose to abstract form - should be easier to k8s later
* Text loader util
* Recreate templates
2024-09-05 16:40:47 +01:00
cybermaggedon
208c219962
Template rejig ( #48 )
...
* document-rag / graph-rag refactor of templates
* Tweaking the docs and categories
* Clarify triple store vs RAG
* Tweak knowledge graph linkage
* Doc embedding for Qdrant
* Fix document RAG on Qdrant
* Fix templates
* Bump version
* Updated templates
2024-09-03 00:09:15 +01:00
cybermaggedon
121f7bb776
Updates for config util repo ( #47 )
...
* Fix pulsar manager template import
* Add pattern categories. Update parameters with new schema.
* Add more parameters to config
2024-09-01 21:29:29 +01:00
cybermaggedon
06a9e913dc
Enable model overrides, added a null option in config-loader ( #45 )
...
Enable model overrides, added a null option in config-loader
2024-08-30 21:33:03 +01:00
cybermaggedon
f7a30006ad
Make templating work more flexibly ( #44 )
...
* Restructure directory
* Config loading
* Variable override points in JSONNET templates, separate pulsar-manager template
* Bump version
* Tidy chunking
* Simplified prompt overrides
* Update config loader
* Fix recursive chunker template
2024-08-30 17:47:35 +01:00
cybermaggedon
32b087fbf6
Switch Milvus for Qdrant in YAMLs ( #43 )
...
* Qdrant working
* - Fix missing prompt templates
- Bump version
- Add Qdrant to packages
* Switch Milvus for Qdrant in config files
2024-08-27 23:37:24 +01:00
cybermaggedon
e4c4774b5d
Extract rows and apply object embeddings ( #42 )
...
* - Restructured the extract directories
- Added an extractor for 'rows' == a row of a table
- Added a row extractor prompt to prompter.
* Add row support to template prompter
* Row extraction working
* Bump version
* Emit extracted info
* Object embeddings store
* Invocation script
* Add script to package, remove cruft output
* Write rows to Cassandra
* Remove output cruft
2024-08-27 21:55:12 +01:00
cybermaggedon
669aed0f8a
Added doc embedding support ( #41 )
...
* document embedding writer & query
* Added test query for doc embeddings
* Bump version
* Added doc rag prompt
* Document RAG service
2024-08-26 23:45:23 +01:00
cybermaggedon
d69de52b04
Increase resources ( #35 )
...
* More memory for Cassandra
* More memory/CPU for embeddings
* Bump version, regenerate templates
2024-08-25 20:38:19 +01:00
cybermaggedon
6edc3f0ee1
Prompt templates ( #33 )
...
* Added prompt-template, allows definiton, relationships and kg query
to be specified in config / command-line.
* Bump version & add prompt-templates to YAMLs
* Apply to graph rag flow
* Break out different templates
2024-08-23 23:34:16 +01:00
cybermaggedon
e7c498be92
Fix neo4j ( #29 )
...
* - Fix Neo4j memory
- Fix neo4j query
* Version to 0.7.6
2024-08-23 12:49:42 +01:00
cybermaggedon
b1b26a3f55
- Updated dashboard ( #27 )
...
- Adjusted limits everything works
- Bump version
2024-08-22 23:23:11 +01:00
cybermaggedon
2bc602fdcf
Fix, was putting Neo in Cassandra config ( #26 )
2024-08-22 22:02:57 +01:00
cybermaggedon
a2ae1d8820
Generate all YAML files ( #24 )
...
* All templates generated, added missing file
* Up version
2024-08-22 21:20:17 +01:00
cybermaggedon
305dda4463
Fix errors in previous update ( #23 )
...
* Increase some limits
* Fix msg errors and update version
2024-08-22 20:58:44 +01:00
cybermaggedon
a01a72ba00
Set resource limits ( #22 )
...
* Added resource limits to resources.
* Boost version number, rebuild YAMLs
2024-08-22 17:54:00 +01:00
cybermaggedon
ada7c405df
Update dependencies ( #20 )
2024-08-22 17:01:47 +01:00
cybermaggedon
d3cdb97528
Fix auto-templates ( #17 )
2024-08-22 00:19:55 +01:00
Cyber MacGeddon
7d00a53711
Some default values in yaml file
2024-08-21 17:17:24 +01:00
cybermaggedon
0e2db095e3
Add a docker-compose for just the stores ( #13 )
...
* - Added docker-compose-storage.yaml, just the infrastructure bits
- Tidied storage invocation
* Util, sits on chunker output and reports histogram of chunk sizes
2024-08-21 16:20:21 +01:00
cybermaggedon
a38f530c5f
Rate limit handling ( #11 )
...
* Added a rate limit exception
* Reduce request/response timeouts because looks like there are major issues
* Add rate limit exception catch to all consumers
* Version to 0.6.3
2024-08-19 22:15:32 +01:00
cybermaggedon
fa0b89b5d4
Simplify templates ( #10 )
...
- Add component template files for all LLM types
- Top-level templates simplified to use just components
- Version to 0.6.2
2024-08-14 20:56:57 +01:00
cybermaggedon
d3e213f194
Add Neo4j support ( #9 )
...
- Add triples-write-neo4j and triples-query-neo4j to interact with neo4j
- Add docker-compose-openai-neo4j to demo Neo4j working
2024-08-14 09:06:33 +01:00
Cyber MacGeddon
2f72fceaa2
Tidy templates directory
2024-08-13 19:22:57 +01:00
cybermaggedon
a3ea1301d6
Breakout store queries ( #8 )
...
- Break out store queries, so not locked into a Milvus/Cassandra backend
- Break out prompting into a separate module, so that prompts can be tailored to other LLMs
- Jsonnet used to generate docker compose templates
- Version to 0.6.0
2024-08-13 17:30:59 +01:00
