apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-04-26 08:56:21 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
1122 commits 76 branches 497 tags 113 MiB
Commit graph

343 commits

Author SHA1 Message Date
cybermaggedon
f1559c5944
Feature/librarian (#310) 
* Add fields to library schema

* Added list function, incomplete

* Librarian list operation
 2025-03-11 16:52:59 +00:00
cybermaggedon
5575e885e5
Fix merge issues with api-key, conflicts with librarian changes (#309) 2025-02-15 12:25:26 +00:00
cybermaggedon
617eb7efd5
Feature/pulsar api key support (#308) 
* Add pulsar API token check

* Added missing api_key references

---------

Co-authored-by: Tyler O <4535788+toliver38@users.noreply.github.com>
 2025-02-15 11:22:48 +00:00
cybermaggedon
f7df2df266
Feature/librarian (#307) 
* Bring QDrant up-to-date

* Tables for data from queue outputs

- Pass single Pulsar client to everything in gateway & librarian
- Pulsar listener-name support in gateway
- PDF and text load working in librarian

* Complete Cassandra schema

* Add librarian support to templates
 2025-02-12 23:39:24 +00:00
cybermaggedon
f350abb415
Maint/asyncio (#305) 
* Move to asyncio services, even though everything is largely sync
 2025-02-11 23:24:46 +00:00
cybermaggedon
a0bf2362f6
Librarian (#304) 2025-02-11 16:01:03 +00:00
Tyler Oliver
e99c0ac238 Add support for Qdrant API Auth (#300) 
Added the necessary changes to support API Key in Qdrant Client Query+Storage
- Doc Embeddings
- Graph Embeddings
 2025-02-08 11:46:22 +00:00
Tyler Oliver
41ccb6c976 Add user and password auth for Cassandra (#301) 2025-02-08 11:42:14 +00:00
Cyber MacGeddon
15c8afa97b Changed setup.py for 0.21 release branch 2025-01-30 21:40:44 +00:00
cybermaggedon
6c3d2e7f97
Feature/fastembed (#286) 
* Copied in base files for fastembed

* Added fastembed implementation

* Added template support for fastembed
 2025-01-28 18:26:17 +00:00
cybermaggedon
75a72b0d2d
Fix ollama embeddings client to work (#285) 2025-01-27 23:47:15 +00:00
cybermaggedon
552637c1f7
Fix deps (#284) 
* Add cryptography module back in

* Lock in known working version of the AI modules, suspecting a problem
in sentence-transformers with Python 3.12?
 2025-01-27 21:57:22 +00:00
cybermaggedon
0ebd770cb5
Back out cryptography (#282) 2025-01-27 19:58:38 +00:00
cybermaggedon
0e03bc05a4
Refactor rate limit handling (#280) 
* - Refactored retry for rate limits into the base class
- ConsumerProducer is derived from Consumer to simplify code
- Added rate_limit_count metrics for rate limit events

* Add rate limit events to VertexAI and Google AI Studio

* Added Grafana rate limit dashboard

* Add rate limit handling to all LLMs
 2025-01-27 17:04:49 +00:00
cybermaggedon
26a586034c
Add cryptography dep (#281) 2025-01-27 17:04:09 +00:00
cybermaggedon
8fc22155dd
Agent tweaks (#272) 
- Removed unused LLM client configuration from agent-manager-react
- Change agent-manager-react template to use prompt-rag instead of
  prompt
- Changed TextCompletion tool to use 'question' instead of 'computation'
  for its parameter.
 2025-01-20 14:05:51 +00:00
Cyber MacGeddon
9bf916f3c6 Update Python files for 0.20 2025-01-16 00:02:18 +00:00
cybermaggedon
bed7423c26
- Added load/save API for document embeddings (#269) 
- tg-load-doc-embeds and tg-save-doc-embeds command line utils
 2025-01-16 00:00:59 +00:00
cybermaggedon
1280af3eff
Fix/kg core save (#267) 
* Add a 'localhost' Pulsar endpoint for docker env

* - Fix broken socket endpoint streamers
- Add unused listener endpoints for publisher/subscriber
- Fix graph embedding serialisation

* Fix GE load

* Remove Gossip settling delay, this is single-node Cassandra.
 2025-01-13 14:42:33 +00:00
cybermaggedon
c603caa3cc
Fix neo4j: container doesn't start currently (#264) 
* Adjust Neo4j memory settings to avoid startup fail

* Upgrade Neo4j to community 5.26.0

* Add Neo4j indexes
 2025-01-11 12:44:49 +00:00
cybermaggedon
dbdf65b84d
Added a metrics endpoint /api/v1/metrics/X which proxies to Prometheus (#251) 2025-01-05 19:24:43 +00:00
cybermaggedon
ae00831e39
Added missing file (#249) 2025-01-04 22:08:24 +00:00
cybermaggedon
6aa212061d
Fix/document embeddings (#247) 
* Update schema for doc embeddings

* Rename embeddings-vectorize to graph-embeddings

* Added document-embeddings processor (broken, needs fixing)

* Added scripts

* Fixed DE queue schema

* Add missing DE process

* Fix doc RAG processing, put graph-rag and doc-rag in appropriate component files.
 2025-01-04 21:51:28 +00:00
cybermaggedon
a458d57af2
Feature/refactor entity embeddings (#235) 
* Make schema changes
* Core entity context flow in place
* extract-def outputs entity contexts
* Refactored qdrant write
* Refactoring of all vector stores in place
 2024-12-30 12:53:19 +00:00
Cyber MacGeddon
cff90cada1 Prepare for 0.19 2024-12-30 10:44:33 +00:00
cybermaggedon
9a71ff6521
Fix ingested metadata (#231) 2024-12-29 20:02:12 +00:00
cybermaggedon
187b0e6581
Fix/websocket capacity increase (#230) 
* Fix invalid variable name invocation
* Fix error responses in websockets
* Increase websocket limits to 50MB max message.  Turn on Pulsar chunking by default.
 2024-12-29 18:08:12 +00:00
cybermaggedon
21e7b856c4
- Added missing document-load / text-load response on websocket (#228) 2024-12-29 11:32:10 +00:00
Cyber MacGeddon
e4fdde541e - Refactor doc load and text load to use a new ServiceSender class, 
similar to ServiceRequestor, but one-way.
- This means these two services are now available over websocket with
  document-load and text-load service IDs.
 2024-12-28 19:50:50 +00:00
cybermaggedon
7f5296feca
Refactor socket threading (#219) 
* Multiple requests can be handled in parallel.
* Refactor to fix timeout issue.
 2024-12-27 10:34:16 +00:00
cybermaggedon
f145d5c324
Add workbench-ui (#216) 2024-12-20 00:45:34 +00:00
cybermaggedon
317ae3186a
Rename /api/v1/mux to /api/v1/socket (#215) 2024-12-19 21:29:53 +00:00
cybermaggedon
03b6b45725
- Fix FalkorDB query API invocations (#214) 
- Shift FalkorDB internal web manager to be port 3010 so doesn't clash with
  Grafana.
 2024-12-19 17:32:05 +00:00
cybermaggedon
a4afff59a0
wip integrate falkordb (#211) (#213) 
Co-authored-by: Avi Avni <avi.avni@gmail.com>
 2024-12-19 16:17:07 +00:00
cybermaggedon
8c1b468eb0
Fix async error (#212) 2024-12-17 12:43:16 +00:00
cybermaggedon
07f9b1f244
From vector DB, often get dupes, which means when end up returning (#210) 
less then top_k elements.  So, fetch top_k=(2 * limit) and limit to
just (limit)
 2024-12-10 22:37:54 +00:00
cybermaggedon
cd8d0c8cbc
Graph embedding query exposed through gateway (#208) 2024-12-10 22:15:56 +00:00
cybermaggedon
8d326d34b3
Use Cosine similarity (#209) 2024-12-10 22:15:42 +00:00
Cyber MacGeddon
d6cdce8391 Open 0.18 branch 2024-12-10 22:13:10 +00:00
cybermaggedon
a714221b22
Add memgraph cypher LIMIT support (#200) 2024-12-07 00:16:52 +00:00
cybermaggedon
656dcb22a9
Feature/general websocket (#199) 
* Split API endpoint into endpoint and requestor
* Service/endpoint separation
* Call out to multiple services working
* Add ID field
* Add mux service on websocket, calls out to all services
 2024-12-06 23:56:10 +00:00
cybermaggedon
55c5c398b6
Fix relative imports (#197) 2024-12-06 13:37:44 +00:00
cybermaggedon
67d69b5285
Fixed a problem with the packages, api/__init__.py appeared in both (#196) 
trustgraph-flow and trustgraph-base, moved the gateway stuff into a
different directory.
 2024-12-06 13:05:56 +00:00
cybermaggedon
bffaf62490
Feature/memgraph optim (#193) 
* Separate memgraph query/write modules to optimise for memgraph
* Used 1GB memory for Memgraph
* Deployed specialised memgraph query/write processors, created memgraph indexes
* One triple is loaded as a single transaction
* Fixed index creation
 2024-12-06 00:12:49 +00:00
cybermaggedon
5770af51ef
Fix async problem on text load (#191) 2024-12-03 21:30:14 +00:00
cybermaggedon
26865a515c
Fix/async problem (#190) 
* Back out previous change
* To multithreads
* Remove aiopulsar dependency
 2024-12-03 18:03:00 +00:00
cybermaggedon
7e78aa6d91
Reduc pulsar connections (#189) 2024-12-03 14:13:40 +00:00
cybermaggedon
df23e29971
Add debug to endpoint (#188) 2024-12-03 09:52:00 +00:00
cybermaggedon
f24eed3023
Fix/pinecone de (#187) 
* Fix Goog AI studio settings
* Fix pinecone startup params
 2024-12-03 09:51:33 +00:00
cybermaggedon
1b9c6be4fc
Feature/gateway auth (#186) 
* Added auth module, just a simple token at this stage
* Pass auth token GATEWAY_SECRET through
* Auth token not mandatory, can be provided in env var
 2024-12-02 19:57:21 +00:00