75 commits

Author	SHA1	Message	Date
cybermaggedon	1280af3eff	Fix/kg core save (#267) ... * Add a 'localhost' Pulsar endpoint for docker env * - Fix broken socket endpoint streamers - Add unused listener endpoints for publisher/subscriber - Fix graph embedding serialisation * Fix GE load * Remove Gossip settling delay, this is single-node Cassandra.	2025-01-13 14:42:33 +00:00
cybermaggedon	c603caa3cc	Fix neo4j: container doesn't start currently (#264) ... * Adjust Neo4j memory settings to avoid startup fail * Upgrade Neo4j to community 5.26.0 * Add Neo4j indexes	2025-01-11 12:44:49 +00:00
cybermaggedon	dbdf65b84d	Added a metrics endpoint /api/v1/metrics/X which proxies to Prometheus (#251)	2025-01-05 19:24:43 +00:00
cybermaggedon	ae00831e39	Added missing file (#249)	2025-01-04 22:08:24 +00:00
cybermaggedon	6aa212061d	Fix/document embeddings (#247) ... * Update schema for doc embeddings * Rename embeddings-vectorize to graph-embeddings * Added document-embeddings processor (broken, needs fixing) * Added scripts * Fixed DE queue schema * Add missing DE process * Fix doc RAG processing, put graph-rag and doc-rag in appropriate component files.	2025-01-04 21:51:28 +00:00
cybermaggedon	a458d57af2	Feature/refactor entity embeddings (#235) ... * Make schema changes * Core entity context flow in place * extract-def outputs entity contexts * Refactored qdrant write * Refactoring of all vector stores in place	2024-12-30 12:53:19 +00:00
Cyber MacGeddon	cff90cada1	Prepare for 0.19	2024-12-30 10:44:33 +00:00
cybermaggedon	9a71ff6521	Fix ingested metadata (#231)	2024-12-29 20:02:12 +00:00
cybermaggedon	187b0e6581	Fix/websocket capacity increase (#230) ... * Fix invalid variable name invocation * Fix error responses in websockets * Increase websocket limits to 50MB max message. Turn on Pulsar chunking by default.	2024-12-29 18:08:12 +00:00
cybermaggedon	21e7b856c4	- Added missing document-load / text-load response on websocket (#228)	2024-12-29 11:32:10 +00:00
Cyber MacGeddon	e4fdde541e	- Refactor doc load and text load to use a new ServiceSender class, ... similar to ServiceRequestor, but one-way. - This means these two services are now available over websocket with document-load and text-load service IDs.	2024-12-28 19:50:50 +00:00
cybermaggedon	7f5296feca	Refactor socket threading (#219) ... * Multiple requests can be handled in parallel. * Refactor to fix timeout issue.	2024-12-27 10:34:16 +00:00
cybermaggedon	f145d5c324	Add workbench-ui (#216)	2024-12-20 00:45:34 +00:00
cybermaggedon	317ae3186a	Rename /api/v1/mux to /api/v1/socket (#215)	2024-12-19 21:29:53 +00:00
cybermaggedon	03b6b45725	- Fix FalkorDB query API invocations (#214) ... - Shift FalkorDB internal web manager to be port 3010 so doesn't clash with Grafana.	2024-12-19 17:32:05 +00:00
cybermaggedon	a4afff59a0	wip integrate falkordb (#211) (#213) ... Co-authored-by: Avi Avni <avi.avni@gmail.com>	2024-12-19 16:17:07 +00:00
cybermaggedon	8c1b468eb0	Fix async error (#212)	2024-12-17 12:43:16 +00:00
cybermaggedon	07f9b1f244	From vector DB, often get dupes, which means when end up returning (#210) ... less then top_k elements. So, fetch top_k=(2 * limit) and limit to just (limit)	2024-12-10 22:37:54 +00:00
cybermaggedon	cd8d0c8cbc	Graph embedding query exposed through gateway (#208)	2024-12-10 22:15:56 +00:00
cybermaggedon	8d326d34b3	Use Cosine similarity (#209)	2024-12-10 22:15:42 +00:00
Cyber MacGeddon	d6cdce8391	Open 0.18 branch	2024-12-10 22:13:10 +00:00
cybermaggedon	a714221b22	Add memgraph cypher LIMIT support (#200)	2024-12-07 00:16:52 +00:00
cybermaggedon	656dcb22a9	Feature/general websocket (#199) ... * Split API endpoint into endpoint and requestor * Service/endpoint separation * Call out to multiple services working * Add ID field * Add mux service on websocket, calls out to all services	2024-12-06 23:56:10 +00:00
cybermaggedon	55c5c398b6	Fix relative imports (#197)	2024-12-06 13:37:44 +00:00
cybermaggedon	67d69b5285	Fixed a problem with the packages, api/__init__.py appeared in both (#196) ... trustgraph-flow and trustgraph-base, moved the gateway stuff into a different directory.	2024-12-06 13:05:56 +00:00
cybermaggedon	bffaf62490	Feature/memgraph optim (#193) ... * Separate memgraph query/write modules to optimise for memgraph * Used 1GB memory for Memgraph * Deployed specialised memgraph query/write processors, created memgraph indexes * One triple is loaded as a single transaction * Fixed index creation	2024-12-06 00:12:49 +00:00
cybermaggedon	5770af51ef	Fix async problem on text load (#191)	2024-12-03 21:30:14 +00:00
cybermaggedon	26865a515c	Fix/async problem (#190) ... * Back out previous change * To multithreads * Remove aiopulsar dependency	2024-12-03 18:03:00 +00:00
cybermaggedon	7e78aa6d91	Reduc pulsar connections (#189)	2024-12-03 14:13:40 +00:00
cybermaggedon	df23e29971	Add debug to endpoint (#188)	2024-12-03 09:52:00 +00:00
cybermaggedon	f24eed3023	Fix/pinecone de (#187) ... * Fix Goog AI studio settings * Fix pinecone startup params	2024-12-03 09:51:33 +00:00
cybermaggedon	1b9c6be4fc	Feature/gateway auth (#186) ... * Added auth module, just a simple token at this stage * Pass auth token GATEWAY_SECRET through * Auth token not mandatory, can be provided in env var	2024-12-02 19:57:21 +00:00
cybermaggedon	6d200c79c5	Feature/wikipedia ddg (#185) ... API-side support for Wikipedia, DBpedia and internet search functions This incorporates a refactor of the API code to break it up, separate classes for endpoints to reduce duplication	2024-12-02 17:41:30 +00:00
Cyber MacGeddon	c844d805e5	Setup for release 0.17 branch	2024-11-29 17:03:31 +00:00
cybermaggedon	9c97ca32f6	Feature/memgraph (#182) ... * Add database override to bolt output, default is neo4j * Add memgraph templates	2024-11-28 19:21:28 +00:00
cybermaggedon	b2f7b34529	Fix/pinecone integration (#180) ... * Add missing pinecone references * Add missing Pinecone executables	2024-11-27 17:36:15 +00:00
cybermaggedon	2f7ccb2ef8	- Reduce back-pressure on tg-load-kg-core (#179) ... - Save entity in correct format in tg-save-core	2024-11-27 17:24:27 +00:00
cybermaggedon	887fafcf8c	Fix/core save api (#172) ... * Acknowledge messaages from Pulsar, doh! * Change API to deliver a boolean e if value is an entity * Change loaders to use new API * Changes, entity-aware API is complete	2024-11-26 16:46:38 +00:00
cybermaggedon	340d7a224f	Feature/rework kg core (#171) ... * Knowledge cores with msgpack * Put it in the cli package * Tidy up msgpack dumper * Created a loader	2024-11-25 20:46:35 +00:00
cybermaggedon	319f9ac04a	Feature/pinecone integration (#170) ... * Added Pinecone for GE write & query * Add templates * Doc embedding support	2024-11-22 23:48:21 +00:00
cybermaggedon	7a64385a57	Fix graph query in Cassandra (#168)	2024-11-21 18:02:49 +00:00
cybermaggedon	dc0f54f236	API supports doc & text load (#167)	2024-11-21 14:53:53 +00:00
cybermaggedon	a1e0edd96f	API gateway in a proper module, restarting publishers & subscribers as appropriate (#166)	2024-11-20 23:17:55 +00:00
cybermaggedon	ba6d6c13af	Fix API gateway integration, added to templates (#165)	2024-11-20 20:56:23 +00:00
cybermaggedon	92b84441eb	Feature/api gateway (#164) ... * Bare bones API gateway * Working for LLM + prompt * RAG query works * Triples query * Added agent API * Embeddings API * Put API tests in a subdir	2024-11-20 19:55:40 +00:00
Cyber MacGeddon	b536d78b57	Prepare for 0.16: Change Python dep restrictions and Gitlab merge criteria	2024-11-20 19:55:05 +00:00
cybermaggedon	f2c78b701e	- Increased max-iterations to 15 default, and made it a configurable (#163) ... parameter with --max-iterations. Was previously hard-coded at 10. - Fixed arg passing causing pull the wrong tool name	2024-11-19 21:28:47 +00:00
cybermaggedon	0253281727	Confusion between OPENAI_KEY and OPENAI_TOKEN: settled on OPENAI_TOKEN. (#161)	2024-11-15 18:54:54 +00:00
cybermaggedon	081dd697c4	- Moved the agent-react template from hard-coded in agent_manager.py to (#151) ... a template prompt managed by prompt manager. The prompt is called agent-react.	2024-11-12 19:04:15 +00:00
cybermaggedon	f654d72515	Fix setup deps (#149)	2024-11-11 22:29:03 +00:00