apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-07-06 03:42:11 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Calling Cohere but not forming triples

Browse source
This commit is contained in:
JackColquitt 2024-08-01 13:25:09 -07:00
parent 7f08d1e3a2
commit 837f41f010
7 changed files with 21 additions and 498 deletions
Download patch file Download diff file Expand all files Collapse all files

102
.ipynb_checkpoints/README-checkpoint.md
View file

@ -1,102 +0,0 @@
# TrustGraph
## Introduction
TrustGraph is a true end-to-end (e2e) knowledge pipeline that performs a `naive extraction` on a text corpus
to build a RDF style knowledge graph coupled with a `RAG` service compatible with cloud LLMs and open-source
SLMs (Small Language Models).
The pipeline processing components are interconnected with a pub/sub engine to
maximize modularity and enable new knowledge processing functions. The core processing components decode documents,
chunk text, perform embeddings, apply a local SLM/LLM, call a LLM API, and generate LM predictions.
The processing showcases the reliability and efficiences of Graph RAG algorithms which can capture
contextual language flags that are missed in conventional RAG approaches. Graph querying algorithms enable retrieving
not just relevant knowledge but language cues essential to understanding semantic uses unique to a text corpus.
Processing modules are executed in containers. Processing can be scaled-up
by deploying multiple containers.
### Features
- PDF decoding
- Text chunking
- Inference of LMs deployed with [Ollama](https://ollama.com)
- Inference of LLMs: Claude, VertexAI and AzureAI serverless endpoints
- Application of a [HuggingFace](https://hf.co) embeddings models
- [RDF](https://www.w3.org/TR/rdf12-schema/)-aligned Knowledge Graph extraction
- Graph edge loading into [Apache Cassandra](https://github.com/apache/cassandra)
- Storing embeddings in [Milvus](https://github.com/milvus-io/milvus)
- Embedding query service
- Graph RAG query service
- All procesing integrates with [Apache Pulsar](https://github.com/apache/pulsar/)
- Containers, so can be deployed using Docker Compose or Kubernetes
- Plug'n'play architecture: switch different LLM modules to suit your needs
## Architecture
![architecture](architecture.png)
TrustGraph is designed to be modular to support as many Language Models and environments as possible. A natural
fit for a modular architecture is to decompose functions into a set modules connected through a pub/sub backbone.
[Apache Pulsar](https://github.com/apache/pulsar/) serves as this pub/sub backbone. Pulsar acts as the data broker
managing inputs and outputs between modules.
**Pulsar Workflows**:
- For processing flows, Pulsar accepts the output of a processing module
and queues it for input to the next subscribed module.
- For services such as LLMs and embeddings, Pulsar provides a client/server
model. A Pulsar queue is used as the input to the service. When
processed, the output is then delivered to a separate queue where a client
subscriber can request that output.
The entire architecture, the pub/sub backbone and set of modules, is bundled into a single Python package. A container image with the
package installed can also run the entire architecture.
## Core Modules
- `chunker-recursive` - Accepts text documents and uses LangChain recursive
chunking algorithm to produce smaller text chunks.
- `embeddings-hf` - A service which analyses text and returns a vector
embedding using one of the HuggingFace embeddings models.
- `embeddings-vectorize` - Uses an embeddings service to get a vector
embedding which is added to the processor payload.
- `graph-rag` - A query service which applies a Graph RAG algorithm to
provide a response to a text prompt.
- `graph-write-cassandra` - Takes knowledge graph edges and writes them to
a Cassandra store.
- `kg-extract-definitions` - knowledge extractor - examines text and
produces graph edges.
describing discovered terms and also their defintions. Definitions are
derived using the input documents.
- `kg-extract-relationships` - knowledge extractor - examines text and
produces graph edges describing the relationships between discovered
terms.
- `loader` - Takes a document and loads into the processing pipeline. Used
e.g. to add PDF documents.
- `pdf-decoder` - Takes a PDF doc and emits text extracted from the document.
Text extraction from PDF is not a perfect science as PDF is a printable
format. For instance, the wrapping of text between lines in a PDF document
is not semantically encoded, so the decoder will see wrapped lines as
space-separated.
- `vector-write-milvus` - Takes vector-entity mappings and records them
in the vector embeddings store.
## LM Specific Modules
- `llm-azure-text` - Sends request to AzureAI serverless endpoint
- `llm-claude-text` - Sends request to Anthropic's API
- `llm-ollama-text` - Sends request to LM running using Ollama
- `llm-vertexai-text` - Sends request to model available through VertexAI API
## Quickstart Guide
See [Quickstart on Docker Compose](docs/README.quickstart-docker-compose.md)
## Development Guide
See [Development on trustgraph](docs/README.development.md)

179
.ipynb_checkpoints/docker-compose-azure-checkpoint.yaml
View file

@ -1,179 +0,0 @@
volumes:
cassandra:
pulsar-conf:
pulsar-data:
etcd:
minio-data:
milvus:
services:
cassandra:
image: docker.io/cassandra:4.1.5
ports:
- "9042:9042"
volumes:
- "cassandra:/var/lib/cassandra"
restart: on-failure:100
pulsar:
image: docker.io/apachepulsar/pulsar:3.3.0
command: bin/pulsar standalone
ports:
- "6650:6650"
- "8080:8080"
volumes:
- "pulsar-conf:/pulsar/conf"
- "pulsar-data:/pulsar/data"
restart: on-failure:100
pulsar-manager:
image: docker.io/apachepulsar/pulsar-manager:v0.3.0
ports:
- "9527:9527"
- "7750:7750"
environment:
SPRING_CONFIGURATION_FILE: /pulsar-manager/pulsar-manager/application.properties
restart: on-failure:100
etcd:
image: quay.io/coreos/etcd:v3.5.5
command:
- "etcd"
- "-advertise-client-urls=http://127.0.0.1:2379"
- "-listen-client-urls"
- "http://0.0.0.0:2379"
- "--data-dir"
- "/etcd"
environment:
ETCD_AUTO_COMPACTION_MODE: revision
ETCD_AUTO_COMPACTION_RETENTION: "1000"
ETCD_QUOTA_BACKEND_BYTES: "4294967296"
ETCD_SNAPSHOT_COUNT: "50000"
ports:
- "2379:2379"
volumes:
- "etcd:/etcd"
restart: on-failure:100
minio:
image: docker.io/minio/minio:RELEASE.2024-07-04T14-25-45Z
command:
- "minio"
- "server"
- "/minio_data"
- "--console-address"
- ":9001"
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
ports:
- "9001:9001"
volumes:
- "minio-data:/minio_data"
restart: on-failure:100
milvus:
image: docker.io/milvusdb/milvus:v2.4.5
command:
- "milvus"
- "run"
- "standalone"
environment:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
ports:
- "9091:9091"
- "19530:19530"
volumes:
- "milvus:/var/lib/milvus"
restart: on-failure:100
pdf-decoder:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "pdf-decoder"
- "-p"
- "pulsar://pulsar:6650"
restart: on-failure:100
chunker:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "chunker-recursive"
- "-p"
- "pulsar://pulsar:6650"
restart: on-failure:100
vectorize:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "embeddings-vectorize"
- "-p"
- "pulsar://pulsar:6650"
restart: on-failure:100
embeddings:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "embeddings-hf"
- "-p"
- "pulsar://pulsar:6650"
restart: on-failure:100
kg-extract-definitions:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "kg-extract-definitions"
- "-p"
- "pulsar://pulsar:6650"
restart: on-failure:100
kg-extract-relationships:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "kg-extract-relationships"
- "-p"
- "pulsar://pulsar:6650"
restart: on-failure:100
vector-write:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "vector-write-milvus"
- "-p"
- "pulsar://pulsar:6650"
- "-t"
- "http://milvus:19530"
restart: on-failure:100
graph-write:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "graph-write-cassandra"
- "-p"
- "pulsar://pulsar:6650"
- "-g"
- "cassandra"
restart: on-failure:100
llm:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "llm-azure-text"
- "-p"
- "pulsar://pulsar:6650"
- "-k"
- ${AZURE_TOKEN}
- "-e"
- ${AZURE_ENDPOINT}
restart: on-failure:100
graph-rag:
image: docker.io/trustgraph/trustgraph-flow:0.1.16
command:
- "graph-rag"
- "-p"
- "pulsar://pulsar:6650"
restart: on-failure:100

4
docker-compose-cohere.yaml
View file

@ -143,6 +143,10 @@ services:
- "chunker-recursive"
- "-p"
- "pulsar://pulsar:6650"
- "--chunk-size"
- "1000"
- "--chunk-overlap"
- "50"
restart: on-failure:100
vectorize:

71
trustgraph/.ipynb_checkpoints/llm_client-checkpoint.py
View file

@ -1,71 +0,0 @@
#!/usr/bin/env python3
import pulsar
import _pulsar
from pulsar.schema import JsonSchema
from trustgraph.schema import TextCompletionRequest, TextCompletionResponse
import hashlib
import uuid
# Ugly
ERROR=_pulsar.LoggerLevel.Error
WARN=_pulsar.LoggerLevel.Warn
INFO=_pulsar.LoggerLevel.Info
DEBUG=_pulsar.LoggerLevel.Debug
class LlmClient:
def __init__(
self, log_level=ERROR, client_id=None,
pulsar_host="pulsar://pulsar:6650",
):
if client_id == None:
client_id = str(uuid.uuid4())
self.client = pulsar.Client(
pulsar_host,
logger=pulsar.ConsoleLogger(log_level),
)
self.producer = self.client.create_producer(
topic='llm-complete-text',
schema=JsonSchema(TextCompletionRequest),
chunking_enabled=True,
)
self.consumer = self.client.subscribe(
'llm-complete-text-response', client_id,
schema=JsonSchema(TextCompletionResponse),
)
def request(self, prompt, timeout=500):
id = str(uuid.uuid4())
r = TextCompletionRequest(
prompt=prompt
)
self.producer.send(r, properties={ "id": id })
while True:
msg = self.consumer.receive(timeout_millis=timeout * 1000)
mid = msg.properties()["id"]
if mid == id:
resp = msg.value().response
self.consumer.acknowledge(msg)
return resp
# Ignore messages with wrong ID
self.consumer.acknowledge(msg)
def __del__(self):
self.producer.close()
self.consumer.close()
self.client.close()

138
trustgraph/.ipynb_checkpoints/prompts-checkpoint.py
View file

@ -1,138 +0,0 @@
def turtle_extract(text):
prompt = f"""<instructions>
Study the following text and extract knowledge as
information in Turtle RDF format.
When declaring any new URIs, use <https://trustgraph.ai/e#> prefix,
and declare appropriate namespace tags.
</instructions>
<text>
{text}
</text>
<requirements>
Do not use placeholders for information you do not know.
You will respond only with raw Turtle RDF data. Do not provide
explanations. Do not use special characters in the abstract text. The
abstract must be written as plain text. Do not add markdown formatting.
</requirements>"""
return prompt
def scholar(text):
# Build the prompt for Article style extraction
jsonexample = """{
"title": "Article title here",
"abstract": "Abstract text here",
"keywords": ["keyword1", "keyword2", "keyword3"],
"people": ["person1", "person2", "person3"]
}"""
promptscholar = f"""Your task is to read the provided text and write a scholarly abstract to fully explain all of the concepts described in the provided text. The abstract must include all conceptual details.
<text>
{text}
</text>
<instructions>
- Structure: For the provided text, write a title, abstract, keywords,
and people for the concepts found in the provided text. Ignore
document formatting in the provided text such as table of contents,
headers, footers, section metadata, and URLs.
- Focus on Concepts The abstract must focus on concepts found in the
provided text. The abstract must be factually accurate. Do not
write any concepts not found in the provided text. Do not
speculate. Do not omit any conceptual details.
- Completeness: The abstract must capture all topics the reader will
need to understand the concepts found in the provided text. Describe
all terms, definitions, entities, people, events, concepts,
conceptual relationships, and any other topics necessary for the
reader to understand the concepts of the provided text.
- Format: Respond in the form of a valid JSON object.
</instructions>
<example>
{jsonexample}
</example>
<requirements>
You will respond only with the JSON object. Do not provide
explanations. Do not use special characters in the abstract text. The
abstract must be written as plain text.
</requirements>"""
return promptscholar
def to_json_ld(text):
prompt = f"""<instructions>
Study the following text and output any facts you discover in
well-structured JSON-LD format.
Use any schema you understand from schema.org to describe the facts.
</instructions>
<text>
{text}
</text>
<requirements>
You will respond only with raw JSON-LD data in JSON format. Do not provide
explanations. Do not use special characters in the abstract text. The
abstract must be written as plain text. Do not add markdown formatting
or headers or prefixes. Do not use information which is not present in
the input text.
</requirements>"""
return prompt
def to_relationships(text):
prompt = f"""<instructions>
Study the following text and derive entity relationships. For each
relationship, derive the subject, predicate and object of the relationship.
Output relationships in JSON format as an arary of objects with fields:
- subject: the subject of the relationship
- predicate: the predicate
- object: the object of the relationship
- object-entity: false if the object is a simple data type: name, value or date. true if it is an entity.
</instructions>
<text>
{text}
</text>
<requirements>
You will respond only with raw JSON format data. Do not provide
explanations. Do not use special characters in the abstract text. The
abstract must be written as plain text. Do not add markdown formatting
or headers or prefixes.
</requirements>"""
return prompt
def to_definitions(text):
prompt = f"""<instructions>
Study the following text and derive definitions for any discovered entities.
Do not provide definitions for entities whose definitions are incomplete
or unknown.
Output relationships in JSON format as an arary of objects with fields:
- entity: the name of the entity
- definition: English text which defines the entity
</instructions>
<text>
{text}
</text>
<requirements>
You will respond only with raw JSON format data. Do not provide
explanations. Do not use special characters in the abstract text. The
abstract will be written as plain text. Do not add markdown formatting
or headers or prefixes. Do not include null or unknown definitions.
</requirements>"""
return prompt

6
trustgraph/.ipynb_checkpoints/rdf-checkpoint.py
View file

@ -1,6 +0,0 @@
RDF_LABEL = "http://www.w3.org/2000/01/rdf-schema#label"
DEFINITION = "http://www.w3.org/2004/02/skos/core#definition"
TRUSTGRAPH_ENTITIES = "http://trustgraph.ai/e/"

19
trustgraph/model/text_completion/cohere/llm.py
View file

@ -5,6 +5,7 @@ Input is prompt, output is response.
"""
import cohere
import re
from .... schema import TextCompletionRequest, TextCompletionResponse
from .... schema import text_completion_request_queue
@ -62,6 +63,7 @@ class Processor(ConsumerProducer):
model=self.model,
#model='c4ai-aya-23-8b',
message=prompt,
preamble = "You are an AI-assistant chatbot. You are trained to read text and find entities in that text. You respond only with well-formed JSON.",
temperature=0.0,
chat_history=[],
prompt_truncation='auto',
@ -72,9 +74,22 @@ class Processor(ConsumerProducer):
if event.event_type == "text-generation":
resp = event.text
print(resp, flush=True)
# Parse output for ```json``` delimiters
pattern = r'```json\s*([\s\S]*?)\s*```'
match = re.search(pattern, resp)
if match:
# If delimiters are found, extract the JSON content
json_content = match.group(1)
json_resp = json_content.strip()
else:
# If no delimiters are found, return the original text
json_resp = resp.strip()
print("Send response...", flush=True)
r = TextCompletionResponse(response=resp)
r = TextCompletionResponse(response=json_resp)
self.send(r, properties={"id": id})
print("Done.", flush=True)