Calling Cohere but not forming triples

.ipynb_checkpoints/README-checkpoint.md

@@ -1,102 +0,0 @@
-
-# TrustGraph
-
-## Introduction
-
-TrustGraph is a true end-to-end (e2e) knowledge pipeline that performs a `naive extraction` on a text corpus
-to build a RDF style knowledge graph coupled with a `RAG` service compatible with cloud LLMs and open-source
-SLMs (Small Language Models).
-
-The pipeline processing components are interconnected with a pub/sub engine to
-maximize modularity and enable new knowledge processing functions. The core processing components decode documents, 
-chunk text, perform embeddings, apply a local SLM/LLM, call a LLM API, and generate LM predictions.
-
-The processing showcases the reliability and efficiences of Graph RAG algorithms which can capture
-contextual language flags that are missed in conventional RAG approaches. Graph querying algorithms enable retrieving
-not just relevant knowledge but language cues essential to understanding semantic uses unique to a text corpus.
-
-Processing modules are executed in containers.  Processing can be scaled-up
-by deploying multiple containers.
-
-### Features
-
-- PDF decoding
-- Text chunking
-- Inference of LMs deployed with [Ollama](https://ollama.com)
-- Inference of LLMs: Claude, VertexAI and AzureAI serverless endpoints
-- Application of a [HuggingFace](https://hf.co) embeddings models
-- [RDF](https://www.w3.org/TR/rdf12-schema/)-aligned Knowledge Graph extraction
-- Graph edge loading into [Apache Cassandra](https://github.com/apache/cassandra)
-- Storing embeddings in [Milvus](https://github.com/milvus-io/milvus)
-- Embedding query service
-- Graph RAG query service
-- All procesing integrates with [Apache Pulsar](https://github.com/apache/pulsar/)
-- Containers, so can be deployed using Docker Compose or Kubernetes
-- Plug'n'play architecture: switch different LLM modules to suit your needs
-
-## Architecture
-
-![architecture](architecture.png)
-
-TrustGraph is designed to be modular to support as many Language Models and environments as possible. A natural
-fit for a modular architecture is to decompose functions into a set modules connected through a pub/sub backbone.
-[Apache Pulsar](https://github.com/apache/pulsar/) serves as this pub/sub backbone. Pulsar acts as the data broker
-managing inputs and outputs between modules.
-
-**Pulsar Workflows**:
-- For processing flows, Pulsar accepts the output of a processing module
-  and queues it for input to the next subscribed module.
-- For services such as LLMs and embeddings, Pulsar provides a client/server
-  model.  A Pulsar queue is used as the input to the service.  When
-  processed, the output is then delivered to a separate queue where a client
-  subscriber can request that output.
-
-The entire architecture, the pub/sub backbone and set of modules, is bundled into a single Python package. A container image with the
-package installed can also run the entire architecture.
-
-## Core Modules
-
-- `chunker-recursive` - Accepts text documents and uses LangChain recursive
-  chunking algorithm to produce smaller text chunks.
-- `embeddings-hf` - A service which analyses text and returns a vector
-  embedding using one of the HuggingFace embeddings models.
-- `embeddings-vectorize` - Uses an embeddings service to get a vector
-  embedding which is added to the processor payload.
-- `graph-rag` - A query service which applies a Graph RAG algorithm to
-  provide a response to a text prompt.
-- `graph-write-cassandra` - Takes knowledge graph edges and writes them to
-  a Cassandra store.
-- `kg-extract-definitions` - knowledge extractor - examines text and
-  produces graph edges.
-  describing discovered terms and also their defintions.  Definitions are
-  derived using the input  documents.
-- `kg-extract-relationships` - knowledge extractor - examines text and
-  produces graph edges describing the relationships between discovered
-  terms.
-- `loader` - Takes a document and loads into the processing pipeline.  Used
-  e.g. to add PDF documents.
-- `pdf-decoder` - Takes a PDF doc and emits text extracted from the document.
-  Text extraction from PDF is not a perfect science as PDF is a printable
-  format.  For instance, the wrapping of text between lines in a PDF document
-  is not semantically encoded, so the decoder will see wrapped lines as
-  space-separated.
-- `vector-write-milvus` - Takes vector-entity mappings and records them
-  in the vector embeddings store.
-
-## LM Specific Modules
-
-- `llm-azure-text` - Sends request to AzureAI serverless endpoint
-- `llm-claude-text` - Sends request to Anthropic's API
-- `llm-ollama-text` -  Sends request to LM running using Ollama
-- `llm-vertexai-text` -  Sends request to model available through VertexAI API
-
-## Quickstart Guide
-
-See [Quickstart on Docker Compose](docs/README.quickstart-docker-compose.md)
-
-## Development Guide
-
-See [Development on trustgraph](docs/README.development.md)
-
-
-

.ipynb_checkpoints/docker-compose-azure-checkpoint.yaml

@@ -1,179 +0,0 @@
-
-volumes:
-  cassandra:
-  pulsar-conf:
-  pulsar-data:
-  etcd:
-  minio-data:
-  milvus:
-
-services:
-
-  cassandra:
-    image: docker.io/cassandra:4.1.5
-    ports:
-      - "9042:9042"
-    volumes:
-      - "cassandra:/var/lib/cassandra"
-    restart: on-failure:100
-
-  pulsar:
-    image: docker.io/apachepulsar/pulsar:3.3.0
-    command: bin/pulsar standalone
-    ports:
-      - "6650:6650"
-      - "8080:8080"
-    volumes:
-      - "pulsar-conf:/pulsar/conf"
-      - "pulsar-data:/pulsar/data"
-    restart: on-failure:100
-
-  pulsar-manager:
-    image: docker.io/apachepulsar/pulsar-manager:v0.3.0
-    ports:
-      - "9527:9527"
-      - "7750:7750"
-    environment:
-      SPRING_CONFIGURATION_FILE: /pulsar-manager/pulsar-manager/application.properties
-    restart: on-failure:100
-
-  etcd:
-    image: quay.io/coreos/etcd:v3.5.5
-    command:
-      - "etcd"
-      - "-advertise-client-urls=http://127.0.0.1:2379"
-      - "-listen-client-urls"
-      - "http://0.0.0.0:2379"
-      - "--data-dir"
-      - "/etcd"
-    environment:
-      ETCD_AUTO_COMPACTION_MODE: revision
-      ETCD_AUTO_COMPACTION_RETENTION: "1000"
-      ETCD_QUOTA_BACKEND_BYTES: "4294967296"
-      ETCD_SNAPSHOT_COUNT: "50000"
-    ports:
-      - "2379:2379"
-    volumes:
-      - "etcd:/etcd"
-    restart: on-failure:100
-    
-  minio:
-    image: docker.io/minio/minio:RELEASE.2024-07-04T14-25-45Z
-    command:
-      - "minio"
-      - "server"
-      - "/minio_data"
-      - "--console-address"
-      - ":9001"
-    environment:
-      MINIO_ROOT_USER: minioadmin
-      MINIO_ROOT_PASSWORD: minioadmin
-    ports:
-      - "9001:9001"
-    volumes:
-      - "minio-data:/minio_data"
-    restart: on-failure:100
-
-  milvus:
-    image: docker.io/milvusdb/milvus:v2.4.5
-    command:
-      - "milvus"
-      - "run"
-      - "standalone"
-    environment:
-      ETCD_ENDPOINTS: etcd:2379
-      MINIO_ADDRESS: minio:9000
-    ports:
-      - "9091:9091"
-      - "19530:19530"
-    volumes:
-      - "milvus:/var/lib/milvus"
-    restart: on-failure:100
-
-  pdf-decoder:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "pdf-decoder"
-      - "-p"
-      - "pulsar://pulsar:6650"
-    restart: on-failure:100
-
-  chunker:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "chunker-recursive"
-      - "-p"
-      - "pulsar://pulsar:6650"
-    restart: on-failure:100
-
-  vectorize:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "embeddings-vectorize"
-      - "-p"
-      - "pulsar://pulsar:6650"
-    restart: on-failure:100
-    
-  embeddings:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "embeddings-hf"
-      - "-p"
-      - "pulsar://pulsar:6650"
-    restart: on-failure:100
-    
-  kg-extract-definitions:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "kg-extract-definitions"
-      - "-p"
-      - "pulsar://pulsar:6650"
-    restart: on-failure:100
-    
-  kg-extract-relationships:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "kg-extract-relationships"
-      - "-p"
-      - "pulsar://pulsar:6650"
-    restart: on-failure:100
-    
-  vector-write:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "vector-write-milvus"
-      - "-p"
-      - "pulsar://pulsar:6650"
-      - "-t"
-      - "http://milvus:19530"
-    restart: on-failure:100
-    
-  graph-write:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "graph-write-cassandra"
-      - "-p"
-      - "pulsar://pulsar:6650"
-      - "-g"
-      - "cassandra"
-    restart: on-failure:100
-    
-  llm:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "llm-azure-text"
-      - "-p"
-      - "pulsar://pulsar:6650"
-      - "-k"
-      - ${AZURE_TOKEN}
-      - "-e"
-      - ${AZURE_ENDPOINT}
-    restart: on-failure:100
-
-  graph-rag:
-    image: docker.io/trustgraph/trustgraph-flow:0.1.16
-    command:
-      - "graph-rag"
-      - "-p"
-      - "pulsar://pulsar:6650"
-    restart: on-failure:100

docker-compose-cohere.yaml

@@ -143,6 +143,10 @@ services:
       - "chunker-recursive"
       - "-p"
       - "pulsar://pulsar:6650"
+      - "--chunk-size"
+      - "1000"
+      - "--chunk-overlap"
+      - "50"
     restart: on-failure:100
 
   vectorize:

trustgraph/.ipynb_checkpoints/llm_client-checkpoint.py

@@ -1,71 +0,0 @@
-#!/usr/bin/env python3
-
-import pulsar
-import _pulsar
-from pulsar.schema import JsonSchema
-from trustgraph.schema import TextCompletionRequest, TextCompletionResponse
-import hashlib
-import uuid
-
-# Ugly
-ERROR=_pulsar.LoggerLevel.Error
-WARN=_pulsar.LoggerLevel.Warn
-INFO=_pulsar.LoggerLevel.Info
-DEBUG=_pulsar.LoggerLevel.Debug
-
-class LlmClient:
-
-    def __init__(
-            self, log_level=ERROR, client_id=None,
-            pulsar_host="pulsar://pulsar:6650",
-    ):
-
-        if client_id == None:
-            client_id = str(uuid.uuid4())
-
-        self.client = pulsar.Client(
-            pulsar_host,
-            logger=pulsar.ConsoleLogger(log_level),
-        )
-
-        self.producer = self.client.create_producer(
-            topic='llm-complete-text',
-            schema=JsonSchema(TextCompletionRequest),
-            chunking_enabled=True,
-        )
-
-        self.consumer = self.client.subscribe(
-            'llm-complete-text-response', client_id,
-            schema=JsonSchema(TextCompletionResponse),
-        )
-
-    def request(self, prompt, timeout=500):
-
-        id = str(uuid.uuid4())
-
-        r = TextCompletionRequest(
-            prompt=prompt
-        )
-
-        self.producer.send(r, properties={ "id": id })
-
-        while True:
-
-            msg = self.consumer.receive(timeout_millis=timeout * 1000)
-
-            mid = msg.properties()["id"]
-
-            if mid == id:
-                resp = msg.value().response
-                self.consumer.acknowledge(msg)
-                return resp
-
-            # Ignore messages with wrong ID
-            self.consumer.acknowledge(msg)
-
-    def __del__(self):
-
-        self.producer.close()
-        self.consumer.close()
-        self.client.close()
-

trustgraph/.ipynb_checkpoints/prompts-checkpoint.py

@@ -1,138 +0,0 @@
-
-def turtle_extract(text):
-
-    prompt = f"""<instructions>
-Study the following text and extract knowledge as
-information in Turtle RDF format.
-When declaring any new URIs, use <https://trustgraph.ai/e#> prefix,
-and declare appropriate namespace tags.
-</instructions>
-    
-<text>
-{text}
-</text>
-    
-<requirements>
-Do not use placeholders for information you do not know.
-You will respond only with raw Turtle RDF data. Do not provide
-explanations. Do not use special characters in the abstract text. The
-abstract must be written as plain text.  Do not add markdown formatting.
-</requirements>"""
-
-    return prompt
-
-def scholar(text):
-
-    # Build the prompt for Article style extraction
-    jsonexample = """{
-    "title": "Article title here",
-    "abstract": "Abstract text here",
-    "keywords": ["keyword1", "keyword2", "keyword3"],
-    "people": ["person1", "person2", "person3"]
-}"""
-    
-    promptscholar = f"""Your task is to read the provided text and write a scholarly abstract to fully explain all of the concepts described in the provided text. The abstract must include all conceptual details.
-<text>
-{text}
-</text>
-<instructions>
-
-- Structure: For the provided text, write a title, abstract, keywords,
-  and people for the concepts found in the provided text. Ignore
-  document formatting in the provided text such as table of contents,
-  headers, footers, section metadata, and URLs.
-- Focus on Concepts The abstract must focus on concepts found in the
-  provided text. The abstract must be factually accurate. Do not
-  write any concepts not found in the provided text. Do not
-  speculate. Do not omit any conceptual details.
-- Completeness: The abstract must capture all topics the reader will
-  need to understand the concepts found in the provided text. Describe
-  all terms, definitions, entities, people, events, concepts,
-  conceptual relationships, and any other topics necessary for the
-  reader to understand the concepts of the provided text.
-
-- Format: Respond in the form of a valid JSON object.
-</instructions>
-<example>
-{jsonexample}
-</example>
-<requirements>
-You will respond only with the JSON object. Do not provide
-explanations. Do not use special characters in the abstract text. The
-abstract must be written as plain text.
-</requirements>"""
-
-    return promptscholar
-
-def to_json_ld(text):
-
-    prompt = f"""<instructions>
-Study the following text and output any facts you discover in
-well-structured JSON-LD format.
-Use any schema you understand from schema.org to describe the facts.
-</instructions>
-
-<text>
-{text}
-</text>
-
-<requirements>
-You will respond only with raw JSON-LD data in JSON format. Do not provide
-explanations. Do not use special characters in the abstract text. The
-abstract must be written as plain text.  Do not add markdown formatting
-or headers or prefixes.  Do not use information which is not present in
-the input text.
-</requirements>"""
-    
-    return prompt
-    
-
-def to_relationships(text):
-
-    prompt = f"""<instructions>
-Study the following text and derive entity relationships.  For each
-relationship, derive the subject, predicate and object of the relationship.
-Output relationships in JSON format as an arary of objects with fields:
-- subject: the subject of the relationship
-- predicate: the predicate
-- object: the object of the relationship
-- object-entity: false if the object is a simple data type: name, value or date.  true if it is an entity.
-</instructions>
-
-<text>
-{text}
-</text>
-
-<requirements>
-You will respond only with raw JSON format data. Do not provide
-explanations. Do not use special characters in the abstract text. The
-abstract must be written as plain text.  Do not add markdown formatting
-or headers or prefixes.
-</requirements>"""
-    
-    return prompt
-    
-def to_definitions(text):
-
-    prompt = f"""<instructions>
-Study the following text and derive definitions for any discovered entities.
-Do not provide definitions for entities whose definitions are incomplete
-or unknown.
-Output relationships in JSON format as an arary of objects with fields:
-- entity: the name of the entity
-- definition: English text which defines the entity
-</instructions>
-
-<text>
-{text}
-</text>
-
-<requirements>
-You will respond only with raw JSON format data. Do not provide
-explanations. Do not use special characters in the abstract text. The
-abstract will be written as plain text.  Do not add markdown formatting
-or headers or prefixes.  Do not include null or unknown definitions.
-</requirements>"""
-    
-    return prompt
-    

trustgraph/.ipynb_checkpoints/rdf-checkpoint.py

@@ -1,6 +0,0 @@
-
-RDF_LABEL = "http://www.w3.org/2000/01/rdf-schema#label"
-DEFINITION = "http://www.w3.org/2004/02/skos/core#definition"
-
-TRUSTGRAPH_ENTITIES = "http://trustgraph.ai/e/"
-

trustgraph/model/text_completion/cohere/llm.py

@@ -5,6 +5,7 @@ Input is prompt, output is response.
 """
 
 import cohere
+import re
 
 from .... schema import TextCompletionRequest, TextCompletionResponse
 from .... schema import text_completion_request_queue
@@ -62,6 +63,7 @@ class Processor(ConsumerProducer):
         model=self.model,
         #model='c4ai-aya-23-8b',
         message=prompt,
+        preamble = "You are an AI-assistant chatbot. You are trained to read text and find entities in that text. You respond only with well-formed JSON.",
         temperature=0.0,
         chat_history=[],
         prompt_truncation='auto',
@@ -72,9 +74,22 @@ class Processor(ConsumerProducer):
             if event.event_type == "text-generation":
                 resp = event.text
                 print(resp, flush=True)
-                
+
+        # Parse output for ```json``` delimiters
+        pattern = r'```json\s*([\s\S]*?)\s*```'
+        match = re.search(pattern, resp)
+
+        if match:
+            # If delimiters are found, extract the JSON content
+            json_content = match.group(1)
+            json_resp = json_content.strip()
+        
+        else:
+            # If no delimiters are found, return the original text
+            json_resp = resp.strip()
+        
         print("Send response...", flush=True)
-        r = TextCompletionResponse(response=resp)
+        r = TextCompletionResponse(response=json_resp)
         self.send(r, properties={"id": id})
 
         print("Done.", flush=True)