trustgraph/README.md

# For AMD...
# pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.1/

----------------------------------------------------------------------------

docker network create trustgraph --driver bridge

docker volume create cassandra
docker volume create pulsar-conf
docker volume create pulsar-data
docker volume create etcd
docker volume create minio-data
docker volume create milvus

----------------------------------------------------------------------------

Cassandra:
docker run -i -t \
    --name cassandra \
    --network trustgraph \
    -p 9042:9042 \
    -v cassandra:/var/lib/cassandra \
    docker.io/cassandra:4.1.5

----------------------------------------------------------------------------

Pulsar:

docker run -it \
    --name pulsar \
    --network trustgraph \
    -p 6650:6650 \
    -p 8080:8080 \
    -v pulsar-conf:/pulsar/conf \
    -v pulsar-data:/pulsar/data \
    docker.io/apachepulsar/pulsar:3.3.0 \
        bin/pulsar standalone

----------------------------------------------------------------------------

Milvus:

docker run -i -t \
    --name etcd \
    --network trustgraph \
    -p 2379:2379 \
    -e ETCD_AUTO_COMPACTION_MODE=revision \
    -e ETCD_AUTO_COMPACTION_RETENTION=1000 \
    -e ETCD_QUOTA_BACKEND_BYTES=4294967296 \
    -e ETCD_SNAPSHOT_COUNT=50000 \
    -v etcd:/etcd \
    quay.io/coreos/etcd:v3.5.5 \
        etcd -advertise-client-urls=http://127.0.0.1:2379 \
        -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd

docker run -i -t \
    --name minio \
    --network trustgraph \
    -p 9000:9000 -p 9001:9001 \
    -e MINIO_ACCESS_KEY=minioadmin \
    -e MINIO_SECRET_KEY=minioadmin \
    -v minio-data:/minio_data \
    docker.io/minio/minio:RELEASE.2024-07-04T14-25-45Z \
        minio server /minio_data --console-address :9001

docker run -i -t \
    --name milvus \
    --network trustgraph \
    -p 19530:19530 -p 9091:9091 \
    -e ETCD_ENDPOINTS=etcd:2379 \
    -e MINIO_ADDRESS=minio:9000 \
    -v milvus:/var/lib/milvus \
    docker.io/milvusdb/milvus:v2.4.5 \
        milvus run standalone

----------------------------------------------------------------------------

. env/bin/activate

scripts/pdf-decoder -p pulsar://localhost:6650/
scripts/chunker -p pulsar://localhost:6650/
scripts/vectorize-minilm -p pulsar://localhost:6650/
scripts/kg-extract-definitions -p pulsar://localhost:6650/
scripts/kg-extract-relationships -p pulsar://localhost:6650/
scripts/vector-write-milvus -p pulsar://localhost:6650/
scripts/graph-write-cassandra -p pulsar://localhost:6650/
scripts/llm-ollama-text  -p pulsar://localhost:6650/ -r http://monster:11434/

----------------------------------------------------------------------------

The pub/sub infra means individual processors can be switched out for other
processors which do the same thing e.g.

- graph-write writes to Cassandra but could be switched out for something else
  which uses a different store
- vector-store writes to Milvus but could be switched out for something which
  writes to a different store
- llm-ollama invokes Ollama, could be switched out for a different LLM with a
  different API

The vectorizer is a bit hard to switch out ATM, different embedding algorithms
have different vector dimensions, but can probably make that work.

kg-extractor turns chunks into edges, so maybe that could be switched out for
different algorithms & different schemas. Potentially a good place for 'value
add' processing to be added.

It would also be possible to have more than one processor running where
kg-extractor sits to have different kinds of semantic extraction: other
proceessors could take as input the output of vectorizer, and drop edges and
edge associations into graph-write and vector-store

Actually, as it stands, kg-extractor does 2 different kinds of processing
itself:

- Creates edges for relationships, and
- Creates edges for definitions

Each of those uses its own LLM invocation and mashes the output into RDF. It
would be useful to split kg-extractor into 2 pieces to demonstrate this bit of
the architecture.

----------------------------------------------------------------------------

- pdf-extractor turns a PDF doc into plain text.  For each PDF page, a
  separate blob of text is emitted currently.
- chunker applies chunking to turn 1 PDF page into several chunks
- vectorizer invokes a sentence embedding model to create vectors.  Doesn't
  store anything.  Output of vectorizer is chunk plus embedding.
- kg-extract turns chunks into edges using LLM prompting. There are 2 outputs:
  - RDF triples
  - An association between the vector embedding and RDF entities. For every
    triple, there's an association between the S, P and O entiites and the
    embedding.
- graph-write loads the triples into a store
- vector-store store the association between the vector and the RDF URI in a
  vector store

----------------------------------------------------------------------------

The demo query algorithm is...

- Take the question, extract embeddings using a sentence model
- Use the vector store to lookup RDF URIs
- Use the RDF URIs to find relevant RDF triples in Cassandra
- Also use Cassandra to turn RDF URIs into plain-text labels if they are known
- Construct the prompt, turn RDF triples into a knowledge graph format (used
  Cypher, seems to work but not sure that's the best choice)
- LLM prompt and show the answer