nomyo-ai/async-semantic-llm-cache
2
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions

docs: udpate readme

Browse source
This commit is contained in:
Alpha Nerd 2026-03-06 16:39:21 +01:00
parent 3d8e5044d6
commit 0d267f210d
1 changed files with 10 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

23
README.md
View file

@ -2,9 +2,6 @@
**Async semantic caching for LLM API calls — reduce costs with one decorator.**
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python](https://img.shields.io/pypi/pyversions/semantic-llm-cache)](https://pypi.org/project/semantic-llm-cache/)
> **Fork of [karthyick/prompt-cache](https://github.com/karthyick/prompt-cache)** — fully converted to async for use with async frameworks (FastAPI, aiohttp, Starlette, etc.).
## Overview
@ -30,7 +27,7 @@ LLM API calls are expensive and slow. In production applications, **20-40% of pr
| `CachedLLM` | `chat()` | adds`achat()` |
| Utility functions | sync | `clear_cache`, `invalidate`, `warm_cache`, `export_cache` all async |
| `StorageBackend` ABC | sync abstract methods | all abstract methods are`async def` |
| Min Python | 3.9 | 3.10 (uses`X | Y` union syntax) |
| Min Python | 3.9 | 3.10 (uses`X |
## Installation
@ -38,7 +35,7 @@ Not yet published to PyPI. Install directly from the repository:
```bash
# Clone
git clone https://github.com/YOUR_ORG/prompt-cache.git
git clone https://github.com/nomyo-ai/async-semantic-llm-cache.git
cd prompt-cache
# Core (exact match only, SQLite backend)
@ -199,14 +196,14 @@ async def my_llm_function(prompt: str) -> str:
### Parameters
| Parameter | Type | Default | Description |
| -------------- | -------------- | ------------- | ----------------------------------------------------------- |
| `similarity` | `float` | `1.0` | Cosine similarity threshold (1.0 = exact, 0.9 = semantic) |
| `ttl` | `int | None` | `3600` | Time-to-live in seconds (None = never expires) |
| `backend` | `Backend` | `None` | Storage backend (None = in-memory) |
| `namespace` | `str` | `"default"` | Isolate different use cases |
| `enabled` | `bool` | `True` | Enable/disable caching |
| `key_func` | `Callable` | `None` | Custom cache key function |
| Parameter | Type | Default | Description |
| -------------- | ------------ | ------------- | ----------------------------------------------------------- |
| `similarity` | `float` | `1.0` | Cosine similarity threshold (1.0 = exact, 0.9 = semantic) |
| `ttl` | `int | None` | `3600` |
| `backend` | `Backend` | `None` | Storage backend (None = in-memory) |
| `namespace` | `str` | `"default"` | Isolate different use cases |
| `enabled` | `bool` | `True` | Enable/disable caching |
| `key_func` | `Callable` | `None` | Custom cache key function |
### Utility Functions