feat: update project version and documentation link
- Bump version from 0.1.0 to 0.1.1 - Update documentation URL to point to GitHub repository
This commit is contained in:
parent
77084737dd
commit
2c6677748a
8 changed files with 1104 additions and 2 deletions
2
.gitignore
vendored
2
.gitignore
vendored
|
|
@ -28,6 +28,7 @@ venv/
|
||||||
ENV/
|
ENV/
|
||||||
env.bak/
|
env.bak/
|
||||||
venv.bak/
|
venv.bak/
|
||||||
|
.claude/
|
||||||
|
|
||||||
# IDE
|
# IDE
|
||||||
.idea/
|
.idea/
|
||||||
|
|
@ -49,6 +50,7 @@ build/
|
||||||
dist/
|
dist/
|
||||||
*.egg-info/
|
*.egg-info/
|
||||||
*.egg
|
*.egg
|
||||||
|
*.sh
|
||||||
|
|
||||||
# Virtual environments
|
# Virtual environments
|
||||||
venv/
|
venv/
|
||||||
|
|
|
||||||
57
doc/README.md
Normal file
57
doc/README.md
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
# NOMYO Secure Client Documentation
|
||||||
|
|
||||||
|
This documentation provides comprehensive information about using the NOMYO Secure Python Chat Client, a drop-in replacement for OpenAI's ChatCompletion API with end-to-end (E2E) encryption.
|
||||||
|
To use this client library you need a paid subscribtion on [NOMYO Inference](https://chat.nomyo.ai/).
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The NOMYO Secure Client provides:
|
||||||
|
|
||||||
|
- **End-to-end encryption** using hybrid encryption (AES-256-GCM + RSA-OAEP)
|
||||||
|
- **OpenAI API compatibility** - same interface as OpenAI's ChatCompletion
|
||||||
|
- **Secure memory protection** - prevents plaintext from being swapped to disk
|
||||||
|
- **Automatic key management** - handles key generation and loading automatically
|
||||||
|
- **HTTPS enforcement** - secure communication by default
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
# Initialize client (defaults to https://api.nomyo.ai)
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
# Simple chat completion
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Hello! How are you today?"}
|
||||||
|
],
|
||||||
|
security_tier="standard", # optional: standard, high or maximum
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
# Run the async function
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Documentation Structure
|
||||||
|
|
||||||
|
1. [Installation](installation.md) - How to install and set up the client
|
||||||
|
2. [Getting Started](getting-started.md) - Quick start guide with examples
|
||||||
|
3. [API Reference](api-reference.md) - Complete API documentation
|
||||||
|
4. [Security Guide](security-guide.md) - Security features and best practices
|
||||||
|
5. [Examples](examples.md) - Advanced usage scenarios
|
||||||
|
6. [Troubleshooting](troubleshooting.md) - Common issues and solutions
|
||||||
|
|
||||||
|
## Key Features
|
||||||
|
|
||||||
|
- **OpenAI Compatibility**: Use the same API as OpenAI's ChatCompletion
|
||||||
|
- **End-to-End Encryption**: All prompts and responses are automatically encrypted/decrypted
|
||||||
|
- **Secure Memory Protection**: Prevents sensitive data from being swapped to disk
|
||||||
|
- **Automatic Key Management**: Keys are generated and loaded automatically
|
||||||
|
- **Flexible Security Tiers**: Control security levels for different data types
|
||||||
197
doc/api-reference.md
Normal file
197
doc/api-reference.md
Normal file
|
|
@ -0,0 +1,197 @@
|
||||||
|
# API Reference
|
||||||
|
|
||||||
|
## SecureChatCompletion Class
|
||||||
|
|
||||||
|
The `SecureChatCompletion` class is the main entry point for using the NOMYO secure client. It provides the same interface as OpenAI's ChatCompletion API with end-to-end encryption.
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```python
|
||||||
|
SecureChatCompletion(
|
||||||
|
base_url: str = "https://api.nomyo.ai",
|
||||||
|
allow_http: bool = False,
|
||||||
|
api_key: Optional[str] = None,
|
||||||
|
secure_memory: bool = True
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `base_url` (str): Base URL of the NOMYO Router (must use HTTPS for production)
|
||||||
|
- `allow_http` (bool): Allow HTTP connections (ONLY for local development, never in production)
|
||||||
|
- `api_key` (Optional[str]): Optional API key for bearer authentication
|
||||||
|
- `secure_memory` (bool): Enable secure memory protection (default: True)
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
#### create(model, messages, **kwargs)
|
||||||
|
|
||||||
|
Creates a new chat completion for the provided messages and parameters.
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `model` (str): The model to use for the chat completion
|
||||||
|
- `messages` (List[Dict]): A list of message objects. Each message has a role ("system", "user", or "assistant") and content
|
||||||
|
- `**kwargs`: Additional parameters that can be passed to the API
|
||||||
|
|
||||||
|
**Supported OpenAI Parameters:**
|
||||||
|
|
||||||
|
- `temperature` (float): Sampling temperature (0-2)
|
||||||
|
- `max_tokens` (int): Maximum tokens to generate
|
||||||
|
- `top_p` (float): Nucleus sampling
|
||||||
|
- `frequency_penalty` (float): Frequency penalty
|
||||||
|
- `presence_penalty` (float): Presence penalty
|
||||||
|
- `stop` (Union[str, List[str]]): Stop sequences
|
||||||
|
- `n` (int): Number of completions
|
||||||
|
- `stream` (bool): Streaming always = False to minimize de-/encryption overhead
|
||||||
|
- `tools` (List): Tool definitions
|
||||||
|
- `tool_choice` (str): Tool selection strategy
|
||||||
|
- `user` (str): User identifier
|
||||||
|
- `security_tier` (str): Security level ("standard", "high", or "maximum")
|
||||||
|
|
||||||
|
**Returns:**
|
||||||
|
A dictionary containing the chat completion response with the following structure:
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
"id": str,
|
||||||
|
"object": "chat.completion",
|
||||||
|
"created": int,
|
||||||
|
"model": str,
|
||||||
|
"choices": [
|
||||||
|
{
|
||||||
|
"index": int,
|
||||||
|
"message": {
|
||||||
|
"role": str,
|
||||||
|
"content": str,
|
||||||
|
"tool_calls": List[Dict] # if tools were used
|
||||||
|
},
|
||||||
|
"finish_reason": str
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"usage": {
|
||||||
|
"prompt_tokens": int,
|
||||||
|
"completion_tokens": int,
|
||||||
|
"total_tokens": int
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### acreate(model, messages, **kwargs)
|
||||||
|
|
||||||
|
Async alias for create() method.
|
||||||
|
|
||||||
|
**Parameters:** Same as create() method
|
||||||
|
|
||||||
|
**Returns:** Same as create() method
|
||||||
|
|
||||||
|
## SecureCompletionClient Class
|
||||||
|
|
||||||
|
The `SecureCompletionClient` class handles the underlying encryption, key management, and API communication.
|
||||||
|
|
||||||
|
### Constructor
|
||||||
|
|
||||||
|
```python
|
||||||
|
SecureCompletionClient(router_url: str = "https://api.nomyo.ai:12434", allow_http: bool = False)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `router_url` (str): Base URL of the NOMYO Router (must use HTTPS for production)
|
||||||
|
- `allow_http` (bool): Allow HTTP connections (ONLY for local development, never in production)
|
||||||
|
|
||||||
|
### Methods
|
||||||
|
|
||||||
|
#### generate_keys(save_to_file: bool = False, key_dir: str = "client_keys", password: Optional[str] = None)
|
||||||
|
|
||||||
|
Generate RSA key pair for secure communication.
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `save_to_file` (bool): Whether to save keys to files
|
||||||
|
- `key_dir` (str): Directory to save keys (if save_to_file is True)
|
||||||
|
- `password` (Optional[str]): Optional password to encrypt private key
|
||||||
|
|
||||||
|
#### load_keys(private_key_path: str, public_key_path: Optional[str] = None, password: Optional[str] = None)
|
||||||
|
|
||||||
|
Load RSA keys from files.
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `private_key_path` (str): Path to private key file
|
||||||
|
- `public_key_path` (Optional[str]): Path to public key file (optional, derived from private key if not provided)
|
||||||
|
- `password` (Optional[str]): Optional password for encrypted private key
|
||||||
|
|
||||||
|
#### fetch_server_public_key()
|
||||||
|
|
||||||
|
Fetch the server's public key from the /pki/public_key endpoint.
|
||||||
|
|
||||||
|
**Returns:**
|
||||||
|
Server's public key as PEM string
|
||||||
|
|
||||||
|
#### encrypt_payload(payload: Dict[str, Any])
|
||||||
|
|
||||||
|
Encrypt a payload using hybrid encryption (AES-256-GCM + RSA-OAEP).
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `payload` (Dict[str, Any]): Dictionary containing the chat completion request
|
||||||
|
|
||||||
|
**Returns:**
|
||||||
|
Encrypted payload as bytes
|
||||||
|
|
||||||
|
#### decrypt_response(encrypted_response: bytes, payload_id: str)
|
||||||
|
|
||||||
|
Decrypt a response from the secure endpoint.
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `encrypted_response` (bytes): Encrypted response bytes
|
||||||
|
- `payload_id` (str): Payload ID for metadata verification
|
||||||
|
|
||||||
|
**Returns:**
|
||||||
|
Decrypted response dictionary
|
||||||
|
|
||||||
|
#### send_secure_request(payload: Dict[str, Any], payload_id: str, api_key: Optional[str] = None, security_tier: Optional[str] = None)
|
||||||
|
|
||||||
|
Send a secure chat completion request to the router.
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
|
||||||
|
- `payload` (Dict[str, Any]): Chat completion request payload
|
||||||
|
- `payload_id` (str): Unique identifier for this request
|
||||||
|
- `api_key` (Optional[str]): Optional API key for bearer authentication
|
||||||
|
- `security_tier` (Optional[str]): Optional security tier for routing
|
||||||
|
|
||||||
|
**Returns:**
|
||||||
|
Decrypted response from the LLM
|
||||||
|
|
||||||
|
## Exception Classes
|
||||||
|
|
||||||
|
### APIError
|
||||||
|
|
||||||
|
Base class for all API-related errors.
|
||||||
|
|
||||||
|
### AuthenticationError
|
||||||
|
|
||||||
|
Raised when authentication fails (e.g., invalid API key).
|
||||||
|
|
||||||
|
### InvalidRequestError
|
||||||
|
|
||||||
|
Raised when the request is invalid (HTTP 400).
|
||||||
|
|
||||||
|
### APIConnectionError
|
||||||
|
|
||||||
|
Raised when there's a connection error.
|
||||||
|
|
||||||
|
### RateLimitError
|
||||||
|
|
||||||
|
Raised when rate limit is exceeded (HTTP 429).
|
||||||
|
|
||||||
|
### ServerError
|
||||||
|
|
||||||
|
Raised when the server returns an error (HTTP 500).
|
||||||
|
|
||||||
|
### SecurityError
|
||||||
|
|
||||||
|
Raised when a security violation is detected.
|
||||||
362
doc/examples.md
Normal file
362
doc/examples.md
Normal file
|
|
@ -0,0 +1,362 @@
|
||||||
|
# Examples
|
||||||
|
|
||||||
|
## Basic Usage Examples
|
||||||
|
|
||||||
|
### Simple Chat Completion
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def simple_chat():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Hello, how are you?"}
|
||||||
|
],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(simple_chat())
|
||||||
|
```
|
||||||
|
|
||||||
|
### Chat with System Message
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def chat_with_system():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "system", "content": "You are a helpful assistant."},
|
||||||
|
{"role": "user", "content": "What is the capital of France?"}
|
||||||
|
],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(chat_with_system())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Advanced Usage Examples
|
||||||
|
|
||||||
|
### Using Different Security Tiers
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def security_tiers():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
# Standard security
|
||||||
|
response1 = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "General query"}],
|
||||||
|
security_tier="standard"
|
||||||
|
)
|
||||||
|
|
||||||
|
# High security for sensitive data
|
||||||
|
response2 = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Bank account info"}],
|
||||||
|
security_tier="high"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Maximum security for classified data
|
||||||
|
response3 = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Medical records"}],
|
||||||
|
security_tier="maximum"
|
||||||
|
)
|
||||||
|
|
||||||
|
asyncio.run(security_tiers())
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using Tools
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def chat_with_tools():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "What's the weather in Paris?"}
|
||||||
|
],
|
||||||
|
tools=[
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "get_weather",
|
||||||
|
"description": "Get weather information",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"location": {"type": "string"}
|
||||||
|
},
|
||||||
|
"required": ["location"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(chat_with_tools())
|
||||||
|
```
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion, AuthenticationError, InvalidRequestError
|
||||||
|
|
||||||
|
async def error_handling():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
try:
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Hello"}]
|
||||||
|
)
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
except AuthenticationError as e:
|
||||||
|
print(f"Authentication failed: {e}")
|
||||||
|
except InvalidRequestError as e:
|
||||||
|
print(f"Invalid request: {e}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Other error: {e}")
|
||||||
|
|
||||||
|
asyncio.run(error_handling())
|
||||||
|
```
|
||||||
|
|
||||||
|
### Custom Base URL
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def custom_base_url():
|
||||||
|
# For local development
|
||||||
|
client = SecureChatCompletion(
|
||||||
|
base_url="https://NOMYO-PRO-ROUTER:12435",
|
||||||
|
allow_http=True
|
||||||
|
)
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Hello"}]
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(custom_base_url())
|
||||||
|
```
|
||||||
|
|
||||||
|
### API Key Authentication
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def api_key_auth():
|
||||||
|
# Initialize with API key
|
||||||
|
client = SecureChatCompletion(
|
||||||
|
api_key="your-api-key-here"
|
||||||
|
)
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Hello"}]
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(api_key_auth())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Real-World Scenarios
|
||||||
|
|
||||||
|
### Chat Application with History
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
class SecureChatApp:
|
||||||
|
def __init__(self):
|
||||||
|
self.client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
self.conversation_history = []
|
||||||
|
|
||||||
|
async def chat(self, message):
|
||||||
|
# Add user message to history
|
||||||
|
self.conversation_history.append({"role": "user", "content": message})
|
||||||
|
|
||||||
|
# Get response from the model
|
||||||
|
response = await self.client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=self.conversation_history,
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
# Add assistant response to history
|
||||||
|
assistant_message = response['choices'][0]['message']
|
||||||
|
self.conversation_history.append(assistant_message)
|
||||||
|
|
||||||
|
return assistant_message['content']
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
app = SecureChatApp()
|
||||||
|
|
||||||
|
# First message
|
||||||
|
response1 = await app.chat("Hello, what's your name?")
|
||||||
|
print(f"Assistant: {response1}")
|
||||||
|
|
||||||
|
# Second message
|
||||||
|
response2 = await app.chat("Can you tell me about secure chat clients?")
|
||||||
|
print(f"Assistant: {response2}")
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data Processing with Tools
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def data_processing():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
# Process data with tool calling
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Process this data: 100, 200, 300, 400"}
|
||||||
|
],
|
||||||
|
tools=[
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "calculate_statistics",
|
||||||
|
"description": "Calculate statistical measures",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"data": {"type": "array", "items": {"type": "number"}}
|
||||||
|
},
|
||||||
|
"required": ["data"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(data_processing())
|
||||||
|
```
|
||||||
|
|
||||||
|
### Batch Processing
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def batch_processing():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
# Process multiple queries concurrently
|
||||||
|
tasks = []
|
||||||
|
|
||||||
|
queries = [
|
||||||
|
"What is the weather today?",
|
||||||
|
"Tell me about Python programming",
|
||||||
|
"How to learn machine learning?"
|
||||||
|
]
|
||||||
|
|
||||||
|
for query in queries:
|
||||||
|
task = client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": query}],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
tasks.append(task)
|
||||||
|
|
||||||
|
# Execute all queries in parallel
|
||||||
|
responses = await asyncio.gather(*tasks)
|
||||||
|
|
||||||
|
for i, response in enumerate(responses):
|
||||||
|
print(f"Query {i+1}: {response['choices'][0]['message']['content'][:100]}...")
|
||||||
|
|
||||||
|
asyncio.run(batch_processing())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration Examples
|
||||||
|
|
||||||
|
### Custom Client Configuration
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def custom_config():
|
||||||
|
# Create a client with custom configuration
|
||||||
|
client = SecureChatCompletion(
|
||||||
|
allow_http=False, # Force HTTPS
|
||||||
|
api_key="your-api-key",
|
||||||
|
secure_memory=True # Explicitly enable secure memory protection (default)
|
||||||
|
)
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Hello"}],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(custom_config())
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment-Based Configuration (strongly recommended)
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def env_config():
|
||||||
|
# Load configuration from environment variables
|
||||||
|
api_key = os.getenv('NOMYO_API_KEY')
|
||||||
|
|
||||||
|
client = SecureChatCompletion(
|
||||||
|
api_key=api_key
|
||||||
|
)
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Hello"}]
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(env_config())
|
||||||
|
```
|
||||||
|
|
||||||
|
##
|
||||||
212
doc/getting-started.md
Normal file
212
doc/getting-started.md
Normal file
|
|
@ -0,0 +1,212 @@
|
||||||
|
# Getting Started
|
||||||
|
|
||||||
|
## Basic Usage
|
||||||
|
|
||||||
|
The NOMYO client provides end-to-end encryption (E2E) for all communications between your application and the NOMYO inference endpoints. This ensures that your prompts and responses are protected from unauthorized access or interception.
|
||||||
|
|
||||||
|
The NOMYO client provides the same interface as OpenAI's ChatCompletion API, making it easy to integrate into existing code.
|
||||||
|
|
||||||
|
The encryption and decryption process is causing overhead, thus inference speed will be lower compared to unencrypted inference. Using high and maximum security_tiers in the client request will add additional latency to the round-trip-time, but guarantees highest confidential use cases.
|
||||||
|
|
||||||
|
To minimize en-/decryption overhead the API is **none**-streaming. OpenAI API compatibily allows to set streaming=True in the request, but this will be ignored on the server side to allow maximum response token generation.
|
||||||
|
|
||||||
|
### Simple Chat Completion
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
# Initialize client
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
# Simple chat completion
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Hello! How are you today?"}
|
||||||
|
],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
### With System Messages
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "system", "content": "You are a helpful assistant."},
|
||||||
|
{"role": "user", "content": "What is the capital of France?"}
|
||||||
|
],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
## API Key Authentication
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
# Initialize with API key (recommended for production)
|
||||||
|
client = SecureChatCompletion(
|
||||||
|
api_key="your-api-key-here"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Or pass API key in the create() method
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Hello!"}
|
||||||
|
],
|
||||||
|
api_key="your-api-key-here" # Overrides instance API key
|
||||||
|
)
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security Tiers
|
||||||
|
|
||||||
|
The client supports different security tiers for controlling data protection levels:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
# Standard security tier (default)
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Hello!"}
|
||||||
|
],
|
||||||
|
security_tier="standard"
|
||||||
|
)
|
||||||
|
|
||||||
|
# High security tier for sensitive data
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "What's my bank account balance?"}
|
||||||
|
],
|
||||||
|
security_tier="high" #enforces secure tokenizer
|
||||||
|
)
|
||||||
|
|
||||||
|
# Maximum security tier for classified data
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Share my personal medical records"}
|
||||||
|
],
|
||||||
|
security_tier="maximum" #HIPAA PHI compliance or other confidential use cases
|
||||||
|
)
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Using Tools
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "What's the weather in Paris?"}
|
||||||
|
],
|
||||||
|
tools=[
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "get_weather",
|
||||||
|
"description": "Get weather information",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"location": {"type": "string"}
|
||||||
|
},
|
||||||
|
"required": ["location"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Async Alias
|
||||||
|
|
||||||
|
The client also provides an `acreate` async alias for convenience:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
client = SecureChatCompletion(api_key="your-api-key-here")
|
||||||
|
|
||||||
|
response = await client.acreate(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Hello!"}
|
||||||
|
],
|
||||||
|
temperature=0.7
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from nomyo import SecureChatCompletion, AuthenticationError, InvalidRequestError
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
client = SecureChatCompletion(base_url="https://api.nomyo.ai:12434")
|
||||||
|
|
||||||
|
try:
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": "Hello!"}
|
||||||
|
]
|
||||||
|
)
|
||||||
|
print(response['choices'][0]['message']['content'])
|
||||||
|
except AuthenticationError as e:
|
||||||
|
print(f"Authentication failed: {e}")
|
||||||
|
except InvalidRequestError as e:
|
||||||
|
print(f"Invalid request: {e}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Other error: {e}")
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
74
doc/installation.md
Normal file
74
doc/installation.md
Normal file
|
|
@ -0,0 +1,74 @@
|
||||||
|
# Installation Guide
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Python 3.7 or higher
|
||||||
|
- pip (Python package installer)
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
### Install from PyPI (recommended)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install nomyo
|
||||||
|
```
|
||||||
|
|
||||||
|
### Install from source
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Clone the repository
|
||||||
|
git clone https://github.com/nomyo-ai/nomyo.git
|
||||||
|
cd nomyo
|
||||||
|
|
||||||
|
# Install dependencies
|
||||||
|
pip install -r requirements.txt
|
||||||
|
|
||||||
|
# Install the package
|
||||||
|
pip install -e .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
The NOMYO client requires the following dependencies:
|
||||||
|
|
||||||
|
- `cryptography` - Cryptographic primitives (RSA, AES, etc.)
|
||||||
|
- `httpx` - Async HTTP client
|
||||||
|
- `anyio` - Async compatibility layer
|
||||||
|
|
||||||
|
These are automatically installed when you install the package via pip.
|
||||||
|
|
||||||
|
## Virtual Environment (Recommended)
|
||||||
|
|
||||||
|
It's recommended to use a virtual environment to avoid conflicts with other Python packages:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create virtual environment
|
||||||
|
python -m venv nomyo_env
|
||||||
|
|
||||||
|
# Activate virtual environment
|
||||||
|
source nomyo_env/bin/activate # On Linux/Mac
|
||||||
|
# or
|
||||||
|
nomyo_env\Scripts\activate # On Windows
|
||||||
|
|
||||||
|
# Install nomyo
|
||||||
|
pip install nomyo
|
||||||
|
```
|
||||||
|
|
||||||
|
## Verify Installation
|
||||||
|
|
||||||
|
To verify the installation worked correctly:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import nomyo
|
||||||
|
print("NOMYO client installed successfully!")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development Installation
|
||||||
|
|
||||||
|
For development purposes, you can install the package in development mode:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -e .[dev]
|
||||||
|
```
|
||||||
|
|
||||||
|
This will install additional development dependencies.
|
||||||
198
doc/security-guide.md
Normal file
198
doc/security-guide.md
Normal file
|
|
@ -0,0 +1,198 @@
|
||||||
|
# Security Guide
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The NOMYO client provides end-to-end encryption for all communications between your application and the NOMYO inference endpoints. This ensures that your prompts and responses are protected from unauthorized access or interception.
|
||||||
|
|
||||||
|
## Encryption Mechanism
|
||||||
|
|
||||||
|
### Hybrid Encryption
|
||||||
|
|
||||||
|
The client uses a hybrid encryption approach combining:
|
||||||
|
|
||||||
|
1. **AES-256-GCM** for payload encryption (authenticated encryption)
|
||||||
|
2. **RSA-OAEP** for key exchange (4096-bit keys)
|
||||||
|
|
||||||
|
This provides both performance (AES for data) and security (RSA for key exchange).
|
||||||
|
|
||||||
|
### Key Management
|
||||||
|
|
||||||
|
#### Automatic Key Generation
|
||||||
|
|
||||||
|
Keys are automatically generated in memory on first use/session init. The client handles all key management internally.
|
||||||
|
|
||||||
|
#### Key Persistence (optional)
|
||||||
|
|
||||||
|
Keys *can* be saved to the `client_keys/` directory for reuse (i.e. in dev scenarios) across sessions [not recommend]:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Generate keys and save to file
|
||||||
|
await client.generate_keys(save_to_file=True, password="your-password")
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Password Protection
|
||||||
|
|
||||||
|
Saved private keys should be password-protected in all environments:
|
||||||
|
|
||||||
|
```python
|
||||||
|
await client.generate_keys(save_to_file=True, password="your-strong-password")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Secure Memory Protection
|
||||||
|
|
||||||
|
### Ephemeral AES Keys
|
||||||
|
|
||||||
|
- **Per-request encryption keys**: A unique AES-256 key is generated for each request
|
||||||
|
- **Automatic rotation**: AES keys are never reused - a fresh key is created for every encryption operation
|
||||||
|
- **Forward secrecy**: Compromise of one AES key only affects that single request
|
||||||
|
- **Secure generation**: AES keys are generated using cryptographically secure random number generation (`secrets.token_bytes`)
|
||||||
|
- **Automatic cleanup**: AES keys are zeroed from memory immediately after use
|
||||||
|
|
||||||
|
### Memory Protection
|
||||||
|
|
||||||
|
The client can use secure memory protection to:
|
||||||
|
|
||||||
|
- Prevent plaintext payloads from being swapped to disk
|
||||||
|
- Guarantee memory is zeroed after encryption
|
||||||
|
- Prevent sensitive data from being stored in memory dumps
|
||||||
|
|
||||||
|
## Security Best Practices
|
||||||
|
|
||||||
|
### For Production Use
|
||||||
|
|
||||||
|
1. **Always use password protection** for private keys
|
||||||
|
2. **Keep private keys secure** (permissions set to 600 - owner-only access)
|
||||||
|
3. **Never share your private key**
|
||||||
|
4. **Verify server's public key fingerprint** before first use
|
||||||
|
5. **Use HTTPS connections** (never allow HTTP in production)
|
||||||
|
|
||||||
|
### Key Management
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Generate keys with password protection
|
||||||
|
await client.generate_keys(
|
||||||
|
save_to_file=True,
|
||||||
|
key_dir="client_keys",
|
||||||
|
password="strong-password-here"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Load existing keys with password
|
||||||
|
await client.load_keys(
|
||||||
|
"client_keys/private_key.pem",
|
||||||
|
"client_keys/public_key.pem",
|
||||||
|
password="strong-password-here"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Security Tiers
|
||||||
|
|
||||||
|
The client supports three security tiers:
|
||||||
|
|
||||||
|
- **Standard**: General secure inference
|
||||||
|
- **High**: Sensitive business data
|
||||||
|
- **Maximum**: Maximum isolation (HIPAA PHI, classified data)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Use different security tiers
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "My sensitive data"}],
|
||||||
|
security_tier="high"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security Features
|
||||||
|
|
||||||
|
### End-to-End Encryption
|
||||||
|
|
||||||
|
All prompts and responses are automatically encrypted and decrypted, ensuring:
|
||||||
|
|
||||||
|
- No plaintext data is sent over the network
|
||||||
|
- No plaintext data is stored in memory
|
||||||
|
- No plaintext data is stored on disk
|
||||||
|
|
||||||
|
### Forward Secrecy
|
||||||
|
|
||||||
|
Each request uses a unique AES key, ensuring that:
|
||||||
|
|
||||||
|
- Compromise of one request's key only affects that request
|
||||||
|
- Previous requests remain secure even if current key is compromised
|
||||||
|
|
||||||
|
### Key Exchange Security
|
||||||
|
|
||||||
|
RSA-OAEP key exchange with 4096-bit keys provides:
|
||||||
|
|
||||||
|
- Strong encryption for key exchange
|
||||||
|
- Protection against known attacks
|
||||||
|
- Forward secrecy for key material
|
||||||
|
|
||||||
|
### Memory Protection
|
||||||
|
|
||||||
|
Secure memory features:
|
||||||
|
|
||||||
|
- Prevents plaintext from being swapped to disk
|
||||||
|
- Guarantees zeroing of sensitive memory
|
||||||
|
- Prevents memory dumps from containing sensitive data
|
||||||
|
|
||||||
|
## Compliance Considerations
|
||||||
|
|
||||||
|
### HIPAA Compliance
|
||||||
|
|
||||||
|
The client can be used for HIPAA-compliant applications when:
|
||||||
|
|
||||||
|
- Keys are password-protected
|
||||||
|
- HTTPS is used for all connections
|
||||||
|
- Private keys are stored securely
|
||||||
|
- Appropriate security measures are in place
|
||||||
|
|
||||||
|
### Data Classification
|
||||||
|
|
||||||
|
- **Standard**: General data
|
||||||
|
- **High**: Sensitive business data
|
||||||
|
- **Maximum**: Classified data (PHI, PII, etc.)
|
||||||
|
|
||||||
|
## Security Testing
|
||||||
|
|
||||||
|
The client includes comprehensive security testing:
|
||||||
|
|
||||||
|
- All encryption/decryption operations are tested
|
||||||
|
- Key management is verified
|
||||||
|
- Memory protection is validated
|
||||||
|
- Error handling is tested
|
||||||
|
|
||||||
|
Run the test suite to verify security:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 test.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting Security Issues
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
1. **Key loading failures**: Ensure private key file permissions are correct (600)
|
||||||
|
2. **Connection errors**: Verify HTTPS is used for production
|
||||||
|
3. **Decryption failures**: Check that the correct API key is used
|
||||||
|
4. **Memory protection errors**: SecureMemory module may not be available on all systems
|
||||||
|
|
||||||
|
### Debugging
|
||||||
|
|
||||||
|
The client adds metadata to responses that can help with debugging:
|
||||||
|
|
||||||
|
```python
|
||||||
|
response = await client.create(
|
||||||
|
model="Qwen/Qwen3-0.6B",
|
||||||
|
messages=[{"role": "user", "content": "Hello"}]
|
||||||
|
)
|
||||||
|
|
||||||
|
print(response["_metadata"]) # Contains security-related information
|
||||||
|
```
|
||||||
|
|
||||||
|
### Logging
|
||||||
|
|
||||||
|
Enable logging to see security operations:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import logging
|
||||||
|
logging.basicConfig(level=logging.DEBUG)
|
||||||
|
```
|
||||||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "nomyo"
|
name = "nomyo"
|
||||||
version = "0.1.0"
|
version = "0.1.1"
|
||||||
description = "OpenAI-compatible secure chat client with end-to-end encryption for NOMYO Inference Endpoints"
|
description = "OpenAI-compatible secure chat client with end-to-end encryption for NOMYO Inference Endpoints"
|
||||||
authors = [
|
authors = [
|
||||||
{name = "NOMYO.AI", email = "ichi@nomyo.ai"},
|
{name = "NOMYO.AI", email = "ichi@nomyo.ai"},
|
||||||
|
|
@ -44,7 +44,7 @@ dependencies = [
|
||||||
|
|
||||||
[project.urls]
|
[project.urls]
|
||||||
Homepage = "https://nomyo.ai"
|
Homepage = "https://nomyo.ai"
|
||||||
Documentation = "https://nomyo.ai/nomyo-docs"
|
Documentation = "https://github.com/nomyo-ai/nomyo/doc"
|
||||||
Repository = "https://github.com/nomyo-ai/nomyo"
|
Repository = "https://github.com/nomyo-ai/nomyo"
|
||||||
Issues = "https://github.com/nomyo-ai/nomyo/issues"
|
Issues = "https://github.com/nomyo-ai/nomyo/issues"
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue