add readme and sample usage

This commit is contained in:
Adil Hafeez 2025-06-10 10:30:21 -07:00
parent 34eeb81b0f
commit 08b4da2a53
No known key found for this signature in database
GPG key ID: 9B18EF7691369645
3 changed files with 85 additions and 12 deletions

View file

@ -0,0 +1,63 @@
# hermesllm
A Rust library for translating LLM (Large Language Model) API requests and responses between Mistral, Groq, Gemini, Deepseek, OpenAI, and other provider-compliant formats.
## Features
- Unified types for chat completions and model metadata across multiple LLM providers
- Builder-pattern API for constructing requests in an idiomatic Rust style
- Easy conversion between provider formats
- Streaming and non-streaming response support
## Supported Providers
- Mistral
- Deepseek
- Groq
- Gemini
- OpenAI
- Claude
- Github
## Installation
Add the following to your `Cargo.toml`:
```toml
[dependencies]
hermesllm = { git = "https://github.com/katanemo/archgw", subdir = "crates/hermesllm" }
```
_Replace the path with the appropriate location if using as a workspace member or published crate._
## Usage
Construct a chat completion request using the builder pattern:
```rust
use hermesllm::Provider;
use hermesllm::providers::openai::types::ChatCompletionsRequest;
let request = ChatCompletionsRequest::builder("gpt-3.5-turbo", vec![Message::new("Hi".to_string())])
.build()
.expect("Failed to build OpenAIRequest");
// Convert to bytes for a specific provider
let bytes = request.to_bytes(Provider::OpenAI)?;
```
## API Overview
- `Provider`: Enum listing all supported LLM providers.
- `ChatCompletionsRequest`: Builder-pattern struct for creating chat completion requests.
- `ChatCompletionsResponse`: Struct for parsing responses.
- Streaming support via `SseChatCompletionIter`.
- Error handling via `OpenAIError`.
## Contributing
Contributions are welcome! Please open issues or pull requests for bug fixes, new features, or provider integrations.
## License
This project is licensed under the terms of the [MIT License](../LICENSE).

View file

@ -49,21 +49,22 @@ impl Display for Provider {
#[cfg(test)]
mod tests {
use crate::providers::openai::types::ChatCompletionsRequest;
use crate::providers::openai::types::{ChatCompletionsRequest, Message};
#[test]
fn openai_builder() {
let request = ChatCompletionsRequest::builder("gpt-3.5-turbo", vec![])
.temperature(0.7)
.top_p(0.9)
.n(1)
.max_tokens(100)
.stream(false)
.stop(vec!["\n".to_string()])
.presence_penalty(0.0)
.frequency_penalty(0.0)
.build()
.expect("Failed to build OpenAIRequest");
let request =
ChatCompletionsRequest::builder("gpt-3.5-turbo", vec![Message::new("Hi".to_string())])
.temperature(0.7)
.top_p(0.9)
.n(1)
.max_tokens(100)
.stream(false)
.stop(vec!["\n".to_string()])
.presence_penalty(0.0)
.frequency_penalty(0.0)
.build()
.expect("Failed to build OpenAIRequest");
assert_eq!(request.model, "gpt-3.5-turbo");
assert_eq!(request.temperature, Some(0.7));

View file

@ -80,6 +80,15 @@ pub struct Message {
pub content: Option<ContentType>,
}
impl Message {
pub fn new(content: String) -> Self {
Self {
role: "user".to_string(),
content: Some(ContentType::Text(content)),
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StreamOptions {
pub include_usage: bool,