This commit is contained in:
Alex Garcia 2024-08-01 02:45:36 -07:00
parent 5be4c980ca
commit 09c3f05759
12 changed files with 120 additions and 47 deletions

View file

@ -0,0 +1,5 @@
# Using `sqlite-vec` in C
The `sqlite-vec` project is a single `sqlite-vec.c` and `sqlite-vec.h` file. They can be vendored into your C or C++ projects and compiled like normal.
"Amalgammation" builds are provided on the [`sqlite-vec` Releases page](https://github.com/asg017/sqlite-vec/releases).

View file

@ -2,6 +2,12 @@
[![Datasette](https://img.shields.io/pypi/v/datasette-sqlite-vec.svg?color=B6B6D9&label=Datasette+plugin&logoColor=white&logo=python)](https://datasette.io/plugins/datasette-sqlite-vec)
[Datasette](https://datasette.io/) users can install `sqlite-vec` into their Datasette instances with the `datasette-sqlite-vec` plugin:
```bash
datasette install datasette-sqlite-vec
```
After installing, future Datasette instances will have `sqlite-vec` SQL functions loaded in.
"Unsafe" functions like static blobs and NumPy file reading are not available with `datasette-sqlite-vec`.

View file

@ -1,6 +1,6 @@
# Using `sqlite-vec` in Go
[![Go Reference](https://pkg.go.dev/badge/github.com/asg017/sqlite-vec-go-bindings/cgo.svg)](https://pkg.go.dev/github.com/asg017/sqlite-vec-go-bindings/cgo) [![Go Reference](https://pkg.go.dev/badge/github.com/asg017/sqlite-vec-go-bindings/ncruces.svg)](https://pkg.go.dev/github.com/asg017/sqlite-vec-go-bindings/ncruces)
There are two ways you can embed `sqlite-vec` into Go applications: a CGO option
for libraries like
@ -8,7 +8,9 @@ for libraries like
WASM-based option with
[`github.com/ncruces/go-sqlite3`](https://github.com/ncruces/go-sqlite3).
## Option 1: CGO
## Option 1: CGO {#cgo}
[![Go Reference](https://pkg.go.dev/badge/github.com/asg017/sqlite-vec-go-bindings/cgo.svg)](https://pkg.go.dev/github.com/asg017/sqlite-vec-go-bindings/cgo)
If using [`github.com/mattn/go-sqlite3`](https://github.com/mattn/go-sqlite3) or another CGO-based SQLite library, then use the `github.com/asg017/sqlite-vec-go-bindings/cgo` module to embed `sqlite-vec` into your Go application.
@ -44,11 +46,17 @@ func main() {
if err != nil {
log.Fatal(err)
}
log.Printf("sqlite_version=%s, vec_version=%s\n",vecVersion)
log.Printf("vec_version=%s\n",vecVersion)
}
```
## Option 2: WASM based with `ncruces/go-sqlite3`
See
[`simple-go-cgo/demo.go`](https://github.com/asg017/sqlite-vec/blob/main/examples/simple-go-cgo/demo.go)
for a more complete Go CGO demo.
## Option 2: WASM based with `ncruces/go-sqlite3` {#ncruces}
[![Go Reference](https://pkg.go.dev/badge/github.com/asg017/sqlite-vec-go-bindings/ncruces.svg)](https://pkg.go.dev/github.com/asg017/sqlite-vec-go-bindings/ncruces)
[`github.com/ncruces/go-sqlite3`](https://github.com/ncruces/go-sqlite3) is an alternative SQLite Go driver that avoids CGO by using a custom WASM build of SQLite. To use `sqlite-vec` from this library, use the specicial WASM binary provided in `github.com/asg017/sqlite-vec-go-bindings/ncruces`.
@ -73,7 +81,7 @@ func main() {
log.Fatal(err)
}
stmt, _, err := db.Prepare(`SELECT sqlite_version(), vec_version()`)
stmt, _, err := db.Prepare(`SELECT vec_version()`)
if err != nil {
log.Fatal(err)
}
@ -84,13 +92,22 @@ func main() {
}
```
See
[`simple-go-ncruces/demo.go`](https://github.com/asg017/sqlite-vec/blob/main/examples/simple-go-ncruces/demo.go)
for a more complete Go ncruces demo.
The `github.com/asg017/sqlite-vec-go-bindings/ncruces` package embeds a custom WASM build of SQLite, so there's no need to use `github.com/ncruces/go-sqlite3/embed`.
## Working with vectors in Go
## Working with vectors in Go
If vectors are provided as a list of floats, use `SerializeFloat32(list)` to serialize them into the compact BLOB format that `sqlite-vec` expects.
```go
TODO
values := []float32{0.1, 0.1, 0.1, 0.1}
v, err := sqlite_vec.SerializeFloat32(values)
if err != nil {
log.Fatal(err)
}
stmt.BindInt(1, id)
```

View file

@ -54,10 +54,9 @@ and use
accessor to bind as a parameter to `sqlite-vec` SQL functions.
```js
// TODO
const embedding = new Float32Array([0.1, 0.2, 0.3, 0.4]);
const stmt = db.prepare("select vec_length(?)");
console.log(stmt.run(embedding.buffer));
console.log(stmt.run(embedding.buffer)); // 4
```
## Node.js
@ -66,6 +65,19 @@ Here's a quick recipe of using `sqlite-vec` with
[`better-sqlite3`](https://github.com/WiseLibs/better-sqlite3) in Node.js.
```js
import * as sqliteVec from "sqlite-vec";
import Database from "better-sqlite3";
const db = new Database(":memory:");
sqliteVec.load(db);
const embedding = new Float32Array([0.1, 0.2, 0.3, 0.4]);
const { result } = db
.prepare("select vec_length(?)",)
.get(embedding);
console.log(result); // 4
```
See
@ -76,13 +88,26 @@ for a more complete Node.js demo.
Here's a quick recipe of using `sqlite-vec` with
[`jsr:@db/sqlite`](https://jsr.io/@db/sqlite) in Deno. It will only work on Deno
version `1.44` or greater, because of a bug in previous Deno version.
version `1.44` or greater, because of a bug in previous Deno versions.
Keep in mind, the `better-sqlite3` example above also works in Deno, you just
need to prefix the `better-sqlite3` import with `npm:`, like
`import * from "npm:better-sqlite3"`.
```ts
import { Database } from "jsr:@db/sqlite@0.11";
import * as sqliteVec from "npm:sqlite-vec@0.0.1-alpha.9";
const db = new Database(":memory:");
db.enableLoadExtension = true;
sqliteVec.load(db);
db.enableLoadExtension = false;
const embedding = new Float32Array([0.1, 0.2, 0.3, 0.4]);
const [result] = db
.prepare("select vec_length(?)")
.value<[string]>(new Uint8Array(embedding.buffer)!);
console.log(result); // 4
```
See
@ -96,6 +121,22 @@ Here's a quick recipe of using `sqlite-vec` with
example above also works with Bun.
```ts
import { Database } from "bun:sqlite";
import * as sqliteVec from "sqlite-vec";
// MacOS *might* have to do this, as the builtin SQLite library on MacOS doesn't allow extensions
Database.setCustomSQLite("/usr/local/opt/sqlite3/lib/libsqlite3.dylib");
const db = new Database(":memory:");
sqliteVec.load(db);
const embedding = new Float32Array([0.1, 0.2, 0.3, 0.4]);
const { result } = db
.prepare("select vec_length(?)",)
.get(embedding);
console.log(result); // 4
```
See

View file

@ -30,6 +30,10 @@ vec_version, = db.execute("select vec_version()").fetchone()
print(f"vec_version={vec_version}")
```
See
[`simple-python/demo.py`](https://github.com/asg017/sqlite-vec/blob/main/examples/simple-python/demo.py)
for a more complete Python demo.
## Working with Vectors
### Lists

View file

@ -9,7 +9,7 @@ Ruby developers can use `sqlite-vec` with the [`sqlite-vec` Gem](https://rubygem
gem install sqlite-vec
```
You can then use `SqliteVss.load()` to load `sqlite-vss` SQL functions in a given SQLite connection.
You can then use `SqliteVec.load()` to load `sqlite-vec` SQL functions in a given SQLite connection.
```ruby
require 'sqlite3'
@ -25,6 +25,9 @@ puts result.first.first
```
See
[`simple-ruby/demo.rb`](https://github.com/asg017/sqlite-vec/blob/main/examples/simple-ruby/demo.rb)
for a more complete Ruby demo.
## Working with vectors in Ruby

View file

@ -34,7 +34,9 @@ fn main()-> Result<()> {
}
```
A full [`sqlite-vec` Rust demo](#TODO) is also available.
See
[`simple-rust/demo.rs`](https://github.com/asg017/sqlite-vec/blob/main/examples/simple-rust/demo.rs)
for a more complete Rust demo.
## Working with vectors in Rust

View file

@ -2,6 +2,9 @@
![sqlite-utils](https://img.shields.io/pypi/v/sqlite-utils-sqlite-vec.svg?color=B6B6D9&label=sqlite-utils+plugin&logoColor=white&logo=python)
[`sqlite-utils`](https://sqlite-utils.datasette.io/en/stable/) users can install `sqlite-vec` into their `sqlite-utils` projects with the `sqlite-utils-sqlite-vec` plugin:
```bash
sqlite-utils install sqlite-utils-sqlite-vec
```

View file

@ -1,5 +1,7 @@
# `sqlite-vec` in the Browser with WebAssembly
`sqlite-vec` can be statically compiled into [official SQLite WASM](https://sqlite.org/wasm/doc/trunk/index.md) builds. The process is a bit complicated, but the result is a vector search in the browser, which is pretty cool!
```html
<html>
<body>
@ -15,3 +17,16 @@
</body>
</html>
```
[*Open in CodePen*](https://codepen.io/asg017_ucsd/pen/MWMpJNY)
It's not possibly to dynamically load a SQLite extension into a WASM build of SQLite. So `sqlite-vec` must be statically compiled into custom WASM builds.
## The `sqlite-vec-wasm-demo` NPM package
A **demonstration** of `sqlite-vec` in WASM is provided with the `sqlite-vec-wasm-demo` NPM package. This package is a demonstration and may change at any time. It doesn't follow the [Semantic version of `sqlite-vec`](../versioning.md).
See
[`simple-wasm/index.html`](https://github.com/asg017/sqlite-vec/blob/main/examples/simple-wasm/index.html)
for a more complete WASM demo using this package.