apunkt/omnigraph
1
0
Fork 0
mirror of https://github.com/ModernRelay/omnigraph.git synced 2026-06-09 01:35:18 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Raise LANCE_MEM_POOL_SIZE to 1 GB in .cargo/config.toml

Browse source 
Fixes flaky omnigraph-server integration suite under parallel cargo
test. Lance defaults to a 100 MB FairSpillPool per Omnigraph instance
(lance-datafusion/src/exec.rs:316). That's fine in prod (one server
process, bounded concurrent sorts) but too small when cargo test
spawns many Omnigraph instances in parallel, each running concurrent
BTree index builds during load.

Failure signature:
  Lance("create BTree index on node:Person(id): ... LanceError(IO):
  Not enough memory to continue external sort. ... 0.0 B remain
  available for the total pool")

Before: 10/41 OOM-fail on parallel run; passed with --test-threads=1.
After:  41/41 pass in parallel in ~3s.

[env] in .cargo/config.toml applies to cargo-launched processes only.
Shipped binaries (release tarballs, Docker images) are unaffected —
they inherit whatever the runtime env provides, defaulting to Lance's
100 MB when unset.
This commit is contained in:
andrew 2026-04-19 22:27:49 +03:00
parent 567ebe5f24
commit dbde85b68d
1 changed files with 22 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

22
.cargo/config.toml Normal file
View file

@ -0,0 +1,22 @@
# Cargo config for the omnigraph workspace.
#
# `[env]` entries here apply to anything cargo launches in this repo —
# `cargo build`, `cargo test`, `cargo run`. Shipped binaries are NOT
# affected: the env var lives only in the cargo-launched process, not in
# release artifacts or Docker images.
[env]
# Lance's DataFusion memory pool per `Omnigraph` instance. Lance defaults
# to 100 MB (`lance-datafusion/src/exec.rs`), which is fine for a single
# server process in production.
#
# Under `cargo test` it's a problem: the omnigraph-server integration
# suite spins up many `Omnigraph` instances in parallel (one per test,
# default `num_cpus` threads), each with concurrent BTree index builds
# sharing that test's 100 MB pool — saturates quickly and surfaces as
# `Not enough memory to continue external sort` / `0.0 B remain
# available for the total pool`.
#
# 1 GB is comfortable headroom per-test pool; prod and CI-for-prod
# builds set the variable themselves (or keep the 100 MB default).
LANCE_MEM_POOL_SIZE = "1073741824"