mirror of
https://github.com/Kaelio/ktx.git
synced 2026-06-07 07:55:13 +02:00
feat(cli): add ktx admin reindex (#160)
* feat(cli): add admin reindex * fix: keep lexical-only reindex incremental
This commit is contained in:
Andrey Avtomonov
GitHub
parent
3db3e724cb
commit
6dbb0c8b3a
53 changed files with 1640 additions and 393 deletions
121
docs-site/content/docs/cli-reference/ktx-admin.mdx
Normal file
121
docs-site/content/docs/cli-reference/ktx-admin.mdx
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
---
|
||||
title: "ktx admin"
|
||||
description: "Low-level project initialization, runtime, and index management."
|
||||
---
|
||||
|
||||
`ktx admin` contains low-level project initialization, managed Python runtime,
|
||||
and local index management commands. Context building lives at the root as
|
||||
[`ktx ingest`](/docs/cli-reference/ktx-ingest). Most users should start with
|
||||
`ktx setup`; use `ktx admin` when preparing local fixtures, checking the bundled
|
||||
runtime, rebuilding local indexes, or debugging runtime state.
|
||||
|
||||
## Command signature
|
||||
|
||||
```bash
|
||||
ktx admin <subcommand> [options]
|
||||
```
|
||||
|
||||
## Subcommands
|
||||
|
||||
| Subcommand | Description |
|
||||
|-----------|-------------|
|
||||
| `init [directory]` | Initialize a Git-backed KTX project directory for maintenance scripts |
|
||||
| `schema` | Print a JSON Schema describing `ktx.yaml` |
|
||||
| `runtime` | Install, start, stop, and inspect the KTX-managed Python runtime |
|
||||
| `reindex` | Sync local wiki and semantic-layer search indexes from disk |
|
||||
|
||||
## `admin init`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--force` | Rewrite `ktx.yaml` and scaffold files in an existing project | `false` |
|
||||
|
||||
## `admin schema`
|
||||
|
||||
`ktx admin schema` does not require a `ktx.yaml` file or a configured project
|
||||
directory. Use it from any directory to generate editor or agent schema files.
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--output <file>` | Write the schema to a file instead of stdout | - |
|
||||
|
||||
## `admin runtime` Subcommands
|
||||
|
||||
| Subcommand | Description |
|
||||
|-----------|-------------|
|
||||
| `install` | Install the bundled Python runtime wheel into the managed runtime |
|
||||
| `start` | Start the KTX-managed Python HTTP daemon |
|
||||
| `stop` | Stop the KTX-managed Python HTTP daemon |
|
||||
| `status` | Show managed Python runtime status and readiness checks |
|
||||
|
||||
## `admin runtime` Options
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--feature <feature>` | Runtime feature level for `install` and `start` (`core` or `local-embeddings`) | `core` |
|
||||
| `--json` | Print JSON output for `status` | `false` |
|
||||
| `--yes` | Accepted by `install` for scripted install commands | `false` |
|
||||
| `--force` | Reinstall for `install`, or restart for `start` | `false` |
|
||||
| `--all` | Stop all recorded or discoverable KTX daemon processes with `stop` | `false` |
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
ktx admin init
|
||||
ktx admin init ./my-project
|
||||
ktx admin init --force
|
||||
|
||||
ktx admin schema
|
||||
ktx admin schema --output ./ktx.schema.json
|
||||
|
||||
ktx admin runtime install --yes
|
||||
ktx admin runtime install --feature local-embeddings --yes
|
||||
ktx admin runtime status
|
||||
ktx admin runtime start
|
||||
ktx admin runtime start --feature local-embeddings
|
||||
ktx admin runtime stop
|
||||
ktx admin runtime stop --all
|
||||
|
||||
ktx admin reindex
|
||||
ktx admin reindex --force
|
||||
ktx admin reindex --output plain
|
||||
ktx admin reindex --json
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Runtime commands print the runtime root, installed features, daemon URL, daemon
|
||||
pid, and log paths where relevant. `ktx admin runtime status --json` includes the
|
||||
runtime status plus readiness checks.
|
||||
|
||||
## `admin reindex`
|
||||
|
||||
`ktx admin reindex` syncs local wiki and semantic-layer search indexes from
|
||||
files on disk into `.ktx/db.sqlite`. The command discovers `wiki/global/`, each
|
||||
`wiki/user/<userId>/` directory, and each `semantic-layer/<connectionId>/`
|
||||
directory except `_schema`.
|
||||
|
||||
```bash
|
||||
ktx admin reindex
|
||||
ktx admin reindex --force
|
||||
ktx admin reindex --output plain
|
||||
ktx admin reindex --json
|
||||
```
|
||||
|
||||
By default, KTX compares stored search text with the files on disk. It only
|
||||
re-embeds changed rows and removes rows for files that no longer exist. With
|
||||
`--force`, KTX clears each discovered scope first and then rebuilds it.
|
||||
|
||||
When embeddings are not configured, KTX still writes lexical FTS rows and
|
||||
prints an embeddings warning. If a scope fails, KTX keeps processing the
|
||||
remaining scopes and exits with code `1` after output is written. If the local
|
||||
state database cannot open or the configured managed embedding runtime is
|
||||
missing, KTX prints the error and exits with code `1`.
|
||||
|
||||
## Common errors
|
||||
|
||||
| Error | Cause | Recovery |
|
||||
|-------|-------|----------|
|
||||
| Runtime status reports missing pieces | Packages, Python environment, or linked CLI are not ready | Run `pnpm install`, `pnpm run setup:dev`, `uv sync --all-groups`, then `ktx admin runtime status` |
|
||||
| Runtime daemon does not start | The managed Python runtime is missing or stale | Run `ktx admin runtime install --yes`, then `ktx admin runtime start` |
|
||||
| Multiple daemon processes remain | Older daemon state files or stray processes exist | Run `ktx admin runtime stop --all`, then start the runtime again |
|
||||
|
|
@ -1,91 +0,0 @@
|
|||
---
|
||||
title: "ktx dev"
|
||||
description: "Low-level project initialization and runtime management."
|
||||
---
|
||||
|
||||
`ktx dev` contains low-level project initialization and managed Python runtime
|
||||
commands. Context building lives at the root as
|
||||
[`ktx ingest`](/docs/cli-reference/ktx-ingest). Most users should start with
|
||||
`ktx setup`; use `ktx dev` when preparing local fixtures, checking the bundled
|
||||
runtime, or debugging runtime state.
|
||||
|
||||
## Command signature
|
||||
|
||||
```bash
|
||||
ktx dev <subcommand> [options]
|
||||
```
|
||||
|
||||
## Subcommands
|
||||
|
||||
| Subcommand | Description |
|
||||
|-----------|-------------|
|
||||
| `init [directory]` | Initialize a Git-backed KTX project directory for maintenance scripts |
|
||||
| `schema` | Print a JSON Schema describing `ktx.yaml` |
|
||||
| `runtime` | Install, start, stop, and inspect the KTX-managed Python runtime |
|
||||
|
||||
## `dev init`
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--force` | Rewrite `ktx.yaml` and scaffold files in an existing project | `false` |
|
||||
|
||||
## `dev schema`
|
||||
|
||||
`ktx dev schema` does not require a `ktx.yaml` file or a configured project
|
||||
directory. Use it from any directory to generate editor or agent schema files.
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--output <file>` | Write the schema to a file instead of stdout | - |
|
||||
|
||||
## `dev runtime` Subcommands
|
||||
|
||||
| Subcommand | Description |
|
||||
|-----------|-------------|
|
||||
| `install` | Install the bundled Python runtime wheel into the managed runtime |
|
||||
| `start` | Start the KTX-managed Python HTTP daemon |
|
||||
| `stop` | Stop the KTX-managed Python HTTP daemon |
|
||||
| `status` | Show managed Python runtime status and readiness checks |
|
||||
|
||||
## `dev runtime` Options
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--feature <feature>` | Runtime feature level for `install` and `start` (`core` or `local-embeddings`) | `core` |
|
||||
| `--json` | Print JSON output for `status` | `false` |
|
||||
| `--yes` | Accepted by `install` for scripted install commands | `false` |
|
||||
| `--force` | Reinstall for `install`, or restart for `start` | `false` |
|
||||
| `--all` | Stop all recorded or discoverable KTX daemon processes with `stop` | `false` |
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
ktx dev init
|
||||
ktx dev init ./my-project
|
||||
ktx dev init --force
|
||||
|
||||
ktx dev schema
|
||||
ktx dev schema --output ./ktx.schema.json
|
||||
|
||||
ktx dev runtime install --yes
|
||||
ktx dev runtime install --feature local-embeddings --yes
|
||||
ktx dev runtime status
|
||||
ktx dev runtime start
|
||||
ktx dev runtime start --feature local-embeddings
|
||||
ktx dev runtime stop
|
||||
ktx dev runtime stop --all
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Runtime commands print the runtime root, installed features, daemon URL, daemon
|
||||
pid, and log paths where relevant. `ktx dev runtime status --json` includes the
|
||||
runtime status plus readiness checks.
|
||||
|
||||
## Common errors
|
||||
|
||||
| Error | Cause | Recovery |
|
||||
|-------|-------|----------|
|
||||
| Runtime status reports missing pieces | Packages, Python environment, or linked CLI are not ready | Run `pnpm install`, `pnpm run setup:dev`, `uv sync --all-groups`, then `ktx dev runtime status` |
|
||||
| Runtime daemon does not start | The managed Python runtime is missing or stale | Run `ktx dev runtime install --yes`, then `ktx dev runtime start` |
|
||||
| Multiple daemon processes remain | Older daemon state files or stray processes exist | Run `ktx dev runtime stop --all`, then start the runtime again |
|
||||
|
|
@ -153,7 +153,7 @@ KTX_INGEST_TRACE_LEVEL=trace ktx ingest metabase
|
|||
| Connection not configured | The connection id is not present in `ktx.yaml` | Add the connection with `ktx setup` or update `ktx.yaml` |
|
||||
| Deep readiness is missing | `--deep` or query history needs model, embedding, and scan-enrichment configuration | Run `ktx setup` or rerun with `--fast` |
|
||||
| Query history is unsupported | The selected database driver does not support query history | Run schema ingest without query-history flags |
|
||||
| Python runtime is missing | The selected ingest target needs runtime-backed SQL analysis or source parsing | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx dev runtime install` command |
|
||||
| Python runtime is missing | The selected ingest target needs runtime-backed SQL analysis or source parsing | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command |
|
||||
| No ingest target was selected | No connection id was provided and `--all` was omitted | Run `ktx ingest <connectionId>` or `ktx ingest --all` |
|
||||
| Source options were ignored | Depth and query-history flags were supplied for a non-database source | Omit database-only flags when ingesting source connections |
|
||||
| Text ingest stops early | `--fail-fast` was used and one item failed | Fix the failed item or rerun without `--fail-fast` to collect all failures |
|
||||
|
|
|
|||
|
|
@ -231,7 +231,7 @@ Use `ktx status` for repeatable readiness checks after setup exits.
|
|||
| Setup resumes an unexpected project | `KTX_PROJECT_DIR` or nearest `ktx.yaml` points to another directory | Pass `--project-dir <path>` explicitly |
|
||||
| Setup cannot run in CI | Required values are missing and `--no-input` disables prompts | Provide the relevant automation flags or create a fixture `ktx.yaml` |
|
||||
| Provider health check fails | Provider key, model id, Vertex project, or Vertex location is invalid | Fix the `env:` or `file:` reference and rerun setup |
|
||||
| Python runtime is missing | The selected setup needs runtime-backed agent, query-history, Looker, or local embedding features | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx dev runtime install` command |
|
||||
| Python runtime is missing | The selected setup needs runtime-backed agent, query-history, Looker, or local embedding features | Accept the interactive prompt, rerun with `--yes`, or run the suggested `ktx admin runtime install` command |
|
||||
| `--enable-query-history` is rejected | The selected database driver does not support query history | Use Postgres, BigQuery, or Snowflake, or rerun without query-history flags |
|
||||
| Source setup rejects location flags | Both `--source-path` and `--source-git-url` were supplied | Choose the local path or the Git URL, not both |
|
||||
| Agent integration missing | Setup skipped the agents step | Run `ktx setup --agents --target <target>` |
|
||||
|
|
|
|||
|
|
@ -165,4 +165,4 @@ ranking score rather than a percentage.
|
|||
| Validation fails | YAML references missing columns, invalid joins, or invalid SQL expressions | Fix the source YAML and rerun `ktx sl validate` |
|
||||
| Query compile fails | Measure, dimension, filter, or segment name is invalid | Search sources with `ktx sl search`, inspect the source YAML in your project files, then retry using declared fields |
|
||||
| Execution returns too many rows | `--max-rows` is missing or too high | Add `--max-rows` with a bounded value before executing |
|
||||
| Runtime install is blocked | Query execution needs the managed Python runtime and prompts are disabled | Run `ktx dev runtime install --feature core --yes`, or rerun `ktx sl query --yes` |
|
||||
| Runtime install is blocked | Query execution needs the managed Python runtime and prompts are disabled | Run `ktx admin runtime install --feature core --yes`, or rerun `ktx sl query --yes` |
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ ktx
|
|||
stop
|
||||
status
|
||||
logs
|
||||
dev
|
||||
admin
|
||||
init [directory]
|
||||
schema
|
||||
runtime
|
||||
|
|
@ -56,6 +56,7 @@ ktx
|
|||
start
|
||||
stop
|
||||
status
|
||||
reindex
|
||||
```
|
||||
|
||||
The public context-build entrypoint is `ktx ingest [connectionId]` or
|
||||
|
|
|
|||
|
|
@ -11,6 +11,6 @@
|
|||
"ktx-wiki",
|
||||
"ktx-status",
|
||||
"ktx-mcp",
|
||||
"ktx-dev"
|
||||
"ktx-admin"
|
||||
]
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue