schema-lint chassis v1.0: DropProperty Soft + code-tagged diagnostics (MR-694) (#90)

* schema-lint chassis v1 (WIP): tier surfacing + plan doc

First commit of the chassis v1 branch. Lands a small, foundational
slice without behavior change, plus a planning doc that lays out the
remaining 7 commits in sequence so the PR can be reviewed
incrementally.

This commit:

- Adds SchemaMigrationStep::diagnostic() returning the full
  &'static DiagnosticCode (family + tier + severity) for
  UnsupportedChange steps with codes. Renderers can now reach the
  tier without re-implementing the code → tier lookup.

- CLI `omnigraph schema plan` output now displays tier alongside
  code:

    unsupported change on node:Person.age [OG-DS-104, destructive]:
        removing property 'Person.age' is not supported in schema
        migration v1

  Operators see at-a-glance the kind of risk each rejection
  represents — not just the rule identifier.

- No behavior change. All 11 existing schema_apply tests still pass.

Planning doc at docs/schema-lint-v1-plan.md tracks the 7 remaining
commits to bring v1 to feature-complete:

  1. (this commit) Tier surfacing in plan output.
  2. Soft / Hard mode enum on drop steps.
  3. Tombstone fields on catalog IR.
  4. Planner emits DropProperty { Soft } by default.
  5. Apply path implements Soft mode.
  6. Convert PR #62 destructive-rejection tests.
  7. --allow-data-loss flag + Hard mode.
  8. (optional) Tombstone unhide / restore command.

Delete the planning doc when v1 lands. Intentionally checked in to
the WIP branch so the scope is reviewable; not intended as a
permanent doc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* schema-lint v1 commit 2: DropMode + dormant Drop* variants

Second commit of the chassis v1 branch. Lands the type-level shape
of soft/hard drops without wiring them up. Variants are reachable
from emitters but the planner doesn't produce them yet; the apply
path returns an explicit not-yet-implemented error if one shows up
via deserialization.

Added:

- `DropMode { Soft, Hard }` — orthogonal to `SafetyTier`. Tier
  classifies the rule's risk class; mode is the operator's intent
  for data treatment.
    - `Soft` → catalog tombstone, data retained. Tier: safe.
    - `Hard` → Lance-level removal. Tier: destructive; will require
      --allow-data-loss to apply (commit 7).

- `SchemaMigrationStep::DropType { type_kind, name, mode }` and
  `SchemaMigrationStep::DropProperty { type_kind, type_name,
  property_name, mode }` variants.

- Re-export `DropMode` from `omnigraph_compiler::DropMode` so
  downstream crates don't reach into the catalog submodule.

- CLI `render_schema_plan_step` arms for both variants, surfacing
  the mode in plan output: `drop property 'Person.age' of node
  'Person' (soft mode)`.

- `apply_schema_with_lock` exhaustive match arm for the two new
  variants that returns `manifest_internal` with a clear
  not-yet-implemented message. If a SchemaIR JSON containing
  Drop{Type,Property} arrives (e.g. from a future tool or hand-
  written), the apply path fails explicitly rather than silently
  misclassifying.

- Two new in-source tests:
    - `drop_steps_round_trip_through_serde` — pins the wire shape
      for all four (variant × mode) combinations.
    - `drop_mode_serde_uses_snake_case` — pins external-tool-
      friendly serialization (`"soft"` / `"hard"`).

Build: clean, only pre-existing warnings.
Tests:
- omnigraph-compiler schema_plan: 6/6 (4 existing + 2 new).
- omnigraph-engine schema_apply: 11/11 (unchanged — planner still
  emits UnsupportedChange for removal paths).

Next commit (commit 3 per docs/schema-lint-v1-plan.md): add the
`tombstoned: bool` fields to NodeIR / EdgeIR / PropertyIR for the
catalog representation of soft-mode tombstones.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* plan doc: reframe v1 around Lance native drop_columns

After a substrate audit of the Lance data-evolution guide on
2026-05-13, the v1 plan was simplified. Two key findings:

1. Lance's `drop_columns()` is already metadata-only and reversible
   via time travel until cleanup. No need for a parallel
   `tombstoned: bool` field in our catalog IR — Lance's version
   graph IS the tombstone.

2. The full schema_apply substrate migration (add_columns,
   drop_columns, alter_columns vs. stage_overwrite across all step
   types) is consolidated in MR-948 as a sibling issue. v1 only
   uses the relevant slice (drop_columns for OG-DS-1XX).

Net plan changes:

- Commit 3 (original): tombstone fields on catalog IR → dropped.
  No catalog IR change needed. The Lance drop_columns commit IS the
  tombstone.

- Commit 5 (original): apply path writes tombstoned: true → replaced
  with: apply path calls Dataset::drop_columns([name]).

- Commit 7 Hard mode: stage_overwrite removing the column → replaced
  with: drop_columns + compact_files + cleanup_old_versions. Same
  APIs omnigraph cleanup already uses.

- Commit 8 (original): omnigraph schema unhide → dropped. Time
  travel is the undo (omnigraph snapshot --at <commit>).

Net result: 8 commits → 5 commits. ~250 LoC less surface. More
substrate-aligned.

The chassis types from commit 2 (DropMode enum, DropType /
DropProperty variants) are kept exactly as designed; only the
implementation strategy changed.

The Lance docs quote is included in the doc so future readers see
the substrate behavior cited verbatim.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* schema-lint v1 commit 3: emit + apply DropProperty { Soft }

Wire the dormant DropProperty variant end-to-end for the Soft case.
Per docs/schema-lint-v1-plan.md, commit #3 of the schema-lint chassis
v1 series (MR-694).

Planner (schema_plan.rs):
- plan_properties: emit DropProperty { type_kind, type_name,
  property_name, mode: Soft } instead of UnsupportedChange when a
  property exists in accepted but not in desired. Plan is now
  supported = true for drop-only changes.

Apply (schema_apply.rs):
- Route DropProperty { Soft } through rewritten_tables. The existing
  batch_for_schema_apply_rewrite path already iterates the *target*
  schema fields, so a property absent from desired_catalog is
  naturally projected away. The prior Lance version retains the
  dropped column for time-travel reversibility (until cleanup runs).
- DropType still errors (lands in commit #4 with different mechanics:
  __manifest entry removal instead of column projection).
- DropProperty { Hard } still errors (lands in commit #5 with
  --allow-data-loss CLI flag + immediate compact_files +
  cleanup_old_versions).

Tests:
- Planner unit test plan_emits_soft_drop_for_removed_nullable_property
  asserts the variant emission + supported = true + no UnsupportedChange.
- Integration test apply_schema_drops_a_nullable_property_softly_
  preserves_prior_version (replaces the former
  apply_schema_rejects_dropping_a_property_with_data) asserts:
  (a) plan contains DropProperty { Soft }
  (b) apply succeeds + manifest advances + row count unchanged
  (c) current dataset schema lacks the dropped column
  (d) snapshot_at_version(pre_drop) still has the dropped column
  (e) reopen consistency — drop preserved across engine restart

Recovery: rides on SidecarKind::SchemaApply per MR-847. No new
sidecar kind needed; the entire apply path is already sidecar-wrapped.

Substrate alignment: this commit uses the stage_overwrite full-rewrite
path (full_rewrite cost class) rather than Lance native drop_columns
(catalog_only cost class). MR-948 is the follow-up substrate-alignment
refactor that introduces a LanceColumnOp surface and switches the
metadata-only case onto drop_columns. Functional outcome is identical;
cost-class improvement deferred.

Test results:
- cargo test -p omnigraph-compiler --lib: 238 passed
- cargo test -p omnigraph-engine --test schema_apply: 11 passed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs: move schema-lint-v1-plan into docs/dev/ + add to index

Post-rebase fixup for the docs split (#93). The plan doc was added
to docs/ at the top level before main reorganized to docs/{user,dev}/.
This moves it into docs/dev/ and adds an entry to docs/dev/index.md
under a new "Active Implementation Plans" section so the
check-agents-md.sh link check passes.

Per the original commit message (617a77d), the plan doc is intentionally
temporary — it will be deleted when v1 lands.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

crates/omnigraph-compiler/src/catalog/schema_plan.rs

@@ -16,6 +16,29 @@ pub enum SchemaTypeKind {
     Edge,
 }
 
+/// How a drop step interacts with data.
+///
+/// - **`Soft`** — catalog tombstone only. The type / property is hidden
+///   from queries but the underlying Lance column / dataset is retained
+///   on disk. Reversible via `omnigraph schema unhide` (forthcoming).
+///   Tier: `safe`.
+/// - **`Hard`** — actual data removal. The Lance column is rewritten
+///   without the property, or the Lance dataset is dropped. Irreversible
+///   short of branch / snapshot restore. Tier: `destructive`; requires
+///   `--allow-data-loss` to apply.
+///
+/// The planner emits `Soft` by default; `--allow-data-loss` on the apply
+/// CLI promotes drops to `Hard`. This is the dimension orthogonal to
+/// `SafetyTier` from the schema-lint chassis (`crate::lint`): tier
+/// describes the rule's class; mode describes the operator's intent for
+/// data treatment.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "snake_case")]
+pub enum DropMode {
+    Soft,
+    Hard,
+}
+
 #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
 pub struct SchemaMigrationPlan {
     pub supported: bool,
@@ -62,6 +85,28 @@ pub enum SchemaMigrationStep {
         property_name: String,
         annotations: Vec<Annotation>,
     },
+    /// Remove a node or edge type. Soft mode tombstones in the catalog
+    /// and retains data on disk; Hard mode drops the Lance dataset and
+    /// requires `--allow-data-loss`.
+    ///
+    /// Dormant in this commit — emitted by the planner in a later
+    /// commit (see `docs/schema-lint-v1-plan.md`).
+    DropType {
+        type_kind: SchemaTypeKind,
+        name: String,
+        mode: DropMode,
+    },
+    /// Remove a property from an existing type. Soft mode tombstones
+    /// the property in the catalog and retains the Lance column; Hard
+    /// mode rewrites the column out and requires `--allow-data-loss`.
+    ///
+    /// Dormant in this commit.
+    DropProperty {
+        type_kind: SchemaTypeKind,
+        type_name: String,
+        property_name: String,
+        mode: DropMode,
+    },
     UnsupportedChange {
         entity: String,
         reason: String,
@@ -93,6 +138,24 @@ impl SchemaMigrationStep {
             _ => None,
         }
     }
+
+    /// If this step carries a schema-lint code, return the full
+    /// catalog entry — including family, safety tier, and default
+    /// severity. Used by renderers that want to display richer
+    /// context than just the code string (e.g. `omnigraph schema
+    /// plan` annotating each line with its tier).
+    ///
+    /// Returns `None` for steps that carry no code (the 12 of 17
+    /// `UnsupportedChange` paths still untagged in v0, plus every
+    /// non-`UnsupportedChange` variant).
+    pub fn diagnostic(&self) -> Option<&'static crate::lint::DiagnosticCode> {
+        match self {
+            Self::UnsupportedChange {
+                code: Some(c), ..
+            } => crate::lint::lookup(c),
+            _ => None,
+        }
+    }
 }
 
 pub fn plan_schema_migration(
@@ -499,18 +562,22 @@ fn plan_properties(
         .iter()
         .filter(|property| !consumed.contains(&property.name))
     {
-        steps.push(SchemaMigrationStep::UnsupportedChange {
-            entity: format!(
-                "{}:{}.{}",
-                schema_type_kind_key(type_kind),
-                type_name,
-                leftover.name
-            ),
-            reason: format!(
-                "removing property '{}.{}' is not supported in schema migration v1",
-                type_name, leftover.name
-            ),
-            code: Some(crate::lint::codes::OG_DS_104.code.to_string()),
+        // Property removed from the desired schema: emit
+        // DropProperty { Soft } per docs/schema-lint-v1-plan.md
+        // commit #3. The Soft mode reuses the existing
+        // stage_overwrite rewrite path — batch_for_schema_apply_rewrite
+        // iterates target_schema.fields(), so the dropped column is
+        // naturally projected away. The prior Lance version retains
+        // the column until cleanup_old_versions runs, matching the
+        // OG-DS-104 destructive-tier expectation that data remains
+        // recoverable via time travel until cleanup. Hard mode (with
+        // immediate compact_files + cleanup_old_versions) lands in
+        // commit #5, gated by --allow-data-loss.
+        steps.push(SchemaMigrationStep::DropProperty {
+            type_kind,
+            type_name: type_name.to_string(),
+            property_name: leftover.name.clone(),
+            mode: DropMode::Soft,
         });
     }
 
@@ -863,6 +930,67 @@ node Account @rename_from("User") {
         }));
     }
 
+    #[test]
+    fn plan_emits_soft_drop_for_removed_nullable_property() {
+        // Removing a property from the desired schema emits
+        // DropProperty { Soft } (schema-lint v1 chassis commit #3,
+        // MR-694). The plan is `supported = true` — the apply path
+        // handles soft drop via the existing stage_overwrite rewrite
+        // projection. Verified at the integration level by
+        // `apply_schema_drops_a_nullable_property_softly_preserves_prior_version`
+        // in `crates/omnigraph/tests/schema_apply.rs`.
+        let accepted = build_schema_ir(
+            &parse_schema(
+                r#"
+node Person {
+    name: String @key
+    age: I32?
+}
+"#,
+            )
+            .unwrap(),
+        )
+        .unwrap();
+        let desired = build_schema_ir(
+            &parse_schema(
+                r#"
+node Person {
+    name: String @key
+}
+"#,
+            )
+            .unwrap(),
+        )
+        .unwrap();
+
+        let plan = plan_schema_migration(&accepted, &desired).unwrap();
+        assert!(
+            plan.supported,
+            "drop-property plan must be supported: {plan:?}"
+        );
+        assert!(
+            plan.steps.iter().any(|step| matches!(
+                step,
+                SchemaMigrationStep::DropProperty {
+                    type_kind: SchemaTypeKind::Node,
+                    type_name,
+                    property_name,
+                    mode: DropMode::Soft,
+                    ..
+                } if type_name == "Person" && property_name == "age"
+            )),
+            "expected DropProperty {{ Soft }} step in plan: {plan:?}",
+        );
+        // Negative: no UnsupportedChange anywhere in the plan.
+        assert!(
+            !plan
+                .steps
+                .iter()
+                .any(|step| matches!(step, UnsupportedChange { .. })),
+            "soft drop must not emit UnsupportedChange: {plan:?}",
+        );
+    }
+
     #[test]
     fn plan_rejects_required_property_addition() {
         let accepted = build_schema_ir(
@@ -935,4 +1063,56 @@ node Person @description("new") {
             }],
         }));
     }
+
+    #[test]
+    fn drop_steps_round_trip_through_serde() {
+        // The DropType / DropProperty variants are dormant in this
+        // commit — the planner doesn't emit them yet — but their
+        // serde shape needs to be stable from day one. A future
+        // SchemaIR JSON containing one of these must deserialize
+        // back to the same value. This test pins the wire format
+        // so a v0 schema-ir consumer never sees a surprise variant
+        // shape after v1 ships.
+        let steps = vec![
+            SchemaMigrationStep::DropType {
+                type_kind: SchemaTypeKind::Node,
+                name: "Person".to_string(),
+                mode: DropMode::Soft,
+            },
+            SchemaMigrationStep::DropType {
+                type_kind: SchemaTypeKind::Edge,
+                name: "Knows".to_string(),
+                mode: DropMode::Hard,
+            },
+            SchemaMigrationStep::DropProperty {
+                type_kind: SchemaTypeKind::Node,
+                type_name: "Person".to_string(),
+                property_name: "age".to_string(),
+                mode: DropMode::Soft,
+            },
+            SchemaMigrationStep::DropProperty {
+                type_kind: SchemaTypeKind::Interface,
+                type_name: "Named".to_string(),
+                property_name: "alias".to_string(),
+                mode: DropMode::Hard,
+            },
+        ];
+
+        for step in steps {
+            let json = serde_json::to_string(&step).expect("serialize");
+            let round_trip: SchemaMigrationStep =
+                serde_json::from_str(&json).expect("deserialize");
+            assert_eq!(step, round_trip, "round-trip mismatch on {json}");
+        }
+    }
+
+    #[test]
+    fn drop_mode_serde_uses_snake_case() {
+        // External tools may write SchemaIR JSON by hand. Pin the
+        // wire form so we don't silently break them later.
+        assert_eq!(serde_json::to_string(&DropMode::Soft).unwrap(), "\"soft\"");
+        assert_eq!(serde_json::to_string(&DropMode::Hard).unwrap(), "\"hard\"");
+        let soft: DropMode = serde_json::from_str("\"soft\"").unwrap();
+        assert_eq!(soft, DropMode::Soft);
+    }
 }

crates/omnigraph-compiler/src/lib.rs

@@ -16,7 +16,7 @@ pub use catalog::schema_ir::{
     schema_ir_pretty_json,
 };
 pub use catalog::schema_plan::{
-    SchemaMigrationPlan, SchemaMigrationStep, SchemaTypeKind, plan_schema_migration,
+    DropMode, SchemaMigrationPlan, SchemaMigrationStep, SchemaTypeKind, plan_schema_migration,
 };
 pub use lint::{DiagnosticCode, Family, SafetyTier, Severity};
 pub use ir::ParamMap;