trustgraph/ai-context/workbench-ui/docs/tech-specs/flow-class-editor.md

Flow Class Visual Editor Technical Specification

Overview

A React-based visual editor for creating and modifying TrustGraph flow class definitions using a node-and-edge graph interface. Built with React Flow, this component allows users to visually design dataflow patterns by dragging processors onto a canvas and connecting them with queues.

Core Technologies

• React Flow - Node-based editor framework
• TypeScript - Type safety for flow definitions
• Chakra UI v3 - UI components and theming
• Zustand - Editor state management
• Zod - Schema validation for flow class structure

Component Architecture

Directory Structure

src/components/flow-editor/
├── FlowClassEditor.tsx           # Main editor component
├── nodes/                         # Custom node components
│   ├── ClassProcessorNode.tsx    # Shared service nodes
│   ├── FlowProcessorNode.tsx     # Flow-specific nodes
│   └── InterfaceNode.tsx         # Entry/exit point nodes
├── edges/                         # Custom edge components
│   ├── PersistentQueueEdge.tsx   # Persistent queue connections
│   └── RequestResponseEdge.tsx   # Request/response pairs
├── panels/                        # Editor UI panels
│   ├── NodePalette.tsx           # Drag-and-drop processor library
│   ├── PropertiesPanel.tsx       # Node/edge configuration
│   └── ValidationPanel.tsx       # Real-time validation feedback
├── hooks/
│   ├── useFlowValidation.ts      # Validation logic
│   ├── useFlowExport.ts          # JSON export/import
│   └── useAutoLayout.ts          # Automatic graph layout
└── types/
    └── flowEditorTypes.ts        # TypeScript definitions

Visual Design

Node Types

1. Class Processor Node ({class})

{
  type: 'classProcessor',
  data: {
    processorName: string,      // e.g., "embeddings"
    queues: {
      request?: string,          // Queue pattern
      response?: string,         // Queue pattern
      [key: string]: string      // Additional queues
    }
  },
  style: {
    background: 'accent.subtle', // Shared service color
    border: '2px solid accent.solid',
    icon: <Share2 />            // Lucide icon indicating shared
  }
}

2. Flow Processor Node ({id})

{
  type: 'flowProcessor',
  data: {
    processorName: string,       // e.g., "chunker"
    queues: {
      input?: string,            // Input queue
      output?: string,           // Output queue
      [key: string]: string      // Additional queues
    }
  },
  style: {
    background: 'primary.subtle', // Flow-specific color
    border: '2px solid primary.solid',
    icon: <Box />                // Lucide icon for isolated
  }
}

3. Interface Node

{
  type: 'interfaceNode',
  data: {
    interfaceName: string,       // e.g., "document-load"
    interfaceType: 'fire-and-forget' | 'request-response',
    queuePattern?: string,       // For fire-and-forget
    request?: string,            // For request-response
    response?: string            // For request-response
  },
  style: {
    background: 'bg.muted',
    border: '2px dashed border.muted',
    icon: <Plug />               // Entry/exit point indicator
  }
}

Edge Types

1. Persistent Queue Edge

• Visual: Solid line with arrow
• Color: Based on namespace (flow: green, request: blue, response: purple)
• Label: Queue name displayed on hover
• Validation: Source/target compatibility checking

2. Non-Persistent Queue Edge

• Visual: Dashed line with arrow
• Color: Lighter variant of namespace colors
• Label: Queue name with non-persistent indicator
• Validation: Request/response pairing validation

User Edit Operations

1. Processor Management

Add Processor

• Drag from palette: Drag processor type from categorized library
• Double-click canvas: Quick-add with processor type selector
• Context menu: Right-click → Add Processor → Select type
• Keyboard shortcut: A key opens add processor dialog

Configure Processor

• Rename: Click processor name to edit inline
• Change type: Toggle between {class} and {id} via properties panel
• Queue management:

  // Add new queue to processor
  addQueue(processorId, {
    name: "custom-queue",
    direction: "input" | "output" | "bidirectional",
    pattern: "persistent://tg/flow/custom:{id}"
  });
  
  // Remove queue
  removeQueue(processorId, queueName);
  
  // Edit queue pattern
  updateQueue(processorId, queueName, newPattern);
  

Delete Processor

• Single: Select + Delete key
• Multiple: Multi-select + Delete key
• Context menu: Right-click → Delete
• Validation: Warn if processor has connections

2. Connection Management

Create Connection

• Drag connection: From output handle to input handle
• Validation rules:
  • Persistence compatibility (persistent ↔ persistent preferred)
  • Namespace compatibility (flow/request/response)
  • Template variable consistency ({class} ↔ {class}, {id} ↔ {id})
  • No self-connections
  • No duplicate connections

Configure Connection

• Auto-naming: Generate queue name from source/target processors
• Custom naming: Override auto-generated queue name
• Persistence mode: Toggle persistent/non-persistent
• Queue pattern template:

  generateQueuePattern({
    persistence: "persistent" | "non-persistent",
    tenant: "tg",
    namespace: "flow" | "request" | "response",
    topic: "document-embeddings",
    template: "{id}" | "{class}"
  });
  // Result: "persistent://tg/flow/document-embeddings:{id}"
  

Delete Connection

• Click to select + Delete key
• Context menu on edge
• Disconnect handle: Drag connection away from handle

3. Interface Operations

Add Interface

• Entry points: Document load, text input, etc.
• Exit points: Response outputs, storage endpoints
• Service interfaces: Request/response pairs

Configure Interface Type

// Fire-and-forget pattern
interface FireAndForgetInterface {
  type: "fire-and-forget";
  queue: string;  // Single queue pattern
}

// Request/response pattern
interface RequestResponseInterface {
  type: "request-response";
  request: string;   // Request queue pattern
  response: string;  // Response queue pattern
}

4. Bulk Operations

Multi-select Actions

• Box select: Click and drag to select multiple nodes
• Shift-click: Add to selection
• Cmd-click: Toggle selection
• Select all: Cmd+A

Group Operations

• Move together: Drag any selected node moves all
• Delete together: Delete key removes all selected
• Duplicate: Cmd+D duplicates selection
• Copy/paste: Cmd+C/Cmd+V for cross-flow copying

5. Layout Operations

Auto-layout

const layoutStrategies = {
  hierarchical: {
    direction: "LR" | "TB",  // Left-right or top-bottom
    nodeSpacing: 150,
    levelSpacing: 200
  },
  force: {
    strength: -1000,
    distance: 150
  },
  circular: {
    radius: 300,
    startAngle: 0
  }
};

Manual Arrangement

• Snap to grid: Optional grid snapping (toggle with G key)
• Alignment tools: Align selected nodes (top/bottom/left/right/center)
• Distribution: Distribute nodes evenly (horizontal/vertical)

6. Template Operations

Apply Template

const templates = {
  "document-rag": {
    description: "Document processing with RAG",
    processors: [
      { type: "pdf-decoder", id: "{id}" },
      { type: "chunker", id: "{id}" },
      { type: "embeddings", id: "{class}" },
      { type: "de-write", id: "{id}" }
    ],
    connections: [
      { from: "pdf-decoder.output", to: "chunker.input" },
      { from: "chunker.output", to: "embeddings.input" },
      { from: "embeddings.output", to: "de-write.input" }
    ]
  }
};

Create Template

• Select nodes/edges → Right-click → "Save as template"
• Provide template name and description
• Template saved to library for reuse

7. Validation Operations

Real-time Validation

interface ValidationRule {
  id: string;
  severity: "error" | "warning" | "info";
  check: (flowClass: FlowClass) => ValidationResult;
}

const validationRules = [
  {
    id: "no-orphans",
    severity: "warning",
    check: (flow) => findOrphanedNodes(flow)
  },
  {
    id: "queue-consistency",
    severity: "error",
    check: (flow) => validateQueuePatterns(flow)
  },
  {
    id: "template-consistency",
    severity: "error",
    check: (flow) => validateTemplateVariables(flow)
  }
];

Fix Suggestions

• Auto-fix: One-click fixes for common issues
• Quick actions: Context-aware suggestions
• Validation overlay: Visual indicators on invalid elements

8. History Management

Undo/Redo Stack

interface HistoryAction {
  type: "add" | "delete" | "update" | "connect" | "disconnect";
  before: FlowState;
  after: FlowState;
  timestamp: number;
}

const historyStack: HistoryAction[] = [];
const redoStack: HistoryAction[] = [];

// Track all operations
const executeOperation = (operation: Operation) => {
  const before = getCurrentState();
  performOperation(operation);
  const after = getCurrentState();
  
  historyStack.push({
    type: operation.type,
    before,
    after,
    timestamp: Date.now()
  });
  
  redoStack.length = 0; // Clear redo on new operation
};

9. Import/Export Operations

Import Flow Class

• From JSON file: Upload or paste JSON
• From Config API: Select from existing flow classes
• Validation: Verify structure before import
• Merge options: Replace or merge with existing

Export Flow Class

• To JSON: Download as .json file
• To Config API: Save directly to backend
• To clipboard: Copy JSON for sharing
• Format options: Minified or pretty-printed

10. Metadata Operations

Edit Flow Properties

interface FlowMetadata {
  id: string;           // Kebab-case identifier
  name: string;         // Human-readable name
  description: string;  // Detailed description
  tags: string[];       // Categorization tags
  version: string;      // Semantic version
  author: string;       // Creator identity
  created: Date;        // Creation timestamp
  modified: Date;       // Last modification
}

Tag Management

• Add tags: Type or select from existing
• Remove tags: Click X on tag chips
• Tag suggestions: Based on processors used
• Tag categories: System tags vs user tags

Core Features

1. Auto-Layout

const handleAutoLayout = () => {
  const layoutedElements = getLayoutedElements(nodes, edges, {
    direction: 'LR',  // Left to right
    nodeSpacing: 150,
    levelSpacing: 200,
    animate: true
  });
  setNodes(layoutedElements.nodes);
  setEdges(layoutedElements.edges);
};

2. Import/Export

// Export to flow class JSON
const exportFlowClass = () => {
  const flowClass = {
    class: extractClassProcessors(nodes),
    flow: extractFlowProcessors(nodes),
    interfaces: extractInterfaces(nodes),
    description: metadata.description,
    tags: metadata.tags
  };
  return JSON.stringify(flowClass, null, 2);
};

// Import from JSON
const importFlowClass = (json: string) => {
  const flowClass = JSON.parse(json);
  const { nodes, edges } = convertToReactFlow(flowClass);
  setNodes(nodes);
  setEdges(edges);
};

3. Connection Validation

const isValidConnection = (connection: Connection) => {
  const sourceNode = getNode(connection.source);
  const targetNode = getNode(connection.target);
  
  // Validate queue compatibility
  if (!areQueuesCompatible(sourceNode, targetNode)) {
    return false;
  }
  
  // Prevent circular dependencies
  if (createsCircularDependency(connection)) {
    return false;
  }
  
  return true;
};

4. Smart Templates

Pre-built flow patterns users can instantiate:

• Document RAG Pipeline: PDF → Chunker → Embeddings → Storage
• Graph RAG Pipeline: Knowledge extraction → Graph embeddings → Query
• Simple Q&A: Prompt → LLM → Response
• Custom Template: User-defined reusable patterns

State Management

interface FlowEditorState {
  // React Flow state
  nodes: Node[];
  edges: Edge[];
  
  // Editor state
  selectedElement: Node | Edge | null;
  validationErrors: ValidationError[];
  isDirty: boolean;
  
  // Metadata
  flowClassName: string;
  description: string;
  tags: string[];
  
  // Actions
  addNode: (type: NodeType, position: XYPosition) => void;
  updateNode: (nodeId: string, data: NodeData) => void;
  deleteNode: (nodeId: string) => void;
  addEdge: (edge: Edge) => void;
  deleteEdge: (edgeId: string) => void;
  validateFlow: () => ValidationResult;
  exportFlow: () => FlowClassDefinition;
  importFlow: (definition: FlowClassDefinition) => void;
}

Visual Indicators

Node States

• Normal: Default appearance
• Selected: Blue glow/border
• Invalid: Red border with error icon
• Connecting: Pulse animation on handles
• Hover: Slight scale increase

Edge States

• Normal: Default appearance
• Selected: Highlighted with thicker stroke
• Invalid: Red dashed line
• Animated: Flow animation for active connections

Queue Handle Types

• Input: Left side of node, inward arrow
• Output: Right side of node, outward arrow
• Bidirectional: Both sides, for request/response

Keyboard Shortcuts

• Delete - Delete selected elements
• Cmd+Z - Undo
• Cmd+Shift+Z - Redo
• Cmd+S - Save flow class
• Cmd+O - Open flow class
• Cmd+E - Export to JSON
• Space - Pan mode
• Cmd+A - Select all nodes
• Cmd+D - Duplicate selected nodes

Integration Points

Config API Integration

The flow class editor uses the existing Config API for all flow class operations:

State Management Hook

// src/state/flow-classes.ts
import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query";
import { useSocket } from "./socket";

export const useFlowClasses = () => {
  const socket = useSocket();
  
  return useQuery({
    queryKey: ["flow-classes"],
    queryFn: async () => {
      const response = await socket.request({
        operation: "get-config",
        path: "flow-classes"
      });
      return response.configuration as FlowClassDefinition[];
    }
  });
};

export const useFlowClass = (flowClassId: string) => {
  const socket = useSocket();
  
  return useQuery({
    queryKey: ["flow-class", flowClassId],
    queryFn: async () => {
      const response = await socket.request({
        operation: "get-config",
        path: `flow-classes/${flowClassId}`
      });
      return response.configuration as FlowClassDefinition;
    }
  });
};

export const useUpdateFlowClass = () => {
  const socket = useSocket();
  const queryClient = useQueryClient();
  
  return useMutation({
    mutationFn: async ({ id, flowClass }: { 
      id: string; 
      flowClass: FlowClassDefinition 
    }) => {
      return await socket.request({
        operation: "set-config",
        path: `flow-classes/${id}`,
        configuration: flowClass
      });
    },
    onSuccess: (_, variables) => {
      queryClient.invalidateQueries(["flow-class", variables.id]);
      queryClient.invalidateQueries(["flow-classes"]);
    }
  });
};

export const useDeleteFlowClass = () => {
  const socket = useSocket();
  const queryClient = useQueryClient();
  
  return useMutation({
    mutationFn: async (id: string) => {
      return await socket.request({
        operation: "delete-config",
        path: `flow-classes/${id}`
      });
    },
    onSuccess: () => {
      queryClient.invalidateQueries(["flow-classes"]);
    }
  });
};

Editor Component Integration

// src/components/flow-editor/FlowClassEditor.tsx
import { useFlowClass, useUpdateFlowClass } from "../../state/flow-classes";
import { useActivity } from "../../state/activity";
import { useNotification } from "../../state/notify";

export const FlowClassEditor = ({ flowClassId }: { flowClassId?: string }) => {
  const notify = useNotification();
  
  // Load existing flow class if ID provided
  const { data: flowClass, isLoading } = useFlowClass(flowClassId);
  const updateMutation = useUpdateFlowClass();
  
  // Track loading state
  useActivity(isLoading, "Loading flow class");
  useActivity(updateMutation.isPending, "Saving flow class");
  
  // Initialize React Flow with loaded data
  useEffect(() => {
    if (flowClass) {
      const { nodes, edges } = convertFlowClassToReactFlow(flowClass);
      setNodes(nodes);
      setEdges(edges);
      setMetadata({
        description: flowClass.description,
        tags: flowClass.tags
      });
    }
  }, [flowClass]);
  
  // Save handler
  const handleSave = async () => {
    const flowClassData = exportFlowClass();
    
    try {
      await updateMutation.mutateAsync({
        id: flowClassId || generateFlowClassId(),
        flowClass: flowClassData
      });
      notify.success("Flow class saved successfully");
    } catch (error) {
      notify.error("Failed to save flow class");
    }
  };
  
  return (
    <ReactFlow
      nodes={nodes}
      edges={edges}
      // ... rest of React Flow config
    />
  );
};

Flow Class List Integration

// src/components/flow-editor/FlowClassList.tsx
import { useFlowClasses, useDeleteFlowClass } from "../../state/flow-classes";

export const FlowClassList = () => {
  const { data: flowClasses, isLoading } = useFlowClasses();
  const deleteMutation = useDeleteFlowClass();
  
  useActivity(isLoading, "Loading flow classes");
  useActivity(deleteMutation.isPending, "Deleting flow class");
  
  return (
    <VStack>
      {flowClasses?.map(flowClass => (
        <Card key={flowClass.id}>
          <HStack>
            <Text>{flowClass.description}</Text>
            <Button onClick={() => openEditor(flowClass.id)}>
              Edit
            </Button>
            <Button onClick={() => deleteMutation.mutate(flowClass.id)}>
              Delete
            </Button>
          </HStack>
        </Card>
      ))}
    </VStack>
  );
};

Config API Request/Response Format

// Request to get all flow classes
{
  operation: "get-config",
  path: "flow-classes"
}

// Response
{
  configuration: [
    {
      id: "document-rag-flow",
      class: { /* class processors */ },
      flow: { /* flow processors */ },
      interfaces: { /* interfaces */ },
      description: "Document RAG pipeline",
      tags: ["rag", "documents"]
    }
  ]
}

// Request to update flow class
{
  operation: "set-config",
  path: "flow-classes/document-rag-flow",
  configuration: {
    class: { /* updated class processors */ },
    flow: { /* updated flow processors */ },
    interfaces: { /* updated interfaces */ },
    description: "Updated description",
    tags: ["rag", "documents", "v2"]
  }
}

Real-time Updates via WebSocket

The editor subscribes to configuration changes to handle external updates:

useEffect(() => {
  const subscription = socket.subscribe(
    `config/flow-classes/${flowClassId}`,
    (update) => {
      // Handle external updates to the flow class
      if (update.source !== currentSessionId) {
        notify.warning("Flow class updated externally. Refreshing...");
        queryClient.invalidateQueries(["flow-class", flowClassId]);
      }
    }
  );
  
  return () => subscription.unsubscribe();
}, [flowClassId]);

With Existing UI

Page Integration

The Flow Class Editor is a separate page in the workbench, controlled by a feature toggle:

// src/components/settings/FeatureSwitchesSection.tsx
// Add to existing feature switches:
<HStack justify="space-between" align="center">
  <VStack gap={1} align="start">
    <Text fontWeight="medium">Flow Class Editor</Text>
    <HStack gap={2} align="center">
      <Text fontSize="sm" color="fg.muted">
        Enable the visual flow class editor for creating and modifying dataflow patterns
      </Text>
      <Tag.Root colorPalette="accent" size="sm">
        <Tag.Label>experimental</Tag.Label>
      </Tag.Root>
    </HStack>
  </VStack>
  <Switch.Root
    checked={flowClassEditor}
    onCheckedChange={(details) =>
      onFlowClassEditorChange(details.checked)
    }
  >
    <Switch.HiddenInput />
    <Switch.Control>
      <Switch.Thumb />
    </Switch.Control>
  </Switch.Root>
</HStack>

Sidebar Navigation

// src/components/Sidebar.tsx
// Add conditional menu item based on feature switch:
{settings.featureSwitches.flowClassEditor && (
  <SidebarItem
    icon={<GitBranch />}
    label="Flow Class Editor"
    path="/flow-class-editor"
    isActive={location.pathname === "/flow-class-editor"}
  />
)}

Route Configuration

// src/App.tsx
// Add route for the editor page:
{settings.featureSwitches.flowClassEditor && (
  <Route path="/flow-class-editor" element={<FlowClassEditorPage />} />
)}

Page Component

// src/pages/FlowClassEditorPage.tsx
import React from "react";
import PageHeader from "../components/common/PageHeader";
import FlowClassEditor from "../components/flow-editor/FlowClassEditor";
import { GitBranch } from "lucide-react";

const FlowClassEditorPage: React.FC = () => {
  return (
    <>
      <PageHeader
        icon={<GitBranch />}
        title="Flow Class Editor"
        description="Visual editor for creating and modifying TrustGraph dataflow patterns"
      />
      <FlowClassEditor />
    </>
  );
};

export default FlowClassEditorPage;

Settings State Update

// src/state/settings.ts
interface FeatureSwitches {
  ontologyEditor: boolean;
  submissions: boolean;
  agentTools: boolean;
  mcpTools: boolean;
  schemas: boolean;
  tokenCost: boolean;
  flowClasses: boolean;        // Existing flow classes management
  flowClassEditor: boolean;     // New visual editor
  structuredQuery: boolean;
}

Integration Features

• Uses consistent Chakra UI theming
• Integrates with notification system via useNotification
• Progress indicators via useActivity
• Follows existing Config API patterns
• Respects user's feature toggle preferences

Responsive Design

Desktop (Primary)

• Full editor with all panels visible
• Optimal canvas size for complex flows
• Properties panel as sidebar

Tablet

• Collapsible panels to maximize canvas
• Touch-friendly node manipulation
• Simplified toolbar

Mobile (View-only)

• Read-only flow visualization
• Pan and zoom only
• Export functionality retained

Performance Considerations

Optimizations

• Virtualization for large flows (100+ nodes)
• Debounced validation during editing
• Memoized node/edge components
• Lazy loading of processor templates
• Web Workers for layout calculations

Limits

• Max 500 nodes per flow class
• Max 1000 edges per flow class
• Auto-layout for flows under 100 nodes
• Real-time validation for flows under 50 nodes

Error Handling

Validation Errors

• Inline error indicators on invalid nodes/edges
• Validation panel with detailed error list
• Prevent export/save when errors exist

Runtime Errors

• Connection rejection with toast notification
• Import failure with detailed parsing errors
• Auto-save recovery for browser crashes

Future Enhancements

Phase 2

• Processor library management - Add custom processors
• Collaborative editing - Real-time multi-user support
• Version control - Flow class versioning and diff view
• Simulation mode - Visualize data flow through the graph

Phase 3

• AI assistance - Suggest connections and optimizations
• Performance profiling - Visualize bottlenecks
• Template marketplace - Share flow patterns
• Code generation - Generate processor stubs from flow

Testing Strategy

Unit Tests

• Node/edge component rendering
• Validation logic
• Import/export transformations
• State management actions

Integration Tests

• Full editor workflow
• Save/load operations
• Template instantiation
• Keyboard shortcuts

E2E Tests

• Create flow from scratch
• Import and modify existing flow
• Export and validate JSON
• Deploy flow instance