mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-04-25 00:16:23 +02:00
Flow configurable parameters (#532)
* Fix pyproject.toml missing requests dep * parameters is now parameter-types * Update flow parameters tech spec for recent changes (no impact on this repo)
This commit is contained in:
parent
7a3bfad826
commit
aa8e422e8c
2 changed files with 48 additions and 23 deletions
|
|
@ -80,8 +80,10 @@ The configurable parameters system requires the following technical components:
|
||||||
Module: trustgraph-flow/trustgraph/flow/launcher.py
|
Module: trustgraph-flow/trustgraph/flow/launcher.py
|
||||||
|
|
||||||
5. **UI Parameter Forms**
|
5. **UI Parameter Forms**
|
||||||
- Dynamic form generation from parameter schemas
|
- Dynamic form generation from flow parameter metadata
|
||||||
- Input validation and error display
|
- Ordered parameter display using `order` field
|
||||||
|
- Descriptive parameter labels using `description` field
|
||||||
|
- Input validation against parameter type definitions
|
||||||
- Parameter presets and templates
|
- Parameter presets and templates
|
||||||
|
|
||||||
Module: trustgraph-ui/components/flow-parameters/
|
Module: trustgraph-ui/components/flow-parameters/
|
||||||
|
|
@ -90,7 +92,7 @@ The configurable parameters system requires the following technical components:
|
||||||
|
|
||||||
#### Parameter Definitions (Stored in Schema/Config)
|
#### Parameter Definitions (Stored in Schema/Config)
|
||||||
|
|
||||||
Parameter definitions are stored centrally in the schema and config system with type "parameters":
|
Parameter definitions are stored centrally in the schema and config system with type "parameter-types":
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
|
|
@ -129,16 +131,32 @@ Parameter definitions are stored centrally in the schema and config system with
|
||||||
|
|
||||||
#### Flow Class with Parameter References
|
#### Flow Class with Parameter References
|
||||||
|
|
||||||
Flow classes reference parameter definitions by name:
|
Flow classes define parameter metadata with type references, descriptions, and ordering:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"flow_class": "document-analysis",
|
"flow_class": "document-analysis",
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"model": "llm-model",
|
"model": {
|
||||||
"size": "model-size",
|
"type": "llm-model",
|
||||||
"temp": "temperature",
|
"description": "LLM model to use for document analysis",
|
||||||
"chunk": "chunk-size"
|
"order": 1
|
||||||
|
},
|
||||||
|
"size": {
|
||||||
|
"type": "model-size",
|
||||||
|
"description": "Model size variant to use",
|
||||||
|
"order": 2
|
||||||
|
},
|
||||||
|
"temp": {
|
||||||
|
"type": "temperature",
|
||||||
|
"description": "Generation temperature for creativity control",
|
||||||
|
"order": 3
|
||||||
|
},
|
||||||
|
"chunk": {
|
||||||
|
"type": "chunk-size",
|
||||||
|
"description": "Document chunk size for processing",
|
||||||
|
"order": 4
|
||||||
|
}
|
||||||
},
|
},
|
||||||
"class": {
|
"class": {
|
||||||
"text-completion:{class}": {
|
"text-completion:{class}": {
|
||||||
|
|
@ -163,15 +181,18 @@ Flow classes reference parameter definitions by name:
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
The `parameters` section maps flow-specific parameter names (keys) to centrally-defined parameter definitions (values).
|
The `parameters` section maps flow-specific parameter names (keys) to parameter metadata objects containing:
|
||||||
```
|
- `type`: Reference to centrally-defined parameter definition (e.g., "llm-model")
|
||||||
|
- `description`: Human-readable description for UI display
|
||||||
|
- `order`: Display order for parameter forms (lower numbers appear first)
|
||||||
|
|
||||||
This approach allows:
|
This approach allows:
|
||||||
- Reusable parameter definitions across multiple flow classes
|
- Reusable parameter type definitions across multiple flow classes
|
||||||
- Centralized parameter management and updates
|
- Centralized parameter type management and validation
|
||||||
- Reduced duplication in flow class definitions
|
- Flow-specific parameter descriptions and ordering
|
||||||
|
- Enhanced UI experience with descriptive parameter forms
|
||||||
- Consistent parameter validation across flows
|
- Consistent parameter validation across flows
|
||||||
- Easy addition of new standard parameters
|
- Easy addition of new standard parameter types
|
||||||
|
|
||||||
#### Flow Launch Request
|
#### Flow Launch Request
|
||||||
|
|
||||||
|
|
@ -191,20 +212,23 @@ The flow launch API accepts parameters using the flow's parameter names:
|
||||||
```
|
```
|
||||||
|
|
||||||
The system will:
|
The system will:
|
||||||
1. Map flow parameter names to their definitions (e.g., `model` → `llm-model`)
|
1. Extract parameter metadata from flow class definition
|
||||||
2. Validate values against the parameter definitions
|
2. Map flow parameter names to their type definitions (e.g., `model` → `llm-model` type)
|
||||||
3. Substitute values into processor parameters during flow instantiation
|
3. Validate user-provided values against the parameter type definitions
|
||||||
|
4. Substitute validated values into processor parameters during flow instantiation
|
||||||
|
|
||||||
### Implementation Details
|
### Implementation Details
|
||||||
|
|
||||||
#### Parameter Resolution Process
|
#### Parameter Resolution Process
|
||||||
|
|
||||||
1. **Flow Class Loading**: Load flow class and extract parameter references
|
1. **Flow Class Loading**: Load flow class and extract parameter metadata
|
||||||
2. **Definition Lookup**: Retrieve parameter definitions from schema/config store
|
2. **Metadata Extraction**: Extract `type`, `description`, and `order` for each parameter
|
||||||
3. **Validation**: Validate user-provided parameters against definitions
|
3. **Type Definition Lookup**: Retrieve parameter type definitions from schema/config store using `type` field
|
||||||
4. **Default Application**: Apply default values for missing parameters
|
4. **UI Form Generation**: Use `description` and `order` fields to create ordered parameter forms
|
||||||
5. **Template Substitution**: Replace parameter placeholders in processor parameters
|
5. **Validation**: Validate user-provided parameters against type definitions
|
||||||
6. **Processor Instantiation**: Create processors with substituted parameters
|
6. **Default Application**: Apply default values for missing parameters from type definitions
|
||||||
|
7. **Template Substitution**: Replace parameter placeholders in processor parameters with validated values
|
||||||
|
8. **Processor Instantiation**: Create processors with substituted parameters
|
||||||
|
|
||||||
#### Pulsar Integration
|
#### Pulsar Integration
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,7 @@ requires-python = ">=3.8"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"pulsar-client",
|
"pulsar-client",
|
||||||
"prometheus-client",
|
"prometheus-client",
|
||||||
|
"requests",
|
||||||
]
|
]
|
||||||
classifiers = [
|
classifiers = [
|
||||||
"Programming Language :: Python :: 3",
|
"Programming Language :: Python :: 3",
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue