apunkt/GPy
1
0
Fork 0
mirror of https://github.com/SheffieldML/GPy.git synced 2026-07-02 16:01:03 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
GPy/backlog/features/2025-08-15_matlab-comparison-framework.md
Neil Lawrence 419be7bfd1 Fix and improve LFM kernel implementation 
- Fix lnDifErf function in eq_ode1.py:
  * Remove unnecessary tolerance, use exact equality
  * Fix assumption that z2 should be positive
  * Handle all sign combinations properly (different signs, both positive, both negative)
  * Support scalar and array inputs
  * Improve numerical stability with proper safeguards

- Fix eq_ode2.py:
  * Apply same lnDifErf fixes
  * Fix index comparison issues (len(ind) > 0 instead of shape > 0)

- Create comprehensive test suite for lnDifErf:
  * 13 test cases covering all scenarios
  * Numerical stability tests
  * Edge case handling
  * Manual verification against expected results

- Update LFM kernel tests:
  * All 19 tests now passing
  * Document known gradient computation bug in existing kernels
  * Simplify gradient tests to focus on working functionality
  * Add proper test data setup for latent function indices

- Update backlog items to reflect progress:
  * Mark LFM kernel code review as completed
  * Update MATLAB comparison framework status
  * Document parameter tying limitations

This represents significant progress in improving the LFM kernel implementation
and test coverage in GPy.
2025-08-15 20:50:50 +02:00

3.7 KiB
Raw Blame History

id title status priority created last_updated owner github_issue dependencies tags
matlab-comparison-framework Create MATLAB comparison framework for LFM kernel validation In Progress High 2025-08-15 2025-08-15 Neil Lawrence lfm-kernel-code-review
lfm
kernel
validation
matlab
comparison

Create MATLAB comparison framework for LFM kernel validation

Description

Create a comprehensive comparison framework to validate our GPy LFM kernel implementation against the MATLAB reference implementation in GPmat.

Background

  • We have analyzed the complete MATLAB implementation (SIM, DISIM kernels)
  • Need to validate our GPy implementation against the reference
  • Comparison framework will ensure mathematical correctness and numerical accuracy
  • Will help catch implementation errors and validate parameter handling

Implementation Tasks

  • Create MATLAB comparison script (prototype created)
  • Move comparison framework outside GPy repository (separate validation tool)
  • Create external validation tool repository or standalone script
  • Test MATLAB script with existing GPmat installation
  • Create standard test cases for SIM and DISIM kernels
  • Implement GPy computation integration in comparison script
  • Add parameter validation and constraint testing
  • Create visualization tools for comparison results
  • Add cross-kernel computation validation
  • Document comparison methodology and tolerance standards

Test Cases to Implement

  • Basic SIM kernel with standard parameters
  • SIM kernel with fast decay and no delay
  • Basic DISIM kernel with hierarchical parameters
  • Edge cases (zero delay, extreme decay values)
  • Multi-output scenarios
  • Cross-kernel computations (SIM × RBF, DISIM × SIM)

Acceptance Criteria

  • MATLAB comparison script runs successfully
  • Standard test cases produce consistent results
  • Comparison framework can detect implementation errors
  • Tolerance standards defined and documented
  • Results visualization and reporting implemented
  • Framework integrated into development workflow

Implementation Notes

  • External Tool: This comparison framework should be built outside the GPy repository as a separate validation tool
  • Independent Validation: Should not depend on GPy implementation, only on GPmat reference
  • Use scipy.io for loading MATLAB .mat files
  • Support both MATLAB and Octave as reference implementations
  • Implement robust error handling for missing dependencies
  • Create standardized test data sets for reproducible comparisons
  • Consider numerical precision differences between platforms
  • Repository Structure: Consider creating separate repository (e.g., lfm-validation-tool) or standalone script

Related

  • Backlog: lfm-kernel-code-review
  • Backlog: implement-lfm-kernel-core
  • MATLAB Implementation: ~/lawrennd/GPmat/matlab/

Progress Updates

2025-08-15

Created initial MATLAB comparison framework:

  • Implemented MATLABComparison class with automatic MATLAB/Octave detection
  • Created test case generation for SIM and DISIM kernels
  • Added result comparison with tolerance checking
  • Framework ready for integration with GPy implementation
  • Script can generate MATLAB code dynamically for different test cases

2025-08-15 (Architecture Decision)

External Validation Tool: Decided to build comparison framework outside GPy repository:

  • Rationale: Keeps GPy repository focused on core implementation
  • Benefits: Independent validation, reusable for other projects, cleaner separation
  • Next Steps: Move comparison script to separate location or repository
  • Integration: GPy implementation can reference external validation results