sam-api/scripts/validation/README.md

Parametric BOM Validation Scripts

This directory contains standalone validation scripts for comprehensive testing of the parametric BOM system.

Scripts Overview

1. validate_bom_system.php

Purpose: Complete system validation across all components

Features:

• Database connectivity testing
• Model operations validation
• Parameter validation testing
• Formula calculation testing
• Condition rule evaluation testing
• BOM resolution testing
• Performance benchmarking
• Error handling validation

Usage:

php scripts/validation/validate_bom_system.php

Exit Codes:

• 0 - All tests passed (≥90% success rate)
• 1 - Some issues found (75-89% success rate)
• 2 - Critical issues found (<75% success rate)

2. test_kss01_scenarios.php

Purpose: Business scenario testing for KSS01 model

Features:

• Residential scenario testing
• Commercial scenario testing
• Edge case scenario testing
• Material type validation
• Installation type validation
• Performance scenario testing

Usage:

php scripts/validation/test_kss01_scenarios.php

Test Scenarios:

• Small bedroom window (800x600)
• Standard patio door (1800x2100)
• Large living room window (2400x1500)
• Restaurant storefront (3000x2500)
• Office building entrance (2200x2400)
• Warehouse opening (4000x3000)
• Minimum size opening (600x400)
• Maximum size opening (3000x2500)

Exit Codes:

• 0 - All scenarios passed (≥95% success rate)
• 1 - Some edge cases failed (85-94% success rate)
• 2 - Critical business logic issues (<85% success rate)

3. performance_test.php

Purpose: Performance and scalability testing

Features:

• Single resolution performance testing
• Batch resolution performance testing
• Memory usage analysis
• Database query efficiency testing
• Parameter variation performance testing
• Concurrent resolution simulation
• Large dataset throughput testing

Usage:

php scripts/validation/performance_test.php

Performance Thresholds:

• Single resolution: <200ms
• Batch 10 resolutions: <1.5s
• Batch 100 resolutions: <12s
• Memory usage: <50MB
• DB queries per resolution: <20
• Concurrent resolution: <500ms avg
• Throughput: ≥10 resolutions/second

Exit Codes:

• 0 - Performance requirements met (≥90% tests passed)
• 1 - Performance issues detected (70-89% tests passed)
• 2 - Critical performance issues (<70% tests passed)

Prerequisites

1. Test Data Setup

Run the KSS01ModelSeeder to create required test data:

php artisan db:seed --class=KSS01ModelSeeder

This creates:

• KSS_DEMO tenant
• demo@kss01.com user (password: kss01demo)
• KSS01 model with parameters, formulas, and rules
• Test materials and products

2. Environment Configuration

Ensure your .env file has proper database configuration:

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=sam_api
DB_USERNAME=your_username
DB_PASSWORD=your_password

3. API Configuration

Ensure API keys are properly configured for testing.

Running All Validations

Run all validation scripts in sequence:

#!/bin/bash
echo "Starting comprehensive BOM system validation..."

echo "\n=== 1. System Validation ==="
php scripts/validation/validate_bom_system.php
SYSTEM_EXIT=$?

echo "\n=== 2. KSS01 Scenarios ==="
php scripts/validation/test_kss01_scenarios.php
SCENARIOS_EXIT=$?

echo "\n=== 3. Performance Testing ==="
php scripts/validation/performance_test.php
PERFORMANCE_EXIT=$?

echo "\n=== FINAL RESULTS ==="
echo "System Validation: $([ $SYSTEM_EXIT -eq 0 ] && echo "PASS" || echo "FAIL")"
echo "KSS01 Scenarios: $([ $SCENARIOS_EXIT -eq 0 ] && echo "PASS" || echo "FAIL")"
echo "Performance Tests: $([ $PERFORMANCE_EXIT -eq 0 ] && echo "PASS" || echo "FAIL")"

# Overall result
if [ $SYSTEM_EXIT -eq 0 ] && [ $SCENARIOS_EXIT -eq 0 ] && [ $PERFORMANCE_EXIT -eq 0 ]; then
    echo "\n🎉 ALL VALIDATIONS PASSED - System ready for production"
    exit 0
else
    echo "\n❌ SOME VALIDATIONS FAILED - Review required"
    exit 1
fi

Output Examples

Successful Validation

=== Parametric BOM System Validation ===
Starting comprehensive system validation...

🔍 Testing Database Connectivity...
   ✅ Database connection
   ✅ Tenant table access
   ✅ Design models table access

🔍 Testing Model Operations...
   ✅ KSS01 model exists
   ✅ Model has parameters
   ✅ Model has formulas
   ✅ Model has condition rules

🔍 Testing Parameter Validation...
   ✅ Valid parameters accepted
   ✅ Parameter range validation
   ✅ Required parameter validation

🔍 Testing Formula Calculations...
   ✅ Basic formula calculations
   ✅ Formula dependency resolution

🔍 Testing Condition Rule Evaluation...
   ✅ Basic rule evaluation
   ✅ Rule action generation
   ✅ Material type rule evaluation

🔍 Testing BOM Resolution...
   ✅ Complete BOM resolution
   ✅ BOM items have required fields
   ✅ BOM preview functionality
   ✅ BOM comparison functionality

🔍 Testing Performance Benchmarks...
   ✅ Single BOM resolution performance (<500ms)
     Duration: 156ms
   ✅ Multiple BOM resolutions performance (10 iterations <2s)
     Duration: 892ms

============================================================
VALIDATION REPORT
============================================================
Total Tests: 18
Passed: 18
Failed: 0
Success Rate: 100.0%

🎉 VALIDATION PASSED - System is ready for production

Failed Validation

❌ Single BOM resolution performance (<500ms)
     Duration: 650ms
❌ BOM comparison functionality
     Error: Undefined index: bom_diff

============================================================
VALIDATION REPORT
============================================================
Total Tests: 18
Passed: 15
Failed: 3
Success Rate: 83.3%

❌ FAILED TESTS:
   • Single BOM resolution performance (<500ms)
   • BOM comparison functionality - Undefined index: bom_diff
   • Multiple BOM resolutions performance (10 iterations <2s)

⚠️ VALIDATION WARNING - Some issues found, review required

Customization

Adding New Test Scenarios

To add new test scenarios to test_kss01_scenarios.php:

// Add to testResidentialScenarios() method
$this->testScenario('Custom Scenario Name', [
    'W0' => 1500,
    'H0' => 1000,
    'screen_type' => 'FABRIC',
    'install_type' => 'WALL'
], [
    'expectMotor' => 'MOTOR_KSS01_STD',
    'expectBrackets' => 2,
    'expectMaterial' => 'FABRIC_KSS01'
]);

Adjusting Performance Thresholds

Modify the THRESHOLDS constant in performance_test.php:

private const THRESHOLDS = [
    'single_resolution_ms' => 300,      // Increase from 200ms
    'batch_10_resolution_ms' => 2000,   // Increase from 1500ms
    'memory_usage_mb' => 75,            // Increase from 50MB
    // ... other thresholds
];

Adding Custom Validation Tests

Add new test methods to validate_bom_system.php:

private function testCustomValidation(): void
{
    $this->output("\n🔍 Testing Custom Validation...");

    $this->test('Custom test name', function() {
        // Your test logic here
        return true; // or false
    });
}

Integration with CI/CD

GitHub Actions Example

name: BOM System Validation

on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Setup PHP
      uses: shivammathur/setup-php@v2
      with:
        php-version: 8.2
        extensions: pdo, mysql

    - name: Install Dependencies
      run: composer install

    - name: Setup Database
      run: |
        php artisan migrate
        php artisan db:seed --class=KSS01ModelSeeder

    - name: Run System Validation
      run: php scripts/validation/validate_bom_system.php

    - name: Run Scenario Tests
      run: php scripts/validation/test_kss01_scenarios.php

    - name: Run Performance Tests
      run: php scripts/validation/performance_test.php

Docker Integration

# Add to Dockerfile for validation image
COPY scripts/validation /app/scripts/validation
RUN chmod +x /app/scripts/validation/*.php

# Validation command
CMD ["php", "scripts/validation/validate_bom_system.php"]

Monitoring and Alerts

Log Analysis

All validation scripts log to Laravel's logging system. Monitor logs for:

grep "BOM Validation" storage/logs/laravel.log
grep "KSS01 Scenarios" storage/logs/laravel.log
grep "BOM Performance" storage/logs/laravel.log

Automated Monitoring

Set up automated monitoring to run validations periodically:

# Crontab entry for daily validation
0 2 * * * cd /path/to/project && php scripts/validation/validate_bom_system.php >> /var/log/bom_validation.log 2>&1

Alerting on Failures

#!/bin/bash
# validation_monitor.sh
php scripts/validation/validate_bom_system.php
if [ $? -ne 0 ]; then
    curl -X POST -H 'Content-type: application/json' \
        --data '{"text":"BOM System Validation Failed!"}' \
        YOUR_SLACK_WEBHOOK_URL
fi

Troubleshooting

Common Issues

1. "KSS_DEMO tenant not found"

   php artisan db:seed --class=KSS01ModelSeeder
   

2. Memory limit exceeded

   php -d memory_limit=512M scripts/validation/performance_test.php
   

3. Database connection errors

   • Check database credentials in .env
   • Verify database server is running
   • Check network connectivity

4. Performance test failures

   • Check system load and available resources
   • Verify database indexes are created
   • Review query optimization

Debug Mode

Enable debug output in validation scripts:

// Add to script top
define('DEBUG_MODE', true);

// Use in test methods
if (defined('DEBUG_MODE') && DEBUG_MODE) {
    $this->output("Debug: " . json_encode($result));
}

Contributing

When adding new validation scripts:

1. Follow the existing error handling patterns
2. Use consistent exit codes
3. Provide clear progress output
4. Include performance considerations
5. Add comprehensive documentation
6. Test with various data scenarios