sam-api/scripts/validation/README.md

# Parametric BOM Validation Scripts

This directory contains standalone validation scripts for comprehensive testing of the parametric BOM system.

## Scripts Overview

### 1. validate_bom_system.php

**Purpose**: Complete system validation across all components

**Features**:
- Database connectivity testing
- Model operations validation
- Parameter validation testing
- Formula calculation testing
- Condition rule evaluation testing
- BOM resolution testing
- Performance benchmarking
- Error handling validation

**Usage**:
```bash
php scripts/validation/validate_bom_system.php
```

**Exit Codes**:
- `0` - All tests passed (≥90% success rate)
- `1` - Some issues found (75-89% success rate)
- `2` - Critical issues found (<75% success rate)

### 2. test_kss01_scenarios.php

**Purpose**: Business scenario testing for KSS01 model

**Features**:
- Residential scenario testing
- Commercial scenario testing
- Edge case scenario testing
- Material type validation
- Installation type validation
- Performance scenario testing

**Usage**:
```bash
php scripts/validation/test_kss01_scenarios.php
```

**Test Scenarios**:
- Small bedroom window (800x600)
- Standard patio door (1800x2100)
- Large living room window (2400x1500)
- Restaurant storefront (3000x2500)
- Office building entrance (2200x2400)
- Warehouse opening (4000x3000)
- Minimum size opening (600x400)
- Maximum size opening (3000x2500)

**Exit Codes**:
- `0` - All scenarios passed (≥95% success rate)
- `1` - Some edge cases failed (85-94% success rate)
- `2` - Critical business logic issues (<85% success rate)

### 3. performance_test.php

**Purpose**: Performance and scalability testing

**Features**:
- Single resolution performance testing
- Batch resolution performance testing
- Memory usage analysis
- Database query efficiency testing
- Parameter variation performance testing
- Concurrent resolution simulation
- Large dataset throughput testing

**Usage**:
```bash
php scripts/validation/performance_test.php
```

**Performance Thresholds**:
- Single resolution: <200ms
- Batch 10 resolutions: <1.5s
- Batch 100 resolutions: <12s
- Memory usage: <50MB
- DB queries per resolution: <20
- Concurrent resolution: <500ms avg
- Throughput: ≥10 resolutions/second

**Exit Codes**:
- `0` - Performance requirements met (≥90% tests passed)
- `1` - Performance issues detected (70-89% tests passed)
- `2` - Critical performance issues (<70% tests passed)

## Prerequisites

### 1. Test Data Setup

Run the KSS01ModelSeeder to create required test data:

```bash
php artisan db:seed --class=KSS01ModelSeeder
```

This creates:
- KSS_DEMO tenant
- demo@kss01.com user (password: kss01demo)
- KSS01 model with parameters, formulas, and rules
- Test materials and products

### 2. Environment Configuration

Ensure your `.env` file has proper database configuration:

```env
DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=sam_api
DB_USERNAME=your_username
DB_PASSWORD=your_password
```

### 3. API Configuration

Ensure API keys are properly configured for testing.

## Running All Validations

Run all validation scripts in sequence:

```bash
#!/bin/bash
echo "Starting comprehensive BOM system validation..."

echo "\n=== 1. System Validation ==="
php scripts/validation/validate_bom_system.php
SYSTEM_EXIT=$?

echo "\n=== 2. KSS01 Scenarios ==="
php scripts/validation/test_kss01_scenarios.php
SCENARIOS_EXIT=$?

echo "\n=== 3. Performance Testing ==="
php scripts/validation/performance_test.php
PERFORMANCE_EXIT=$?

echo "\n=== FINAL RESULTS ==="
echo "System Validation: $([ $SYSTEM_EXIT -eq 0 ] && echo "PASS" || echo "FAIL")"
echo "KSS01 Scenarios: $([ $SCENARIOS_EXIT -eq 0 ] && echo "PASS" || echo "FAIL")"
echo "Performance Tests: $([ $PERFORMANCE_EXIT -eq 0 ] && echo "PASS" || echo "FAIL")"

# Overall result
if [ $SYSTEM_EXIT -eq 0 ] && [ $SCENARIOS_EXIT -eq 0 ] && [ $PERFORMANCE_EXIT -eq 0 ]; then
    echo "\n🎉 ALL VALIDATIONS PASSED - System ready for production"
    exit 0
else
    echo "\n❌ SOME VALIDATIONS FAILED - Review required"
    exit 1
fi
```

## Output Examples

### Successful Validation

```
=== Parametric BOM System Validation ===
Starting comprehensive system validation...

🔍 Testing Database Connectivity...
   ✅ Database connection
   ✅ Tenant table access
   ✅ Design models table access

🔍 Testing Model Operations...
   ✅ KSS01 model exists
   ✅ Model has parameters
   ✅ Model has formulas
   ✅ Model has condition rules

🔍 Testing Parameter Validation...
   ✅ Valid parameters accepted
   ✅ Parameter range validation
   ✅ Required parameter validation

🔍 Testing Formula Calculations...
   ✅ Basic formula calculations
   ✅ Formula dependency resolution

🔍 Testing Condition Rule Evaluation...
   ✅ Basic rule evaluation
   ✅ Rule action generation
   ✅ Material type rule evaluation

🔍 Testing BOM Resolution...
   ✅ Complete BOM resolution
   ✅ BOM items have required fields
   ✅ BOM preview functionality
   ✅ BOM comparison functionality

🔍 Testing Performance Benchmarks...
   ✅ Single BOM resolution performance (<500ms)
     Duration: 156ms
   ✅ Multiple BOM resolutions performance (10 iterations <2s)
     Duration: 892ms

============================================================
VALIDATION REPORT
============================================================
Total Tests: 18
Passed: 18
Failed: 0
Success Rate: 100.0%

🎉 VALIDATION PASSED - System is ready for production
```

### Failed Validation

```
❌ Single BOM resolution performance (<500ms)
     Duration: 650ms
❌ BOM comparison functionality
     Error: Undefined index: bom_diff

============================================================
VALIDATION REPORT
============================================================
Total Tests: 18
Passed: 15
Failed: 3
Success Rate: 83.3%

❌ FAILED TESTS:
   • Single BOM resolution performance (<500ms)
   • BOM comparison functionality - Undefined index: bom_diff
   • Multiple BOM resolutions performance (10 iterations <2s)

⚠️ VALIDATION WARNING - Some issues found, review required
```

## Customization

### Adding New Test Scenarios

To add new test scenarios to `test_kss01_scenarios.php`:

```php
// Add to testResidentialScenarios() method
$this->testScenario('Custom Scenario Name', [
    'W0' => 1500,
    'H0' => 1000,
    'screen_type' => 'FABRIC',
    'install_type' => 'WALL'
], [
    'expectMotor' => 'MOTOR_KSS01_STD',
    'expectBrackets' => 2,
    'expectMaterial' => 'FABRIC_KSS01'
]);
```

### Adjusting Performance Thresholds

Modify the `THRESHOLDS` constant in `performance_test.php`:

```php
private const THRESHOLDS = [
    'single_resolution_ms' => 300,      // Increase from 200ms
    'batch_10_resolution_ms' => 2000,   // Increase from 1500ms
    'memory_usage_mb' => 75,            // Increase from 50MB
    // ... other thresholds
];
```

### Adding Custom Validation Tests

Add new test methods to `validate_bom_system.php`:

```php
private function testCustomValidation(): void
{
    $this->output("\n🔍 Testing Custom Validation...");

    $this->test('Custom test name', function() {
        // Your test logic here
        return true; // or false
    });
}
```

## Integration with CI/CD

### GitHub Actions Example

```yaml
name: BOM System Validation

on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Setup PHP
      uses: shivammathur/setup-php@v2
      with:
        php-version: 8.2
        extensions: pdo, mysql

    - name: Install Dependencies
      run: composer install

    - name: Setup Database
      run: |
        php artisan migrate
        php artisan db:seed --class=KSS01ModelSeeder

    - name: Run System Validation
      run: php scripts/validation/validate_bom_system.php

    - name: Run Scenario Tests
      run: php scripts/validation/test_kss01_scenarios.php

    - name: Run Performance Tests
      run: php scripts/validation/performance_test.php
```

### Docker Integration

```dockerfile
# Add to Dockerfile for validation image
COPY scripts/validation /app/scripts/validation
RUN chmod +x /app/scripts/validation/*.php

# Validation command
CMD ["php", "scripts/validation/validate_bom_system.php"]
```

## Monitoring and Alerts

### Log Analysis

All validation scripts log to Laravel's logging system. Monitor logs for:

```bash
grep "BOM Validation" storage/logs/laravel.log
grep "KSS01 Scenarios" storage/logs/laravel.log
grep "BOM Performance" storage/logs/laravel.log
```

### Automated Monitoring

Set up automated monitoring to run validations periodically:

```bash
# Crontab entry for daily validation
0 2 * * * cd /path/to/project && php scripts/validation/validate_bom_system.php >> /var/log/bom_validation.log 2>&1
```

### Alerting on Failures

```bash
#!/bin/bash
# validation_monitor.sh
php scripts/validation/validate_bom_system.php
if [ $? -ne 0 ]; then
    curl -X POST -H 'Content-type: application/json' \
        --data '{"text":"BOM System Validation Failed!"}' \
        YOUR_SLACK_WEBHOOK_URL
fi
```

## Troubleshooting

### Common Issues

1. **"KSS_DEMO tenant not found"**
   ```bash
   php artisan db:seed --class=KSS01ModelSeeder
   ```

2. **Memory limit exceeded**
   ```bash
   php -d memory_limit=512M scripts/validation/performance_test.php
   ```

3. **Database connection errors**
   - Check database credentials in `.env`
   - Verify database server is running
   - Check network connectivity

4. **Performance test failures**
   - Check system load and available resources
   - Verify database indexes are created
   - Review query optimization

### Debug Mode

Enable debug output in validation scripts:

```php
// Add to script top
define('DEBUG_MODE', true);

// Use in test methods
if (defined('DEBUG_MODE') && DEBUG_MODE) {
    $this->output("Debug: " . json_encode($result));
}
```

## Contributing

When adding new validation scripts:

1. Follow the existing error handling patterns
2. Use consistent exit codes
3. Provide clear progress output
4. Include performance considerations
5. Add comprehensive documentation
6. Test with various data scenarios