Skip to content

REMS Testing Documentation

Overview

This document explains how REMS (Raizcorp CRM) testing works in development environments, including the test database setup, service implementations, and data seeding process.

Architecture

Production vs Testing

In production environments, the system connects to the live REMS database via the rems connection. However, in development and testing environments, the same rems connection name is used but points to a local testing database (configured via environment variables), and the system uses test service implementations that provide mock data without requiring access to the production REMS system.

Service Layer Pattern

The system uses a service interface pattern to switch between production and testing implementations. The service binding is determined by the APP_ENV environment variable:

// AppServiceProvider.php
$this->app->bind(RemsProgrammeServiceInterface::class, function ($app) use ($env) {
    return match ($env) {
        'staging', 'prod', 'production' => $app->make(RemsProgrammeService::class),
        default => $app->make(RemsProgrammeServiceMock::class),
    };
});

$this->app->bind(EnrolmentListServiceInterface::class, function ($app) use ($env) {
    return match ($env) {
        'staging', 'prod', 'production' => $app->make(EnrolmentListService::class),
        default => $app->make(EnrolmentListServiceTest::class),
    };
});

Service Selection Logic:

  • Production/Staging: Uses RemsProgrammeService (connects to live REMS database)
  • Development/Testing: Uses RemsProgrammeServiceMock (uses local test database)

Database Connections

Production REMS Connection

// config/database.php
'rems' => [
    'driver' => env('REMS_DB_DRIVER', 'sqlsrv'),
    'host' => env('REMS_DB_HOST', '127.0.0.1'),
    'port' => env('REMS_DB_PORT', '1433'),
    'database' => env('REMS_DB_DATABASE', 'rems'),
    'username' => env('REMS_DB_USERNAME', 'forge'),
    'password' => env('REMS_DB_PASSWORD', ''),
    'charset' => 'utf8',
    'prefix' => '',
    'prefix_indexes' => true,
    'strict' => true,
    'engine' => null,
],

Testing REMS Connection

The testing connection uses the same configuration but typically points to a local or development database. The system automatically switches between production and testing services based on the environment.

Environment Variables

📋 Quick Reference: See Environment Variables Reference for a complete deployment checklist.

Required Environment Variables

The following environment variables must be configured for REMS functionality:

  • REMS_DB_DRIVER - Database driver (mysql for testing, sqlsrv for production)
  • REMS_DB_HOST - Database server hostname
  • REMS_DB_PORT - Database server port (3306 for MySQL, 1433 for SQL Server)
  • REMS_DB_DATABASE - Database name
  • REMS_DB_USERNAME - Database username
  • REMS_DB_PASSWORD - Database password
  • APP_ENV - Application environment (determines service binding)

Service Binding

The APP_ENV variable determines which REMS service implementation is used:

  • local or testing: Uses RemsProgrammeServiceMock (test database)
  • production or staging: Uses RemsProgrammeService (live REMS database)

Parallel Testing Configuration

For parallel test execution, the system automatically appends test tokens to database names:

# Base configuration
REMS_DB_DATABASE=rems_test

# Automatically becomes during parallel testing:
# rems_test_test_1, rems_test_test_2, etc.

Test Services

RemsProgrammeServiceMock

Located at app/Services/RemsProgrammeServiceMock.php, this service provides mock programme data for testing by extending BaseRemsProgrammeService:

final class RemsProgrammeServiceMock extends BaseRemsProgrammeService implements RemsProgrammeServiceInterface
{
    public function getRemsProgrammes(): Collection|JsonResponse
    {
        try {
            $queryResult = DB::connection('rems')
                ->table('rems_programmes')
                ->select([
                    'rems_product_id',
                    'rems_programme_year',
                    'name',
                ])
                ->distinct()
                ->get();
        } catch (QueryException $e) {
            logger()->error('REMS (testing) connection error: ' . $e->getMessage());
            return response()->json([
                'error' => 'An error occurred while trying to connect to the REMS testing database.',
            ], 503);
        }

        // Filter out already linked programmes using base class method
        return $this->filterRemsProgrammesEnvoy($queryResult);
    }
}

Key Features:

  • Extends BaseRemsProgrammeService to inherit linkRemsProgrammes() logic
  • Filters out already linked programmes to prevent duplicates
  • Provides incremental IDs for admin interface compatibility
  • Graceful error handling with JSON responses
  • Mirrors production service interface
  • Uses local test database via rems connection

EnrolmentListServiceTest

Located at app/Services/EnrolmentListServiceTest.php, this service provides mock enrolment data:

// This is pseudocode:
public function getRemsEnrolments() {
    // Cache for 7 days (same as production)
    return Cache.remember('rems-enrolments', 7_days, () => {
        return RemsTestEnrolment.all()
    })
}

Key Features:

  • Caches data for 7 days (matching production behavior)
  • Uses RemsTestEnrolment model for data access
  • Comprehensive error handling and logging
  • Returns appropriate HTTP status codes

Test Data Models

RemsTestEnrolment

Located at app/Models/RemsTestEnrolment.php, this model represents test enrolment data:

// This is pseudocode:
class RemsTestEnrolment {
    connection = 'rems'
    table = 'rems_test_enrolments'

    fillable = [
        'foreign_uid', 'name', 'email', 'company_name',
        'rems_programme', 'status'
        // ... other fields
    ]
}

Key Features:

  • Uses the rems database connection
  • Includes all fields needed for enrolment processing
  • Supports mass assignment for seeding

Data Seeding

RemsTestDataSeeder

Located at database/seeders/RemsTestDataSeeder.php, this seeder populates the testing database with realistic data:

// This is pseudocode:
class RemsTestDataSeeder {
    run() {
        // Clear existing data
        truncate('rems_test_enrolments')
        truncate('rems_programmes')

        // Create test programmes
        programmes = [
            {id: 1001, year: 'C1', name: 'C1 : Flowcode Trial'},
            {id: 1002, year: 'C2', name: 'C2 : Flowcode Trial'}
        ]

        // Create test enrolments
        enrolments = [
            {uid: '1001', name: 'WonderWho Testerson',
             company: 'Opal Motor', programme: 'C1 : Flowcode Trial'}
        ]

        insert(programmes)
        insert(enrolments)
    }
}

Key Features:

  • Clears existing test data before seeding
  • Creates realistic programme catalogue
  • Generates diverse test enrolments
  • Uses proper database connections
  • Provides progress feedback

Database Migrations

REMS Testing Tables

The testing database includes several tables that mirror production REMS structure:

rems_programmes

  • rems_product_id: Unique product identifier
  • rems_programme_year: Programme year (e.g., C1, C2, C3)
  • name: Full programme name

rems_test_enrolments

  • foreign_uid: External system identifier
  • first_name, last_name, name: Entrepreneur details
  • email, phone: Contact information
  • company_name, company_status: Company details
  • delivery_location: Physical location
  • rems_programme: Full programme name
  • rems_product_id, rems_programme_year: Programme identifiers
  • start_date_at, ignition_date_at: Programme dates
  • status: Enrolment status

Usage in Development

Running the Seeder

# Seed test data
php artisan db:seed --class=RemsTestDataSeeder

# Or run all seeders
php artisan db:seed

Testing the Connection

# Test REMS connection via Tinker
php artisan tinker

# Test connection
DB::connection('rems')->getPdo();

# Query test data
DB::connection('rems')->table('rems_programmes')->get();

Switching Between Environments

The system automatically switches between production and testing services based on the environment:

  • Production/Staging: Uses RemsProgrammeService and EnrolmentListService
  • Development/Testing: Uses RemsProgrammeServiceMock and EnrolmentListServiceMock

Integration Points

Filament Admin Interface

The testing services integrate with Filament admin panels:

// ProgrammeResource.php
->getSearchResultsUsing(function (string $search) {
    $service = app(RemsProgrammeServiceInterface::class);
    return $service->getRemsProgrammes();
})

API Endpoints

Testing services provide the same interface as production services, ensuring API compatibility:

interface RemsProgrammeServiceInterface {
    getRemsProgrammes(): Collection|JsonResponse  // Both services implement this
}

Best Practices

Data Consistency

  • Always clear existing test data before seeding
  • Use realistic but clearly identifiable test data
  • Maintain referential integrity between related tables

Error Handling

  • Implement comprehensive error handling in test services
  • Log errors for debugging
  • Return appropriate HTTP status codes

Performance

  • Use caching where appropriate (matching production behavior)
  • Limit result sets to reasonable sizes
  • Implement proper database indexing

Security

  • Never include production data in test databases
  • Use separate database connections for testing
  • Implement proper access controls

Troubleshooting

Common Issues

  1. Connection Errors: Verify database credentials and connection settings
  2. Missing Tables: Ensure migrations have been run for the testing database
  3. Service Binding Issues: Check AppServiceProvider configuration
  4. Data Not Appearing: Verify seeder execution and data integrity
  5. Environment Variable Issues: Ensure all required REMS environment variables are set
  6. Parallel Testing Issues: Check that test databases are properly isolated
  7. Date Parsing Errors: Verify date format compatibility in test data

Debug Commands

# Check database connections
php artisan tinker
DB::connection('rems')->getDatabaseName();

# Verify test data
DB::connection('rems')->table('rems_programmes')->count();
DB::connection('rems')->table('rems_test_enrolments')->count();

# Test service binding
app(RemsProgrammeServiceInterface::class)->getRemsProgrammes();

# Check environment variables
php artisan tinker
config('database.connections.rems');

# Test date parsing
php artisan tinker
$service = app(\App\Services\EnrolmentCreationService::class);
$reflection = new ReflectionClass($service);
$method = $reflection->getMethod('parseDate');
$method->setAccessible(true);
$method->invoke($service, '25/12/2024');

Environment Variable Checklist

📋 Complete Setup Guide: See Environment Variables Reference for detailed configuration examples and deployment checklist.

Required for All Environments

  • REMS_DB_DRIVER
  • REMS_DB_HOST
  • REMS_DB_PORT
  • REMS_DB_DATABASE
  • REMS_DB_USERNAME
  • REMS_DB_PASSWORD

Environment-Specific

  • .env: APP_ENV=local
  • .env.testing: APP_ENV=testing
  • .env.production: APP_ENV=production

Recent Improvements

Enhanced Test Infrastructure

Parallel Testing Support

The test suite now includes full parallel testing support for REMS connections:

// TestCase.php - Parallel database configuration
$baseRemsDatabase = str_replace('_testing', '', env('REMS_DB_DATABASE', 'rems'));
$remsDatabase = $baseRemsDatabase.'_test_'.$token;

config(['database.connections.rems.database' => $remsDatabase]);
DB::purge('rems');
DB::reconnect('rems');

Transaction Management

  • Automatic transaction wrapping in base TestCase
  • Proper rollback handling for all database connections
  • Eliminated manual transaction management in individual tests
  • Race condition protection with proper locking mechanisms

Date Parsing Enhancements

The EnrolmentCreationService now supports multiple date formats:

// Supports both formats gracefully
'25/12/2024'  // d/m/Y format (from REMS)
'2024-12-25'  // Y-m-d format (normalized)

Error Handling Improvements

  • Graceful fallback for unparseable dates
  • No exceptions thrown for invalid date formats
  • Comprehensive logging for debugging
  • Race condition handling in concurrent operations

Testing Best Practices

Service Testing

  • Extend base classes instead of mocking interfaces directly
  • Test actual business logic rather than empty mock methods
  • Verify error handling paths in base service classes
  • Use proper inheritance to test real code paths

Database Testing

  • Use base TestCase transactions for automatic cleanup
  • Avoid manual transaction management in individual tests
  • Leverage parallel testing for better performance
  • Test with realistic data using proper seeders

Future Enhancements

Planned Improvements

  • Dynamic Data Generation: Use factories for more varied test data
  • Performance Testing: Add load testing capabilities
  • Data Validation: Implement schema validation for test data
  • Automated Testing: Add integration tests for REMS services
  • Date Format Detection: Automatic format detection for date parsing

Monitoring

  • Connection Health Checks: Regular database connectivity tests
  • Data Quality Metrics: Validation of test data integrity
  • Performance Metrics: Response time monitoring for test services
  • Error Rate Tracking: Monitor date parsing and service failures

Conclusion

The REMS testing system provides a robust foundation for development and testing without requiring access to production systems. By using service interfaces and mock data, developers can work efficiently while maintaining data consistency and security.

For additional information, refer to:

  • ../engauge.test/docs/dev/general/rems.md (internal ops reference)
  • config/database.php (database configuration in the app repo)
  • app/Services/ (service implementations in the app repo)