๐ŸŽฏ Patterns and Lessons

Complete library of reusable patterns and critical lessons distilled during migration of 50 modules + 5 advanced migrations. Practical knowledge to accelerate future implementations.

๐Ÿ”ง

Proven Patterns

35 PostgreSQL patterns validated in production

โš ๏ธ

Documented Errors

Common problems identified and solved

๐Ÿ’ก

Ready Solutions

Copy-paste code for future implementations

๐ŸŽฏ Reusable PostgreSQL Patterns

Collection of proven and optimized patterns developed during migration of 50 modules.

๐Ÿ” Base Validation Pattern

Used in 50 modules

Purpose: Standard validation for required fields, email, data cleaning and default values.

Pattern Code:

CREATE OR REPLACE FUNCTION llx_{tabla}_before_insert() 
RETURNS trigger 
LANGUAGE plpgsql
AS $$
BEGIN
    -- 1. Required fields validation
    IF NEW.required_field IS NULL OR trim(NEW.required_field) = '' THEN
        RAISE EXCEPTION 'ErrorFieldRequired: required_field';
    END IF;
    
    -- 2. Automatic data cleaning
    NEW.text_field = trim(NEW.text_field);
    
    -- 3. Email validation with regex
    IF NEW.email IS NOT NULL AND NEW.email != '' THEN
        NEW.email = trim(lower(NEW.email));
        IF NOT (NEW.email ~* '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$') THEN
            RAISE EXCEPTION 'ErrorBadEMail: %', NEW.email;
        END IF;
    END IF;
    
    -- 4. Phone formatting
    IF NEW.phone IS NOT NULL THEN
        NEW.phone = regexp_replace(NEW.phone, '[^0-9+\-\.\s()]', '', 'g');
        NEW.phone = trim(NEW.phone);
    END IF;
    
    -- 5. Default values
    NEW.entity := COALESCE(NEW.entity, 1);
    NEW.status := COALESCE(NEW.status, 1);
    NEW.datec := COALESCE(NEW.datec, NOW());
    
    -- 6. Numeric values validation
    IF NEW.numeric_field IS NOT NULL AND NEW.numeric_field < 0 THEN
        RAISE EXCEPTION 'Field cannot be negative';
    END IF;
    
    RETURN NEW;
END;
$$;

โœ… Benefits:

  • Guaranteed automatic validation
  • Always clean and normalized data
  • Consistent error messages
  • Facilitates debugging and maintenance

๐Ÿท๏ธ Code Generation Pattern

Used in 7 modules

Purpose: Automatic generation of unique and consecutive codes with customizable formats.

Implemented Variants:

๐Ÿ“… Year-Month-Sequence Format (PR2501-0001)
CREATE OR REPLACE FUNCTION llx_tabla_get_next_ref(p_entity integer DEFAULT 1)
RETURNS varchar AS $$
DECLARE
    v_current_year varchar(2);
    v_current_month varchar(2);
    v_current_num integer;
    v_new_ref varchar(30);
BEGIN
    -- Get current year and month
    v_current_year := to_char(CURRENT_DATE, 'YY');
    v_current_month := to_char(CURRENT_DATE, 'MM');
    
    -- Find the next number for this year-month
    SELECT COALESCE(MAX(
        CAST(
            CASE 
                WHEN ref ~ '^PR[0-9]{2}[0-9]{2}-[0-9]{4}$' 
                THEN substring(ref from 8 for 4)
                ELSE '0'
            END AS integer
        )
    ), 0) + 1 INTO v_current_num
    FROM llx_tabla
    WHERE ref ~ ('^PR' || v_current_year || v_current_month || '-[0-9]{4}$')
    AND entity = p_entity;
    
    -- Generate new reference
    v_new_ref := 'PR' || v_current_year || v_current_month || '-' || 
                 lpad(v_current_num::text, 4, '0');
    
    RETURN v_new_ref;
END;
$$ LANGUAGE plpgsql;
๐Ÿ”ข Prefix + Sequence Format (CU0001)
CREATE OR REPLACE FUNCTION llx_tabla_get_next_code(
    p_prefix varchar, 
    p_entity integer DEFAULT 1
) RETURNS varchar AS $$
DECLARE
    v_current_num integer;
    v_new_code varchar(50);
BEGIN
    -- Find the next number for the prefix
    SELECT COALESCE(MAX(
        CAST(substring(code_field from '[0-9]+$') AS integer)
    ), 0) + 1 INTO v_current_num
    FROM llx_tabla
    WHERE code_field ~ ('^' || p_prefix || '[0-9]+$') 
    AND entity = p_entity;
    
    -- Generate new code
    v_new_code := p_prefix || lpad(v_current_num::text, 4, '0');
    
    RETURN v_new_code;
END;
$$ LANGUAGE plpgsql;

๐Ÿงฎ Automatic Calculations Pattern

Used in 6 modules

Purpose: Automatic recalculation of totals, subtotals, prices with VAT and discounts.

Calculation Examples:

๐Ÿ’ฐ Detail Line Calculation
-- Trigger for quotation/order lines
CREATE OR REPLACE FUNCTION llx_ligne_before_insert_update()
RETURNS trigger AS $$
BEGIN
    -- Calculate line total HT
    NEW.total_ht := NEW.qty * NEW.subprice * (1 - NEW.remise_percent / 100);
    
    -- Calculate VAT
    NEW.total_tva := NEW.total_ht * NEW.tva_tx / 100;
    
    -- Calculate total TTC
    NEW.total_ttc := NEW.total_ht + NEW.total_tva;
    
    -- Set rang automatically
    IF NEW.rang IS NULL THEN
        SELECT COALESCE(MAX(rang), 0) + 1 INTO NEW.rang
        FROM llx_ligne
        WHERE fk_parent = NEW.fk_parent;
    END IF;
    
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;
๐Ÿ“Š Header Totals Update
-- Function to recalculate document totals
CREATE OR REPLACE FUNCTION llx_documento_update_totals(p_doc_id integer)
RETURNS void AS $$
DECLARE
    v_total_ht numeric(24,8) := 0;
    v_total_tva numeric(24,8) := 0;
    v_total_ttc numeric(24,8) := 0;
BEGIN
    -- Sum from detail lines
    SELECT 
        COALESCE(SUM(total_ht), 0),
        COALESCE(SUM(total_tva), 0),
        COALESCE(SUM(total_ttc), 0)
    INTO v_total_ht, v_total_tva, v_total_ttc
    FROM llx_documentodet
    WHERE fk_documento = p_doc_id;
    
    -- Update header
    UPDATE llx_documento
    SET 
        total_ht = v_total_ht,
        total_tva = v_total_tva,
        total_ttc = v_total_ttc,
        tms = NOW()
    WHERE rowid = p_doc_id;
END;
$$ LANGUAGE plpgsql;

๐Ÿ”„ State Control Pattern

Used in 8 modules

Purpose: Strict control of state transitions and business flow validations.

CREATE OR REPLACE FUNCTION llx_documento_validate_status_change()
RETURNS trigger AS $$
BEGIN
    -- Only process if state changed
    IF OLD.fk_statut != NEW.fk_statut THEN
        
        -- Validate allowed transitions
        CASE OLD.fk_statut
            WHEN 0 THEN -- Draft
                IF NEW.fk_statut NOT IN (1, -1) THEN
                    RAISE EXCEPTION 'Invalid transition from draft to state %', NEW.fk_statut;
                END IF;
                
            WHEN 1 THEN -- Validated
                IF NEW.fk_statut NOT IN (2, 3, -1) THEN
                    RAISE EXCEPTION 'Invalid transition from validated to state %', NEW.fk_statut;
                END IF;
                
            WHEN 2 THEN -- In process
                IF NEW.fk_statut NOT IN (3, -1) THEN
                    RAISE EXCEPTION 'Invalid transition from in process to state %', NEW.fk_statut;
                END IF;
                
            WHEN 3 THEN -- Closed
                RAISE EXCEPTION 'Cannot modify a closed document';
                
            WHEN -1 THEN -- Cancelled
                IF NEW.fk_statut != 0 THEN
                    RAISE EXCEPTION 'Can only reopen a cancelled document';
                END IF;
        END CASE;
        
        -- Specific actions per transition
        IF NEW.fk_statut = 1 AND OLD.fk_statut = 0 THEN
            -- When validating: set date and generate reference
            NEW.date_valid := COALESCE(NEW.date_valid, NOW());
            IF NEW.ref LIKE '(PROV%' THEN
                NEW.ref := llx_documento_get_next_ref(NEW.entity);
            END IF;
        END IF;
        
        IF NEW.fk_statut = -1 THEN
            -- When cancelling: clear process dates
            NEW.date_valid := NULL;
            NEW.date_close := NULL;
        END IF;
        
        IF NEW.fk_statut = 3 THEN
            -- When closing: set close date
            NEW.date_close := COALESCE(NEW.date_close, NOW());
        END IF;
    END IF;
    
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

๐Ÿ“ Automatic Audit Pattern

Used in 50 modules

Purpose: Automatic tracking of changes, timestamps and history management.

-- Trigger for automatic audit
CREATE OR REPLACE FUNCTION llx_audit_changes()
RETURNS trigger AS $$
DECLARE
    v_changes jsonb := '{}';
    v_campo text;
    v_old_value text;
    v_new_value text;
BEGIN
    -- For UPDATE: record changes
    IF TG_OP = 'UPDATE' THEN
        -- Update timestamp automatically
        NEW.tms := NOW();
        
        -- Detect significant changes for history
        FOR v_campo IN SELECT column_name 
                       FROM information_schema.columns 
                       WHERE table_name = TG_TABLE_NAME 
                       AND column_name NOT IN ('tms', 'rowid', 'datec')
        LOOP
            EXECUTE format('SELECT ($1).%I::text, ($2).%I::text', v_campo, v_campo)
            INTO v_old_value, v_new_value
            USING OLD, NEW;
            
            IF v_old_value IS DISTINCT FROM v_new_value THEN
                v_changes := v_changes || jsonb_build_object(
                    v_campo, jsonb_build_object(
                        'old', v_old_value,
                        'new', v_new_value
                    )
                );
            END IF;
        END LOOP;
        
        -- Record in audit table if there are significant changes
        IF v_changes != '{}' THEN
            INSERT INTO llx_audit_log (
                table_name, record_id, operation, changes, 
                user_id, change_date
            ) VALUES (
                TG_TABLE_NAME, NEW.rowid, 'UPDATE', v_changes,
                COALESCE(NEW.fk_user_modif, NEW.fk_user_author, 1), NOW()
            );
        END IF;
    END IF;
    
    -- For INSERT: record creation
    IF TG_OP = 'INSERT' THEN
        NEW.datec := COALESCE(NEW.datec, NOW());
        NEW.tms := COALESCE(NEW.tms, NOW());
        
        INSERT INTO llx_audit_log (
            table_name, record_id, operation, 
            user_id, change_date
        ) VALUES (
            TG_TABLE_NAME, NEW.rowid, 'INSERT',
            COALESCE(NEW.fk_user_author, 1), NOW()
        );
    END IF;
    
    RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;

โš ๏ธ Common Errors and Solutions

Lessons learned during migration to avoid recurring errors.

๐Ÿ—๏ธ Database Structure Errors

โŒ Error: Missing Relationship Tables

Frequency: High

Problem: PHP code expects relationship tables that don't exist in all installations.

-- Typical error in PostgreSQL
ERROR: relation "llx_categorie_fichinter" does not exist
โœ… Solution:
-- 1. ALWAYS verify existence before implementing functions
SELECT EXISTS (
    SELECT FROM information_schema.tables 
    WHERE table_name = 'llx_categorie_fichinter'
);

-- 2. Create table if it doesn't exist
CREATE TABLE IF NOT EXISTS llx_categorie_fichinter (
    fk_categorie integer NOT NULL,
    fk_fichinter integer NOT NULL,
    import_key varchar(14),
    PRIMARY KEY (fk_categorie, fk_fichinter)
);

-- 3. Add to structure test
SELECT has_table('llx_categorie_fichinter', 'Relationship table must exist');

โŒ Error: Data Type Mismatch

Frequency: Medium

Problem: PostgreSQL is strict with types, numeric vs integer comparisons fail.

โœ… Solution:
-- Use explicit cast in comparisons
SELECT is(
    (SELECT entity FROM llx_tabla WHERE id = 1)::integer,
    1,
    'entity must be 1'
);

-- In functions, declare explicit types
DECLARE
    v_amount numeric(24,8);
    v_count integer;
BEGIN
    -- Code with well-defined types
END;

๐Ÿงช Testing Errors

โŒ Error: Failing Timestamp Tests

Frequency: High

Problem: Triggers set tms=NOW() simultaneously with datec, comparisons fail.

-- Failed test
SELECT ok(
    (SELECT tms > datec FROM llx_tabla WHERE id = 1),
    'tms must be later than datec'
); -- FAILS because both are set to NOW()
โœ… Solution:
-- Use time window instead of direct comparison
SELECT ok(
    (SELECT tms >= CURRENT_TIMESTAMP - INTERVAL '1 second' 
     FROM llx_tabla WHERE id = 1),
    'tms must be updated recently'
);

-- Or verify that tms is present
SELECT ok(
    (SELECT tms IS NOT NULL FROM llx_tabla WHERE id = 1),
    'tms must be set automatically'
);

โŒ Error: Test Data Interference

Frequency: Medium

Problem: Tests modify data affecting subsequent tests.

โœ… Solution:
-- 1. Use specific data for each test
INSERT INTO llx_don (firstname, lastname, email, amount, fk_user_author)
VALUES ('Test', 'Specific', 'test_specific_case_29@test.com', 75, 1);

-- 2. Use transactions for isolation
BEGIN;
    -- Tests here
    SELECT * FROM finish();
ROLLBACK; -- Everything is automatically undone

-- 3. Clean up at the end of each critical test
DELETE FROM llx_tabla WHERE email LIKE 'test_%@test.com';

โš™๏ธ Trigger Errors

โŒ Error: Trigger Execution Order

Frequency: Medium

Problem: Triggers execute in alphabetical order, can cause problems.

โœ… Solution:
-- Use numbering in trigger names
CREATE TRIGGER trg_tabla_01_before_insert
    BEFORE INSERT ON llx_tabla
    FOR EACH ROW EXECUTE FUNCTION llx_tabla_validate();

CREATE TRIGGER trg_tabla_02_before_insert_generate
    BEFORE INSERT ON llx_tabla
    FOR EACH ROW EXECUTE FUNCTION llx_tabla_generate_code();

CREATE TRIGGER trg_tabla_03_after_insert_audit
    AFTER INSERT ON llx_tabla
    FOR EACH ROW EXECUTE FUNCTION llx_tabla_audit();

โŒ Error: Recursive Triggers

Frequency: Low

Problem: Trigger updates table that fires the same trigger in a loop.

โœ… Solution:
-- Use conditions to avoid recursion
CREATE OR REPLACE FUNCTION llx_tabla_update_parent()
RETURNS trigger AS $$
BEGIN
    -- Only update if something significant really changed
    IF OLD.total_amount IS DISTINCT FROM NEW.total_amount THEN
        -- Use UPDATE with specific WHERE to avoid trigger
        UPDATE llx_parent_table 
        SET total = (SELECT SUM(amount) FROM llx_tabla WHERE fk_parent = NEW.fk_parent)
        WHERE rowid = NEW.fk_parent
        AND rowid != NEW.rowid; -- Avoid self-reference
    END IF;
    
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

โšก Performance Errors

โŒ Error: Slow Queries in Code Generation

Frequency: Medium

Problem: Code generation functions can be slow without proper indexes.

โœ… Solution:
-- 1. Create specific indexes for code searches
CREATE INDEX idx_societe_code_client ON llx_societe(code_client) 
WHERE code_client IS NOT NULL;

CREATE INDEX idx_propal_ref_pattern ON llx_propal(ref) 
WHERE ref ~ '^PR[0-9]{4}-[0-9]{4}$';

-- 2. Use LIMIT in generation queries
SELECT COALESCE(MAX(numero), 0) + 1
FROM (
    SELECT CAST(substring(ref from 8 for 4) AS integer) as numero
    FROM llx_tabla 
    WHERE ref ~ '^PR2501-[0-9]{4}$'
    ORDER BY numero DESC
    LIMIT 100  -- Limit search to last N
) AS subconsulta;

๐ŸŒŸ Established Best Practices

๐Ÿ“‹ Documentation

  • ALWAYS create MIGRATION_PLAN.md before implementing
  • Keep MIGRATION_LOG.md updated in real time
  • Comment PostgreSQL code in English
  • Document technical decisions and discarded alternatives
  • Include usage examples in function comments

๐Ÿงช Testing

  • Structure tests BEFORE functions
  • 100% mandatory coverage in pgTAP tests
  • Comparative tests to validate PHP parity
  • Isolated and predictable test data
  • Use transactions for test isolation

โš™๏ธ Implementation

  • Divide complex tasks into small subtasks
  • Use numbering in trigger names (01_, 02_)
  • Validate ALL expected relationship tables
  • Create backups (.orig) before modifying PHP
  • REMOVE all PHP logic without exceptions

๐Ÿ”ง Code

  • Functions with a specific and clear purpose
  • Error messages consistent with original PHP
  • Use COALESCE for default values
  • Explicit cast in type comparisons
  • Explanatory comments in complex logic

๐ŸŽฏ Critical Lessons Learned

Day 1

๐Ÿ” "Understand Before Implementing"

It's fundamental to completely analyze existing PHP behavior before writing the first line of PostgreSQL. Comparative tests are essential to validate parity.

Day 1

๐Ÿ“Š "Structure Tests First"

ALWAYS verify that all tables and fields exist before implementing functions. A failed structure test can save hours of debugging.

Day 2

๐Ÿ”ง "Divide and Conquer"

Complex tasks are best solved by dividing them into small, manageable parts. A complex module may require 10-15 specific subtasks.

Day 2

โš–๏ธ "Don't Improve, Just Migrate"

The goal is to replicate EXACTLY the PHP behavior, not improve it. Optimizations can introduce subtle discrepancies.

Day 3

๐Ÿ—‚๏ธ "Verify Relationship Tables"

Relationship tables may not exist if modules were added later. ALWAYS verify and create missing ones before implementing.

Day 3

๐ŸŽฏ "Testing Precision"

In high-precision work, not a single failing test can be allowed. 100% of tests passing is a non-negotiable requirement.

๐Ÿ› ๏ธ Developed Tools

๐Ÿ“Š sync-test-databases-final.sh

Automated script to synchronize the 3 testing databases, ensuring identical environments for comparative tests.

  • Complete structure synchronization
  • Selective copy of test data
  • Post-sync integrity validation

๐Ÿงช run_pgtap_tests.sh

Automated executor for all pgTAP tests with detailed reporting and coverage statistics.

  • Parallel test execution
  • Reporting with colors and statistics
  • Automatic detection of new tests

โš–๏ธ Comparative Tests

System of PHP and SQL scripts to compare behavior between original implementation and PostgreSQL.

  • Automatic capture of PHP behavior
  • Parallel execution on both databases
  • Automated comparison of results