๐ฏ Patterns and Lessons
Complete library of reusable patterns and critical lessons distilled during migration of 50 modules + 5 advanced migrations. Practical knowledge to accelerate future implementations.
Proven Patterns
35 PostgreSQL patterns validated in production
Documented Errors
Common problems identified and solved
Ready Solutions
Copy-paste code for future implementations
๐ฏ Reusable PostgreSQL Patterns
Collection of proven and optimized patterns developed during migration of 50 modules.
๐ Base Validation Pattern
Used in 50 modulesPurpose: Standard validation for required fields, email, data cleaning and default values.
Pattern Code:
CREATE OR REPLACE FUNCTION llx_{tabla}_before_insert()
RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
-- 1. Required fields validation
IF NEW.required_field IS NULL OR trim(NEW.required_field) = '' THEN
RAISE EXCEPTION 'ErrorFieldRequired: required_field';
END IF;
-- 2. Automatic data cleaning
NEW.text_field = trim(NEW.text_field);
-- 3. Email validation with regex
IF NEW.email IS NOT NULL AND NEW.email != '' THEN
NEW.email = trim(lower(NEW.email));
IF NOT (NEW.email ~* '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$') THEN
RAISE EXCEPTION 'ErrorBadEMail: %', NEW.email;
END IF;
END IF;
-- 4. Phone formatting
IF NEW.phone IS NOT NULL THEN
NEW.phone = regexp_replace(NEW.phone, '[^0-9+\-\.\s()]', '', 'g');
NEW.phone = trim(NEW.phone);
END IF;
-- 5. Default values
NEW.entity := COALESCE(NEW.entity, 1);
NEW.status := COALESCE(NEW.status, 1);
NEW.datec := COALESCE(NEW.datec, NOW());
-- 6. Numeric values validation
IF NEW.numeric_field IS NOT NULL AND NEW.numeric_field < 0 THEN
RAISE EXCEPTION 'Field cannot be negative';
END IF;
RETURN NEW;
END;
$$;
โ Benefits:
- Guaranteed automatic validation
- Always clean and normalized data
- Consistent error messages
- Facilitates debugging and maintenance
๐ท๏ธ Code Generation Pattern
Used in 7 modulesPurpose: Automatic generation of unique and consecutive codes with customizable formats.
Implemented Variants:
๐ Year-Month-Sequence Format (PR2501-0001)
CREATE OR REPLACE FUNCTION llx_tabla_get_next_ref(p_entity integer DEFAULT 1)
RETURNS varchar AS $$
DECLARE
v_current_year varchar(2);
v_current_month varchar(2);
v_current_num integer;
v_new_ref varchar(30);
BEGIN
-- Get current year and month
v_current_year := to_char(CURRENT_DATE, 'YY');
v_current_month := to_char(CURRENT_DATE, 'MM');
-- Find the next number for this year-month
SELECT COALESCE(MAX(
CAST(
CASE
WHEN ref ~ '^PR[0-9]{2}[0-9]{2}-[0-9]{4}$'
THEN substring(ref from 8 for 4)
ELSE '0'
END AS integer
)
), 0) + 1 INTO v_current_num
FROM llx_tabla
WHERE ref ~ ('^PR' || v_current_year || v_current_month || '-[0-9]{4}$')
AND entity = p_entity;
-- Generate new reference
v_new_ref := 'PR' || v_current_year || v_current_month || '-' ||
lpad(v_current_num::text, 4, '0');
RETURN v_new_ref;
END;
$$ LANGUAGE plpgsql;
๐ข Prefix + Sequence Format (CU0001)
CREATE OR REPLACE FUNCTION llx_tabla_get_next_code(
p_prefix varchar,
p_entity integer DEFAULT 1
) RETURNS varchar AS $$
DECLARE
v_current_num integer;
v_new_code varchar(50);
BEGIN
-- Find the next number for the prefix
SELECT COALESCE(MAX(
CAST(substring(code_field from '[0-9]+$') AS integer)
), 0) + 1 INTO v_current_num
FROM llx_tabla
WHERE code_field ~ ('^' || p_prefix || '[0-9]+$')
AND entity = p_entity;
-- Generate new code
v_new_code := p_prefix || lpad(v_current_num::text, 4, '0');
RETURN v_new_code;
END;
$$ LANGUAGE plpgsql;
๐งฎ Automatic Calculations Pattern
Used in 6 modulesPurpose: Automatic recalculation of totals, subtotals, prices with VAT and discounts.
Calculation Examples:
๐ฐ Detail Line Calculation
-- Trigger for quotation/order lines
CREATE OR REPLACE FUNCTION llx_ligne_before_insert_update()
RETURNS trigger AS $$
BEGIN
-- Calculate line total HT
NEW.total_ht := NEW.qty * NEW.subprice * (1 - NEW.remise_percent / 100);
-- Calculate VAT
NEW.total_tva := NEW.total_ht * NEW.tva_tx / 100;
-- Calculate total TTC
NEW.total_ttc := NEW.total_ht + NEW.total_tva;
-- Set rang automatically
IF NEW.rang IS NULL THEN
SELECT COALESCE(MAX(rang), 0) + 1 INTO NEW.rang
FROM llx_ligne
WHERE fk_parent = NEW.fk_parent;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
๐ Header Totals Update
-- Function to recalculate document totals
CREATE OR REPLACE FUNCTION llx_documento_update_totals(p_doc_id integer)
RETURNS void AS $$
DECLARE
v_total_ht numeric(24,8) := 0;
v_total_tva numeric(24,8) := 0;
v_total_ttc numeric(24,8) := 0;
BEGIN
-- Sum from detail lines
SELECT
COALESCE(SUM(total_ht), 0),
COALESCE(SUM(total_tva), 0),
COALESCE(SUM(total_ttc), 0)
INTO v_total_ht, v_total_tva, v_total_ttc
FROM llx_documentodet
WHERE fk_documento = p_doc_id;
-- Update header
UPDATE llx_documento
SET
total_ht = v_total_ht,
total_tva = v_total_tva,
total_ttc = v_total_ttc,
tms = NOW()
WHERE rowid = p_doc_id;
END;
$$ LANGUAGE plpgsql;
๐ State Control Pattern
Used in 8 modulesPurpose: Strict control of state transitions and business flow validations.
CREATE OR REPLACE FUNCTION llx_documento_validate_status_change()
RETURNS trigger AS $$
BEGIN
-- Only process if state changed
IF OLD.fk_statut != NEW.fk_statut THEN
-- Validate allowed transitions
CASE OLD.fk_statut
WHEN 0 THEN -- Draft
IF NEW.fk_statut NOT IN (1, -1) THEN
RAISE EXCEPTION 'Invalid transition from draft to state %', NEW.fk_statut;
END IF;
WHEN 1 THEN -- Validated
IF NEW.fk_statut NOT IN (2, 3, -1) THEN
RAISE EXCEPTION 'Invalid transition from validated to state %', NEW.fk_statut;
END IF;
WHEN 2 THEN -- In process
IF NEW.fk_statut NOT IN (3, -1) THEN
RAISE EXCEPTION 'Invalid transition from in process to state %', NEW.fk_statut;
END IF;
WHEN 3 THEN -- Closed
RAISE EXCEPTION 'Cannot modify a closed document';
WHEN -1 THEN -- Cancelled
IF NEW.fk_statut != 0 THEN
RAISE EXCEPTION 'Can only reopen a cancelled document';
END IF;
END CASE;
-- Specific actions per transition
IF NEW.fk_statut = 1 AND OLD.fk_statut = 0 THEN
-- When validating: set date and generate reference
NEW.date_valid := COALESCE(NEW.date_valid, NOW());
IF NEW.ref LIKE '(PROV%' THEN
NEW.ref := llx_documento_get_next_ref(NEW.entity);
END IF;
END IF;
IF NEW.fk_statut = -1 THEN
-- When cancelling: clear process dates
NEW.date_valid := NULL;
NEW.date_close := NULL;
END IF;
IF NEW.fk_statut = 3 THEN
-- When closing: set close date
NEW.date_close := COALESCE(NEW.date_close, NOW());
END IF;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
๐ Automatic Audit Pattern
Used in 50 modulesPurpose: Automatic tracking of changes, timestamps and history management.
-- Trigger for automatic audit
CREATE OR REPLACE FUNCTION llx_audit_changes()
RETURNS trigger AS $$
DECLARE
v_changes jsonb := '{}';
v_campo text;
v_old_value text;
v_new_value text;
BEGIN
-- For UPDATE: record changes
IF TG_OP = 'UPDATE' THEN
-- Update timestamp automatically
NEW.tms := NOW();
-- Detect significant changes for history
FOR v_campo IN SELECT column_name
FROM information_schema.columns
WHERE table_name = TG_TABLE_NAME
AND column_name NOT IN ('tms', 'rowid', 'datec')
LOOP
EXECUTE format('SELECT ($1).%I::text, ($2).%I::text', v_campo, v_campo)
INTO v_old_value, v_new_value
USING OLD, NEW;
IF v_old_value IS DISTINCT FROM v_new_value THEN
v_changes := v_changes || jsonb_build_object(
v_campo, jsonb_build_object(
'old', v_old_value,
'new', v_new_value
)
);
END IF;
END LOOP;
-- Record in audit table if there are significant changes
IF v_changes != '{}' THEN
INSERT INTO llx_audit_log (
table_name, record_id, operation, changes,
user_id, change_date
) VALUES (
TG_TABLE_NAME, NEW.rowid, 'UPDATE', v_changes,
COALESCE(NEW.fk_user_modif, NEW.fk_user_author, 1), NOW()
);
END IF;
END IF;
-- For INSERT: record creation
IF TG_OP = 'INSERT' THEN
NEW.datec := COALESCE(NEW.datec, NOW());
NEW.tms := COALESCE(NEW.tms, NOW());
INSERT INTO llx_audit_log (
table_name, record_id, operation,
user_id, change_date
) VALUES (
TG_TABLE_NAME, NEW.rowid, 'INSERT',
COALESCE(NEW.fk_user_author, 1), NOW()
);
END IF;
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
โ ๏ธ Common Errors and Solutions
Lessons learned during migration to avoid recurring errors.
๐๏ธ Database Structure Errors
โ Error: Missing Relationship Tables
Frequency: HighProblem: PHP code expects relationship tables that don't exist in all installations.
-- Typical error in PostgreSQL
ERROR: relation "llx_categorie_fichinter" does not exist
โ Solution:
-- 1. ALWAYS verify existence before implementing functions
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_name = 'llx_categorie_fichinter'
);
-- 2. Create table if it doesn't exist
CREATE TABLE IF NOT EXISTS llx_categorie_fichinter (
fk_categorie integer NOT NULL,
fk_fichinter integer NOT NULL,
import_key varchar(14),
PRIMARY KEY (fk_categorie, fk_fichinter)
);
-- 3. Add to structure test
SELECT has_table('llx_categorie_fichinter', 'Relationship table must exist');
โ Error: Data Type Mismatch
Frequency: MediumProblem: PostgreSQL is strict with types, numeric vs integer comparisons fail.
โ Solution:
-- Use explicit cast in comparisons
SELECT is(
(SELECT entity FROM llx_tabla WHERE id = 1)::integer,
1,
'entity must be 1'
);
-- In functions, declare explicit types
DECLARE
v_amount numeric(24,8);
v_count integer;
BEGIN
-- Code with well-defined types
END;
๐งช Testing Errors
โ Error: Failing Timestamp Tests
Frequency: HighProblem: Triggers set tms=NOW() simultaneously with datec, comparisons fail.
-- Failed test
SELECT ok(
(SELECT tms > datec FROM llx_tabla WHERE id = 1),
'tms must be later than datec'
); -- FAILS because both are set to NOW()
โ Solution:
-- Use time window instead of direct comparison
SELECT ok(
(SELECT tms >= CURRENT_TIMESTAMP - INTERVAL '1 second'
FROM llx_tabla WHERE id = 1),
'tms must be updated recently'
);
-- Or verify that tms is present
SELECT ok(
(SELECT tms IS NOT NULL FROM llx_tabla WHERE id = 1),
'tms must be set automatically'
);
โ Error: Test Data Interference
Frequency: MediumProblem: Tests modify data affecting subsequent tests.
โ Solution:
-- 1. Use specific data for each test
INSERT INTO llx_don (firstname, lastname, email, amount, fk_user_author)
VALUES ('Test', 'Specific', 'test_specific_case_29@test.com', 75, 1);
-- 2. Use transactions for isolation
BEGIN;
-- Tests here
SELECT * FROM finish();
ROLLBACK; -- Everything is automatically undone
-- 3. Clean up at the end of each critical test
DELETE FROM llx_tabla WHERE email LIKE 'test_%@test.com';
โ๏ธ Trigger Errors
โ Error: Trigger Execution Order
Frequency: MediumProblem: Triggers execute in alphabetical order, can cause problems.
โ Solution:
-- Use numbering in trigger names
CREATE TRIGGER trg_tabla_01_before_insert
BEFORE INSERT ON llx_tabla
FOR EACH ROW EXECUTE FUNCTION llx_tabla_validate();
CREATE TRIGGER trg_tabla_02_before_insert_generate
BEFORE INSERT ON llx_tabla
FOR EACH ROW EXECUTE FUNCTION llx_tabla_generate_code();
CREATE TRIGGER trg_tabla_03_after_insert_audit
AFTER INSERT ON llx_tabla
FOR EACH ROW EXECUTE FUNCTION llx_tabla_audit();
โ Error: Recursive Triggers
Frequency: LowProblem: Trigger updates table that fires the same trigger in a loop.
โ Solution:
-- Use conditions to avoid recursion
CREATE OR REPLACE FUNCTION llx_tabla_update_parent()
RETURNS trigger AS $$
BEGIN
-- Only update if something significant really changed
IF OLD.total_amount IS DISTINCT FROM NEW.total_amount THEN
-- Use UPDATE with specific WHERE to avoid trigger
UPDATE llx_parent_table
SET total = (SELECT SUM(amount) FROM llx_tabla WHERE fk_parent = NEW.fk_parent)
WHERE rowid = NEW.fk_parent
AND rowid != NEW.rowid; -- Avoid self-reference
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
โก Performance Errors
โ Error: Slow Queries in Code Generation
Frequency: MediumProblem: Code generation functions can be slow without proper indexes.
โ Solution:
-- 1. Create specific indexes for code searches
CREATE INDEX idx_societe_code_client ON llx_societe(code_client)
WHERE code_client IS NOT NULL;
CREATE INDEX idx_propal_ref_pattern ON llx_propal(ref)
WHERE ref ~ '^PR[0-9]{4}-[0-9]{4}$';
-- 2. Use LIMIT in generation queries
SELECT COALESCE(MAX(numero), 0) + 1
FROM (
SELECT CAST(substring(ref from 8 for 4) AS integer) as numero
FROM llx_tabla
WHERE ref ~ '^PR2501-[0-9]{4}$'
ORDER BY numero DESC
LIMIT 100 -- Limit search to last N
) AS subconsulta;
๐ Established Best Practices
๐ Documentation
- ALWAYS create MIGRATION_PLAN.md before implementing
- Keep MIGRATION_LOG.md updated in real time
- Comment PostgreSQL code in English
- Document technical decisions and discarded alternatives
- Include usage examples in function comments
๐งช Testing
- Structure tests BEFORE functions
- 100% mandatory coverage in pgTAP tests
- Comparative tests to validate PHP parity
- Isolated and predictable test data
- Use transactions for test isolation
โ๏ธ Implementation
- Divide complex tasks into small subtasks
- Use numbering in trigger names (01_, 02_)
- Validate ALL expected relationship tables
- Create backups (.orig) before modifying PHP
- REMOVE all PHP logic without exceptions
๐ง Code
- Functions with a specific and clear purpose
- Error messages consistent with original PHP
- Use COALESCE for default values
- Explicit cast in type comparisons
- Explanatory comments in complex logic
๐ฏ Critical Lessons Learned
๐ "Understand Before Implementing"
It's fundamental to completely analyze existing PHP behavior before writing the first line of PostgreSQL. Comparative tests are essential to validate parity.
๐ "Structure Tests First"
ALWAYS verify that all tables and fields exist before implementing functions. A failed structure test can save hours of debugging.
๐ง "Divide and Conquer"
Complex tasks are best solved by dividing them into small, manageable parts. A complex module may require 10-15 specific subtasks.
โ๏ธ "Don't Improve, Just Migrate"
The goal is to replicate EXACTLY the PHP behavior, not improve it. Optimizations can introduce subtle discrepancies.
๐๏ธ "Verify Relationship Tables"
Relationship tables may not exist if modules were added later. ALWAYS verify and create missing ones before implementing.
๐ฏ "Testing Precision"
In high-precision work, not a single failing test can be allowed. 100% of tests passing is a non-negotiable requirement.
๐ ๏ธ Developed Tools
๐ sync-test-databases-final.sh
Automated script to synchronize the 3 testing databases, ensuring identical environments for comparative tests.
- Complete structure synchronization
- Selective copy of test data
- Post-sync integrity validation
๐งช run_pgtap_tests.sh
Automated executor for all pgTAP tests with detailed reporting and coverage statistics.
- Parallel test execution
- Reporting with colors and statistics
- Automatic detection of new tests
โ๏ธ Comparative Tests
System of PHP and SQL scripts to compare behavior between original implementation and PostgreSQL.
- Automatic capture of PHP behavior
- Parallel execution on both databases
- Automated comparison of results