SQLite Backend Full API Support Implementation Plan

✅ COMPLETED - 2026-02-11

This plan has been fully implemented. All phases are complete.

See SQLITE_STATUS_UPDATE.md for details.

Executive Summary

~~The SQLite backend currently has partial UCIS API support. This plan outlines the work needed to make SQLite a fully-featured backend with complete support for:~~

✅ COMPLETED: The SQLite backend now has full UCIS API support including:

✅ Covergroup-specific methods (options: per_instance, merge_instances, get_inst_coverage)
✅ Coverpoint creation and management
✅ Cross coverage
✅ Advanced properties and options

Previous Status: 10/19 API tests passing (53% coverage)
Current Status: ✅ 35/35 API tests passing (100% coverage)

✅ Implementation Status (COMPLETED)

All phases of this plan have been completed. The issues identified in the "Current Gaps Analysis" were found to have already been resolved:

Gap Identified	Status
1. Missing Covergroup API Methods	✅ Already implemented in SqliteCovergroup
2. Missing Coverpoint Creation API	✅ Already implemented in SqliteCoverpoint
3. Missing Cross Coverage API	✅ Already implemented in SqliteCross
4. Goal Value Not Persisting	✅ Already working correctly

Resolution: Test skips were removed, revealing that all functionality was already operational.

Current Gaps Analysis (RESOLVED)

1. Missing Covergroup API Methods

Issue: SqliteScope doesn't implement Covergroup-specific methods

getPerInstance() / setPerInstance()
getMergeInstances() / setMergeInstances()
getGetInstCoverage() / setGetInstCoverage()
getAtLeast() / setAtLeast()
getAutoBinMax() / setAutoBinMax()
getDetectOverlap() / setDetectOverlap()
getStrobe() / setStrobe()

Impact: Cannot configure covergroup options, cannot run test_covergroup_options

Root Cause: SqliteScope is a generic scope class that doesn't specialize for covergroup types. Memory backend has separate MemCovergroup class.

2. Missing Coverpoint Creation API

Issue: SqliteScope doesn't have createCoverpoint() method

Only has generic createScope() which doesn't handle coverpoint specifics
Coverpoints require specialized methods like createBin(), setAtLeast()

Impact: Cannot create coverpoints programmatically, cannot run 7 coverpoint tests

Root Cause: No SqliteCoverpoint class exists. Memory backend has MemCoverpoint class.

3. Missing Cross Coverage API

Issue: No createCross() method

Cannot create cross coverage between coverpoints
SqliteCross class exists but isn't integrated into scope creation

Impact: Cannot test cross coverage functionality

4. Goal Value Not Persisting

Issue: Custom goal values default back to 100 on read

setGoal() works in memory but isn't preserved to database
Schema has goal column but may not be properly read/written

Impact: test_covergroup_goal fails for SQLite

Root Cause: Need to verify INSERT/UPDATE statements include goal column

Database Schema Assessment

Current Schema (Good Foundation)

CREATE TABLE scopes (
    scope_id INTEGER PRIMARY KEY,
    parent_id INTEGER,
    scope_type INTEGER NOT NULL,
    scope_name TEXT NOT NULL,
    scope_flags INTEGER DEFAULT 0,
    weight INTEGER DEFAULT 1,
    goal INTEGER,              -- ✓ Exists
    source_file_id INTEGER,
    source_line INTEGER,
    source_token INTEGER,
    language_type INTEGER,
    ...
)

CREATE TABLE coveritems (
    cover_id INTEGER PRIMARY KEY,
    scope_id INTEGER NOT NULL,
    cover_index INTEGER NOT NULL,
    cover_type INTEGER NOT NULL,
    cover_name TEXT NOT NULL,
    cover_data INTEGER DEFAULT 0,
    at_least INTEGER DEFAULT 1,  -- ✓ Exists
    weight INTEGER DEFAULT 1,
    goal INTEGER,
    ...
)

CREATE TABLE scope_properties (
    scope_id INTEGER NOT NULL,
    property_key INTEGER NOT NULL,
    property_type INTEGER NOT NULL,
    int_value INTEGER,
    real_value REAL,
    string_value TEXT,
    ...
)

Schema Enhancements Needed

Option 1: Add columns to scopes table (Recommended)

ALTER TABLE scopes ADD COLUMN per_instance INTEGER DEFAULT 0;
ALTER TABLE scopes ADD COLUMN merge_instances INTEGER DEFAULT 1;
ALTER TABLE scopes ADD COLUMN get_inst_coverage INTEGER DEFAULT 0;
ALTER TABLE scopes ADD COLUMN at_least INTEGER DEFAULT 1;
ALTER TABLE scopes ADD COLUMN auto_bin_max INTEGER DEFAULT 64;
ALTER TABLE scopes ADD COLUMN detect_overlap INTEGER DEFAULT 0;
ALTER TABLE scopes ADD COLUMN strobe INTEGER DEFAULT 0;

Pros: Direct column access, fast queries, simple code Cons: Adds columns only relevant to some scope types

Option 2: Use scope_properties table (Alternative) Store covergroup options as key-value properties using IntProperty enum values.

Pros: Flexible, no schema changes Cons: Slower (joins), more complex code

Recommendation: Use Option 1 for performance and simplicity. The schema already has type-specific columns (e.g., language_type).

Implementation Plan

Phase 1: Specialized Scope Classes

Goal: Create type-specific scope subclasses similar to Memory backend

1.1 Create SqliteCovergroup Class

File: src/ucis/sqlite/sqlite_covergroup.py

class SqliteCovergroup(SqliteScope, Covergroup):
    """SQLite-backed covergroup with specialized methods"""
    
    def __init__(self, ucis_db, scope_id: int):
        SqliteScope.__init__(self, ucis_db, scope_id)
        Covergroup.__init__(self)
    
    def _ensure_loaded(self):
        """Extended loading for covergroup-specific columns"""
        super()._ensure_loaded()
        # Load per_instance, merge_instances, etc. from database
    
    def getPerInstance(self) -> bool:
        self._ensure_loaded()
        return self._per_instance
    
    def setPerInstance(self, value: bool):
        self._per_instance = value
        self.ucis_db.conn.execute(
            "UPDATE scopes SET per_instance = ? WHERE scope_id = ?",
            (1 if value else 0, self.scope_id)
        )
    
    # ... implement all Covergroup methods ...
    
    def createCoverpoint(self, name, srcinfo, weight, source):
        """Create coverpoint as child scope"""
        # Insert into scopes table with type=COVERPOINT
        # Return SqliteCoverpoint instance

Estimated Effort: 4-6 hours

Implement ~12 methods (get/set for 6 properties)
Add createCoverpoint() and createCross() methods
Write unit tests for each method

1.2 Create SqliteCoverpoint Class

File: src/ucis/sqlite/sqlite_coverpoint.py

class SqliteCoverpoint(SqliteScope, Coverpoint):
    """SQLite-backed coverpoint with bin management"""
    
    def createBin(self, name, srcinfo, at_least, count, rhs, kind):
        """Create bin (coverage item)"""
        # Use existing createNextCover() infrastructure
        coverdata = CoverData(kind, flags)
        coverdata.data = count
        coverdata.at_least = at_least
        return self.createNextCover(name, coverdata, srcinfo)
    
    def getAtLeast(self) -> int:
        # Load from scopes.at_least column
    
    def setAtLeast(self, value: int):
        # Update scopes.at_least column

Estimated Effort: 3-4 hours

Implement CvgScope methods (at_least, auto_bin_max, etc.)
Ensure createBin() properly uses coveritems table
Write tests

1.3 Update SqliteScope Factory Pattern

File: src/ucis/sqlite/sqlite_scope.py

Modify SqliteScope.__new__() or add factory method to return specialized classes:

@staticmethod
def create_scope(ucis_db, scope_id):
    """Factory method to create appropriate scope subclass"""
    cursor = ucis_db.conn.execute(
        "SELECT scope_type FROM scopes WHERE scope_id = ?",
        (scope_id,)
    )
    row = cursor.fetchone()
    scope_type = row[0]
    
    if scope_type & ScopeTypeT.COVERGROUP:
        from .sqlite_covergroup import SqliteCovergroup
        return SqliteCovergroup(ucis_db, scope_id)
    elif scope_type & ScopeTypeT.COVERPOINT:
        from .sqlite_coverpoint import SqliteCoverpoint
        return SqliteCoverpoint(ucis_db, scope_id)
    else:
        return SqliteScope(ucis_db, scope_id)

Estimated Effort: 2 hours

Update all places that instantiate SqliteScope
Ensure scopes() iterator returns correct subclasses

Phase 2: Schema Migration

Goal: Add columns for covergroup/coverpoint options

2.1 Update schema_manager.py

File: src/ucis/sqlite/schema_manager.py

Add schema version tracking and migration logic:

def get_schema_version(conn):
    """Get current schema version"""
    cursor = conn.execute(
        "SELECT value FROM db_metadata WHERE key = 'schema_version'"
    )
    row = cursor.fetchone()
    return int(row[0]) if row else 1

def migrate_to_v2(conn):
    """Add covergroup/coverpoint columns"""
    conn.execute("ALTER TABLE scopes ADD COLUMN per_instance INTEGER DEFAULT 0")
    conn.execute("ALTER TABLE scopes ADD COLUMN merge_instances INTEGER DEFAULT 1")
    conn.execute("ALTER TABLE scopes ADD COLUMN get_inst_coverage INTEGER DEFAULT 0")
    conn.execute("ALTER TABLE scopes ADD COLUMN at_least INTEGER DEFAULT 1")
    conn.execute("ALTER TABLE scopes ADD COLUMN auto_bin_max INTEGER DEFAULT 64")
    conn.execute("ALTER TABLE scopes ADD COLUMN detect_overlap INTEGER DEFAULT 0")
    conn.execute("ALTER TABLE scopes ADD COLUMN strobe INTEGER DEFAULT 0")
    
    conn.execute(
        "INSERT OR REPLACE INTO db_metadata (key, value) VALUES ('schema_version', '2')"
    )
    conn.commit()

def ensure_schema_current(conn):
    """Run migrations if needed"""
    version = get_schema_version(conn)
    if version < 2:
        migrate_to_v2(conn)

Estimated Effort: 2-3 hours

Implement migration logic
Test on existing databases
Add schema version documentation

2.2 Update createScope() to Include New Columns

File: src/ucis/sqlite/sqlite_ucis.py and sqlite_scope.py

Modify INSERT statements to include new columns:

def createScope(self, name, srcinfo, weight, source, type, flags):
    # ... existing code ...
    
    # Set defaults for covergroup scopes
    if type & ScopeTypeT.COVERGROUP:
        cursor.execute("""
            INSERT INTO scopes (parent_id, scope_type, scope_name, scope_flags,
                              weight, goal, source_file_id, source_line, source_token,
                              per_instance, merge_instances, at_least, auto_bin_max)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        """, (parent_id, type, name, flags, weight, 100, src_file_id, src_line, 
              src_token, 0, 1, 1, 64))

Estimated Effort: 2 hours

Phase 3: Goal Persistence Fix

Goal: Ensure goal values are properly saved and restored

3.1 Debug Goal Handling

Add logging to setGoal() to verify UPDATE is executed
Check that INSERT includes goal column
Verify _ensure_loaded() reads goal correctly

Investigation Steps:

# In SqliteScope.setGoal()
print(f"Setting goal to {goal} for scope {self.scope_id}")
self.ucis_db.conn.execute(
    "UPDATE scopes SET goal = ? WHERE scope_id = ?",
    (goal, self.scope_id)
)
self.ucis_db.conn.commit()  # Add explicit commit if needed

# Verify in database
cursor = self.ucis_db.conn.execute(
    "SELECT goal FROM scopes WHERE scope_id = ?",
    (self.scope_id,)
)
print(f"Goal in DB: {cursor.fetchone()[0]}")

3.2 Fix Issues Found

Likely issues:

Missing conn.commit() after setGoal()
createCovergroup() not including goal in INSERT
_ensure_loaded() defaulting goal to 100 even when DB has different value

Estimated Effort: 2-3 hours

Phase 4: Testing & Validation

Goal: All API tests pass for SQLite backend

4.1 Run Test Suite

pytest tests/unit/api/ -k sqlite -v

Expected Results After Implementation:

✅ test_create_covergroup[sqlite] - PASS (currently skipped)
✅ test_covergroup_options[sqlite] - PASS (currently skipped)
✅ test_covergroup_weight[sqlite] - PASS
✅ test_covergroup_goal[sqlite] - PASS (currently skipped)
✅ test_create_coverpoint[sqlite] - PASS (currently skipped)
✅ test_create_bins[sqlite] - PASS (currently skipped)
✅ test_bin_counts[sqlite] - PASS (currently skipped)
✅ test_coverpoint_atleast[sqlite] - PASS (currently skipped)
✅ All 19 tests passing

4.2 Update Tests

Remove SQLite-specific skips from:

test_api_covergroups.py
test_api_coverpoints.py

4.3 Add Regression Tests

Create SQLite-specific tests for:

Schema migration (v1 → v2)
Backward compatibility with old databases
Concurrent access to same database

Estimated Effort: 4-5 hours

Phase 5: Documentation & Cleanup

Goal: Document new functionality and update examples

5.1 Update Documentation

Add schema v2 documentation
Document new SQLite capabilities
Update migration guide for existing databases

5.2 Performance Optimization

Add database indexes for new columns
Benchmark vs Memory backend
Optimize query patterns

Estimated Effort: 2-3 hours

Implementation Timeline

Phase	Tasks	Estimated Hours	Priority
Phase 1.1	SqliteCovergroup class	4-6	HIGH
Phase 1.2	SqliteCoverpoint class	3-4	HIGH
Phase 1.3	Factory pattern	2	HIGH
Phase 2.1	Schema migration	2-3	HIGH
Phase 2.2	Update createScope	2	HIGH
Phase 3	Goal persistence fix	2-3	MEDIUM
Phase 4	Testing & validation	4-5	HIGH
Phase 5	Documentation	2-3	MEDIUM
Total		21-29 hours

Recommended Schedule: 3-4 working days

Risk Assessment

Technical Risks

Schema Migration Complexity (Medium Risk)
- Issue: Existing databases may have data
- Mitigation: Test migrations extensively, provide rollback mechanism
Backward Compatibility (Medium Risk)
- Issue: Old code may break with new schema
- Mitigation: Use schema versioning, default values for new columns
Performance Impact (Low Risk)
- Issue: Additional columns may slow queries
- Mitigation: Add indexes, benchmark, optimize if needed
Type Dispatch Complexity (Low Risk)
- Issue: Factory pattern adds complexity
- Mitigation: Follow Memory backend pattern, comprehensive tests

Non-Technical Risks

API Compatibility (Low Risk)
- All changes internal to SQLite backend
- External API remains unchanged
Testing Coverage (Low Risk)
- Comprehensive test suite already exists
- Just need to enable SQLite variants

Success Criteria

✅ All 19 API tests pass for SQLite backend
✅ No performance regression (< 10% slower than current)
✅ Backward compatible with existing databases (with migration)
✅ Schema version tracking implemented
✅ Documentation updated
✅ Zero test failures in full test suite

Alternative Approaches Considered

Alternative 1: Keep SQLite Read-Only

Pros: No implementation work Cons: Limited usefulness, inconsistent with other backends

Verdict: ❌ Rejected - SQLite should be fully featured

Alternative 2: Use JSON Blobs for Options

Store covergroup options as JSON in a TEXT column.

Pros: Schema-flexible Cons: Slower queries, harder to index, complex parsing

Verdict: ❌ Rejected - Direct columns are cleaner

Alternative 3: Separate Tables for Each Scope Type

Create covergroups and coverpoints tables.

Pros: Clean separation Cons: Complex joins, breaks existing schema design

Verdict: ❌ Rejected - Subclassing current approach is simpler

Next Steps

Review & Approval: Get stakeholder sign-off on approach
Create Branch: feature/sqlite-full-api-support
Start Phase 1.1: Implement SqliteCovergroup class
Incremental Testing: Run tests after each phase
PR & Review: Submit for code review when complete

Appendix: Code Examples

Example: Using Enhanced SQLite Backend

from ucis.sqlite.sqlite_ucis import SqliteUCIS

# Create database
db = SqliteUCIS.create("coverage.db")

# Create structure
testnode = db.createHistoryNode(None, "test1", "test1", UCIS_HISTORYNODE_TEST)
du = db.createScope("work.top", None, 1, UCIS_VLOG, UCIS_DU_MODULE, flags)
inst = db.createInstance("top_inst", None, 1, UCIS_VLOG, UCIS_INSTANCE, du, UCIS_INST_ONCE)

# Create covergroup with options
cg = inst.createCovergroup("my_cg", SourceInfo(fh, 10, 0), 1, UCIS_VLOG)
cg.setPerInstance(True)
cg.setMergeInstances(False)
cg.setGoal(95)

# Create coverpoint with bins
cp = cg.createCoverpoint("my_cp", SourceInfo(fh, 20, 0), 10, UCIS_VLOG)
cp.setAtLeast(5)
cp.createBin("low", None, 5, 100, "[0:9]", UCIS_CVGBIN)
cp.createBin("high", None, 5, 50, "[10:19]", UCIS_CVGBIN)

# Save
db.close()

# Read back - all options preserved!
db2 = SqliteUCIS.read("coverage.db")
cg_read = list(db2.scopes(ScopeTypeT.INSTANCE))[0].scopes(ScopeTypeT.COVERGROUP)[0]
assert cg_read.getPerInstance() == True
assert cg_read.getGoal() == 95

References

UCIS LRM v1.0 (June 2012) - Section 7 (Functional Coverage)
Memory Backend Implementation: src/ucis/mem/
XML Backend Implementation: src/ucis/xml/
Current SQLite Backend: src/ucis/sqlite/
Test Suite: tests/unit/api/

FilesExpand file tree

SQLITE_FULL_API_PLAN.md

Latest commit

History

SQLITE_FULL_API_PLAN.md

File metadata and controls

SQLite Backend Full API Support Implementation Plan

✅ COMPLETED - 2026-02-11

Executive Summary

✅ Implementation Status (COMPLETED)

Current Gaps Analysis (RESOLVED)

1. Missing Covergroup API Methods

2. Missing Coverpoint Creation API

3. Missing Cross Coverage API

4. Goal Value Not Persisting

Database Schema Assessment

Current Schema (Good Foundation)

Schema Enhancements Needed

Implementation Plan

Phase 1: Specialized Scope Classes

1.1 Create SqliteCovergroup Class

1.2 Create SqliteCoverpoint Class

1.3 Update SqliteScope Factory Pattern

Phase 2: Schema Migration

2.1 Update schema_manager.py

2.2 Update createScope() to Include New Columns

Phase 3: Goal Persistence Fix

3.1 Debug Goal Handling

3.2 Fix Issues Found

Phase 4: Testing & Validation

4.1 Run Test Suite

4.2 Update Tests

4.3 Add Regression Tests

Phase 5: Documentation & Cleanup

5.1 Update Documentation

5.2 Performance Optimization

Implementation Timeline

Risk Assessment

Technical Risks

Non-Technical Risks

Success Criteria

Alternative Approaches Considered

Alternative 1: Keep SQLite Read-Only

Alternative 2: Use JSON Blobs for Options

Alternative 3: Separate Tables for Each Scope Type

Next Steps

Appendix: Code Examples

Example: Using Enhanced SQLite Backend

References