diff --git a/CHANGELOG.md b/CHANGELOG.md index de3bb80..8f0345e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,38 +4,61 @@ All notable changes to this project will be documented in this file. ## [1.3.4](https://github.com/NatLabRockies/plexosdb/compare/v1.3.3...v1.3.4) (2026-03-27) - ### 🧩 CI -* use release/v1 tag for pypa/gh-action-pypi-publish ([#107](https://github.com/NatLabRockies/plexosdb/issues/107)) ([c4e58b8](https://github.com/NatLabRockies/plexosdb/commit/c4e58b8dc062f3302216b3caa7c9c6c1cc423c86)) - +- use release/v1 tag for pypa/gh-action-pypi-publish + ([#107](https://github.com/NatLabRockies/plexosdb/issues/107)) + ([c4e58b8](https://github.com/NatLabRockies/plexosdb/commit/c4e58b8dc062f3302216b3caa7c9c6c1cc423c86)) ### 📦 Build -* **deps:** bump actions/cache from 5.0.3 to 5.0.4 ([#115](https://github.com/NatLabRockies/plexosdb/issues/115)) ([1ff162a](https://github.com/NatLabRockies/plexosdb/commit/1ff162afe66dfe3bcad7d0dbb0a534b4a9d3374a)) -* **deps:** bump astral-sh/setup-uv from 7.5.0 to 7.6.0 ([#117](https://github.com/NatLabRockies/plexosdb/issues/117)) ([13fb3cf](https://github.com/NatLabRockies/plexosdb/commit/13fb3cfb4c7a2affd7df504a2e152c9a2b0c1295)) -* **deps:** bump astral-sh/setup-uv from b75dde52aef63a238519e7aecbbe79a4a52e4315 to e06108dd0aef18192324c70427afc47652e63a82 ([#114](https://github.com/NatLabRockies/plexosdb/issues/114)) ([96f3975](https://github.com/NatLabRockies/plexosdb/commit/96f397540b06e68e45075a5d700d2b0a91ebe112)) -* **deps:** bump codecov/codecov-action from 5.5.2 to 5.5.3 ([#116](https://github.com/NatLabRockies/plexosdb/issues/116)) ([c86c8e2](https://github.com/NatLabRockies/plexosdb/commit/c86c8e254a85909043c8a7b25a10ff7d169d1e02)) -* **deps:** bump googleapis/release-please-action from c3fc4de07084f75a2b61a5b933069bda6edf3d5c to 16a9c90856f42705d54a6fda1823352bdc62cf38 ([#112](https://github.com/NatLabRockies/plexosdb/issues/112)) ([4150d56](https://github.com/NatLabRockies/plexosdb/commit/4150d56065f41cf916912daf9cda39281cf4e3df)) -* **deps:** bump peaceiris/actions-gh-pages from e9c66a37f080288a11235e32cbe2dc5fb3a679cc to 4f9cc6602d3f66b9c108549d475ec49e8ef4d45e ([#113](https://github.com/NatLabRockies/plexosdb/issues/113)) ([c697f1d](https://github.com/NatLabRockies/plexosdb/commit/c697f1d7642dac4c16cd5ea9e11e327d684ff548)) +- **deps:** bump actions/cache from 5.0.3 to 5.0.4 + ([#115](https://github.com/NatLabRockies/plexosdb/issues/115)) + ([1ff162a](https://github.com/NatLabRockies/plexosdb/commit/1ff162afe66dfe3bcad7d0dbb0a534b4a9d3374a)) +- **deps:** bump astral-sh/setup-uv from 7.5.0 to 7.6.0 + ([#117](https://github.com/NatLabRockies/plexosdb/issues/117)) + ([13fb3cf](https://github.com/NatLabRockies/plexosdb/commit/13fb3cfb4c7a2affd7df504a2e152c9a2b0c1295)) +- **deps:** bump astral-sh/setup-uv from + b75dde52aef63a238519e7aecbbe79a4a52e4315 to + e06108dd0aef18192324c70427afc47652e63a82 + ([#114](https://github.com/NatLabRockies/plexosdb/issues/114)) + ([96f3975](https://github.com/NatLabRockies/plexosdb/commit/96f397540b06e68e45075a5d700d2b0a91ebe112)) +- **deps:** bump codecov/codecov-action from 5.5.2 to 5.5.3 + ([#116](https://github.com/NatLabRockies/plexosdb/issues/116)) + ([c86c8e2](https://github.com/NatLabRockies/plexosdb/commit/c86c8e254a85909043c8a7b25a10ff7d169d1e02)) +- **deps:** bump googleapis/release-please-action from + c3fc4de07084f75a2b61a5b933069bda6edf3d5c to + 16a9c90856f42705d54a6fda1823352bdc62cf38 + ([#112](https://github.com/NatLabRockies/plexosdb/issues/112)) + ([4150d56](https://github.com/NatLabRockies/plexosdb/commit/4150d56065f41cf916912daf9cda39281cf4e3df)) +- **deps:** bump peaceiris/actions-gh-pages from + e9c66a37f080288a11235e32cbe2dc5fb3a679cc to + 4f9cc6602d3f66b9c108549d475ec49e8ef4d45e + ([#113](https://github.com/NatLabRockies/plexosdb/issues/113)) + ([c697f1d](https://github.com/NatLabRockies/plexosdb/commit/c697f1d7642dac4c16cd5ea9e11e327d684ff548)) ## [1.3.3](https://github.com/NatLabRockies/plexosdb/compare/v1.3.2...v1.3.3) (2026-03-16) - ### 🐛 Bug Fixes -* **ci:** harden all workflows per zizmor audit ([#105](https://github.com/NatLabRockies/plexosdb/issues/105)) ([67ca845](https://github.com/NatLabRockies/plexosdb/commit/67ca84584d1e66410dc66b014a9b710a24b00b95)) - +- **ci:** harden all workflows per zizmor audit + ([#105](https://github.com/NatLabRockies/plexosdb/issues/105)) + ([67ca845](https://github.com/NatLabRockies/plexosdb/commit/67ca84584d1e66410dc66b014a9b710a24b00b95)) ### ⚡ Performance -* Improving performance of adding memberships from records ([#104](https://github.com/NatLabRockies/plexosdb/issues/104)) ([1ea4a39](https://github.com/NatLabRockies/plexosdb/commit/1ea4a39612a1bef1a0f290eaeb40441874a2b8f0)) - +- Improving performance of adding memberships from records + ([#104](https://github.com/NatLabRockies/plexosdb/issues/104)) + ([1ea4a39](https://github.com/NatLabRockies/plexosdb/commit/1ea4a39612a1bef1a0f290eaeb40441874a2b8f0)) ### 📦 Build -* **deps:** bump actions/download-artifact from 7 to 8 ([#101](https://github.com/NatLabRockies/plexosdb/issues/101)) ([0e572a0](https://github.com/NatLabRockies/plexosdb/commit/0e572a07a930f6e25f196e98fc879f65f7dd9daa)) -* **deps:** bump actions/upload-artifact from 6 to 7 ([#102](https://github.com/NatLabRockies/plexosdb/issues/102)) ([22b8374](https://github.com/NatLabRockies/plexosdb/commit/22b8374aa7ed9d29eb36258a6a5ad16feb2e21c5)) +- **deps:** bump actions/download-artifact from 7 to 8 + ([#101](https://github.com/NatLabRockies/plexosdb/issues/101)) + ([0e572a0](https://github.com/NatLabRockies/plexosdb/commit/0e572a07a930f6e25f196e98fc879f65f7dd9daa)) +- **deps:** bump actions/upload-artifact from 6 to 7 + ([#102](https://github.com/NatLabRockies/plexosdb/issues/102)) + ([22b8374](https://github.com/NatLabRockies/plexosdb/commit/22b8374aa7ed9d29eb36258a6a5ad16feb2e21c5)) ## [1.3.2](https://github.com/NatLabRockies/plexosdb/compare/v1.3.1...v1.3.2) (2026-02-12) diff --git a/docs/source/howtos/add_attributes.md b/docs/source/howtos/add_attributes.md index d1481c7..d21dced 100644 --- a/docs/source/howtos/add_attributes.md +++ b/docs/source/howtos/add_attributes.md @@ -1,7 +1,7 @@ # Adding Attributes to the objects Objects in PlexosDB can have attributes that are saved on the `t_attribute_data` -table. +table and represent static metadata (e.g. "Step Type", "Chrono Step Count"). ## Listing available attributes per `ClassEnum` diff --git a/docs/source/howtos/bulk_operations.md b/docs/source/howtos/bulk_operations.md index 3ddb074..5cad05f 100644 --- a/docs/source/howtos/bulk_operations.md +++ b/docs/source/howtos/bulk_operations.md @@ -227,3 +227,24 @@ your model: This approach can dramatically improve performance when creating large, complex models. + +## Bulk Inserting Attributes + +For efficiently adding multiple attribute values, use +`add_attributes_from_records`. + +### Basic Usage (Wide Format) + +```python +from plexosdb.enums import ClassEnum + +records = [ + {"name": "Horizon1", "Step Type": 4.0, "Chrono Step Count": 366.0}, + {"name": "Horizon2", "Step Type": 4.0, "Chrono Step Count": 365.0}, +] + +db.add_attributes_from_records( + records, + object_class=ClassEnum.Horizon, +) +``` diff --git a/docs/source/howtos/copy_objects.md b/docs/source/howtos/copy_objects.md index 9a227bf..5f59ccb 100644 --- a/docs/source/howtos/copy_objects.md +++ b/docs/source/howtos/copy_objects.md @@ -1,11 +1,11 @@ # Copying Objects PlexosDB allows you to create copies of existing objects along with their -properties, memberships, and related property records. +properties, memberships, attributes and related property records. ## Basic Object Copying -Copy an object and all its properties: +Copy an object and all its memberships, properties and attributes: ```python from plexosdb import PlexosDB @@ -32,20 +32,26 @@ db.add_property( scenario="Base Case" ) -# Copy a generator with all its properties +# Copy a generator with its memberships, properties and attributes new_object_id = db.copy_object( object_class=ClassEnum.Generator, original_object_name="Generator1", new_object_name="Generator1_Copy", - copy_properties=True # Default is True + copy_properties=True, # Default is True + copy_attributes=True # Default is True ) print(f"Created new object with ID: {new_object_id}") ``` +```{note} +Object-level attributes (stored in `t_attribute_data`) are also copied when using `copy_object`. +``` + ## Copying Objects Without Properties -You can also copy the object and its memberships without copying properties: +You can also copy only the object and its memberships by disabling property and +attribute copying: ```python # Add Generator object @@ -65,7 +71,8 @@ new_object_id = db.copy_object( object_class=ClassEnum.Generator, original_object_name="Generator1", new_object_name="Generator1_Skeleton", - copy_properties=False + copy_properties=False, + copy_attributes=False ) ``` @@ -93,7 +100,8 @@ new_object_id = db.copy_object( object_class=ClassEnum.Node, original_object_name="Node1", new_object_name="Node1_Copy", - copy_properties=False + copy_properties=False, + copy_attributes=False ) # Check the memberships of the new object diff --git a/src/plexosdb/db.py b/src/plexosdb/db.py index 85a5a85..513920c 100644 --- a/src/plexosdb/db.py +++ b/src/plexosdb/db.py @@ -39,6 +39,7 @@ normalize_names, plan_property_inserts, resolve_membership_id, + _normalize_attribute_records, ) from .xml_handler import XMLHandler @@ -959,6 +960,140 @@ def add_properties_from_records( logger.debug(f"Successfully processed {len(records)} property and text records in batches") return + def add_attributes_from_records( + self, + records: list[dict[str, Any]], + /, + *, + object_class: ClassEnum, + chunksize: int = 10_000, + ) -> None: + """Bulk insert attribute values for objects. + + Efficiently adds multiple object-level attribute values in batches. + This method is much more efficient than calling `add_attribute` repeatedly. + + Each record should contain: + - 'name': object name + - 'attribute': attribute name + - 'value': attribute value + + Alternatively, records may be provided in the following format: + - {'name': ..., 'Attr1': val1, 'Attr2': val2} + + Optional: + - 'state' + + Parameters + ---------- + records : list[dict[str, Any]] + List of attribute records in explicit or wide format. + object_class : ClassEnum + Class of the objects. + chunksize : int, optional + Batch size for inserts, by default 10_000. + + Returns + ------- + None + + See Also + -------- + add_attribute + add_properties_from_records + add_memberships_from_records + + Examples + -------- + >>> records = [ + ... {"name": "2020", "Step Type": 4.0, "Chrono Step Count": 366.0}, + ... ] + + >>> db.add_attributes_from_records(records, object_class=ClassEnum.Horizon) + """ + if not records: + logger.warning("No records provided for bulk attribute insertion") + return + + records = _normalize_attribute_records(records) + + if chunksize < 1: + msg = f"chunksize must be >= 1, received {chunksize}" + raise ValueError(msg) + + class_id = self.get_class_id(object_class) + + object_names = tuple({record["name"] for record in records}) + name_to_object_id: dict[str, int] = {} + + CHUNK = 900 # noqa: N806 + for i in range(0, len(object_names), CHUNK): + chunk = object_names[i : i + CHUNK] + object_placeholders = ", ".join("?" for _ in chunk) + + object_rows = self._db.query( + f""" + SELECT object_id, name + FROM t_object + WHERE class_id = ? + AND name IN ({object_placeholders}) + """, + (class_id, *chunk), + ) + name_to_object_id.update({name: object_id for object_id, name in object_rows}) + + attribute_rows = self._db.query( + """ + SELECT attribute_id, name + FROM t_attribute + WHERE class_id = ? + """, + (class_id,), + ) + name_to_attribute_id = {name: attribute_id for attribute_id, name in attribute_rows} + + params: list[tuple[int, int, Any, Any]] = [] + seen: set[tuple[int, int]] = set() + + for record in records: + try: + object_id = name_to_object_id[record["name"]] + attribute_id = name_to_attribute_id[record["attribute"]] + except KeyError as exc: + raise KeyError(f"Invalid attribute record: {record}") from exc + + key = (object_id, attribute_id) + if key in seen: + raise ValueError( + f"Duplicate attribute record for object={record['name']!r}, " + f"attribute={record['attribute']!r}" + ) + seen.add(key) + + params.append( + ( + object_id, + attribute_id, + record["value"], + record.get("state"), + ) + ) + + query = f""" + INSERT INTO {Schema.AttributeData.name} + (object_id, attribute_id, value, state) + VALUES (?, ?, ?, ?) + """ + + with self._db.transaction(): + for batch in batched(params, chunksize): + result = self._db.executemany(query, list(batch)) + if not result: + msg = f"Failed to add attribute values for {object_class}." + raise RuntimeError(msg) + + logger.debug("Added {} attribute values.", len(params)) + def _handle_dates( self, data_id: int, @@ -1884,14 +2019,25 @@ def copy_object( original_object_name: str, new_object_name: str, copy_properties: bool = True, + copy_attributes: bool = True, ) -> int: - """Copy an object and its properties, tags, and texts.""" + """Copy an object and its memberships, attributes, properties, tags and texts.""" object_id = self.get_object_id(object_class, name=original_object_name) category_id = self.query("SELECT category_id from t_object WHERE object_id = ?", (object_id,)) category = self.query("SELECT name from t_category WHERE category_id = ?", (category_id[0][0],)) + new_object_id = self.add_object(object_class, new_object_name, category=category[0][0]) + + if copy_attributes: + self._copy_object_attributes( + old_object_id=object_id, + new_object_id=new_object_id, + ) + membership_mapping = self.copy_object_memberships( - object_class=object_class, original_name=original_object_name, new_name=new_object_name + object_class=object_class, + original_name=original_object_name, + new_name=new_object_name, ) system_collection = get_default_collection(object_class) @@ -2053,6 +2199,16 @@ def _copy_object_properties(self, membership_mapping: dict[int, int]) -> bool: self._db.execute("DROP TABLE IF EXISTS temp_data_mapping") return True + def _copy_object_attributes(self, old_object_id: int, new_object_id: int) -> bool: + """Copy attribute values from original object to new object.""" + query = """ + INSERT INTO t_attribute_data (object_id, attribute_id, value, state) + SELECT ?, attribute_id, value, state + FROM t_attribute_data + WHERE object_id = ? + """ + return self._db.execute(query, (new_object_id, old_object_id)) + def create_object_scenario( self, object_name: str, @@ -2116,7 +2272,31 @@ def delete_attribute( object_class: ClassEnum, ) -> None: """Delete an attribute from an object.""" - raise NotImplementedError # pragma: no cover + if not self.check_object_exists(object_class, object_name): + msg = f"Object = `{object_name}` does not exist for class `{object_class}`." + raise NotFoundError(msg) + + object_id = self.get_object_id(object_class, name=object_name) + attribute_id = self.get_attribute_id(object_class, name=attribute_name) + + find_query = """ + SELECT 1 + FROM t_attribute_data + WHERE object_id = ? AND attribute_id = ? + """ + result = self._db.fetchone(find_query, (object_id, attribute_id)) + + if not result: + msg = f"Attribute '{attribute_name}' not found for object '{object_name}'." + raise NotFoundError(msg) + + delete_query = """ + DELETE FROM t_attribute_data + WHERE object_id = ? AND attribute_id = ? + """ + + with self._db.transaction(): + self._db.execute(delete_query, (object_id, attribute_id)) def delete_category(self, category: str, /, *, class_name: ClassEnum) -> None: """Delete a category from the database.""" diff --git a/src/plexosdb/utils.py b/src/plexosdb/utils.py index 010235b..0215eff 100644 --- a/src/plexosdb/utils.py +++ b/src/plexosdb/utils.py @@ -266,6 +266,39 @@ def _flatten_property_records(records: list[dict[str, Any]]) -> tuple[list[dict[ return normalized_records, deprecated_format_used +def _normalize_attribute_records( + records: list[dict[str, Any]], +) -> list[dict[str, Any]]: + """Normalize wide or explicit attribute records into explicit records.""" + reserved_fields = {"name", "attribute", "value", "state"} + normalized: list[dict[str, Any]] = [] + + for record in records: + if "attribute" in record and "value" in record: + if "name" not in record: + raise KeyError(f"Attribute record is missing required 'name': {record}") + normalized.append(record) + continue + + if "name" not in record: + raise KeyError(f"Attribute record is missing required 'name': {record}") + + for key, value in record.items(): + if key in reserved_fields: + continue + + normalized.append( + { + "name": record["name"], + "attribute": key, + "value": value, + "state": record.get("state"), + } + ) + + return normalized + + def plan_property_inserts( db: PlexosDB, records: list[dict[str, Any]], diff --git a/tests/conftest.py b/tests/conftest.py index e1f4ef9..24af0ea 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -156,6 +156,23 @@ def db_instance_with_schema() -> PlexosDB: # type: ignore db._db.close() +@pytest.fixture(scope="function") +def db_with_model_attributes(db_instance_with_schema) -> PlexosDB: # type: ignore + """Create a schema-backed DB with Model attribute definitions.""" + db_instance_with_schema._db.executemany( + """ + INSERT INTO t_attribute (attribute_id, class_id, name) + VALUES (?, ?, ?) + """, + [ + (1001, 8, "Enabled"), + (1002, 8, "Random Number Seed"), + ], + ) + + yield db_instance_with_schema + + @pytest.fixture(scope="function") def db_manager_instance_empty_with_schema() -> Generator[SQLiteManager, None, None]: db: PlexosDB = PlexosDB() diff --git a/tests/test_plexosdb_attributes.py b/tests/test_plexosdb_attributes.py index bfa1e0f..2d3befb 100644 --- a/tests/test_plexosdb_attributes.py +++ b/tests/test_plexosdb_attributes.py @@ -2,6 +2,8 @@ from typing import TYPE_CHECKING +import pytest + if TYPE_CHECKING: from plexosdb.db import PlexosDB @@ -34,3 +36,88 @@ def test_list_attributes(db_base: PlexosDB): result = db.list_attributes(ClassEnum.Generator) assert result assert len(result) == 2 + + +def test_delete_attribute_removes_single_attribute(db_with_model_attributes: PlexosDB): + """Delete one attribute value without removing other attributes.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "Model1") + + db.add_attributes_from_records( + [{"name": "Model1", "Enabled": -1, "Random Number Seed": 1000}], + object_class=ClassEnum.Model, + ) + + db.delete_attribute("Enabled", object_name="Model1", object_class=ClassEnum.Model) + + rows = db._db.fetchall( + """ + SELECT attr.name, data.value + FROM t_attribute_data AS data + JOIN t_attribute AS attr ON attr.attribute_id = data.attribute_id + JOIN t_object AS obj ON obj.object_id = data.object_id + WHERE obj.name = ? + ORDER BY attr.name + """, + ("Model1",), + ) + + assert rows == [("Random Number Seed", 1000.0)] + + +def test_delete_attribute_does_not_affect_other_objects(db_with_model_attributes: PlexosDB): + """Delete an attribute from one object without changing another object.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "Model1") + db.add_object(ClassEnum.Model, "Model2") + + db.add_attributes_from_records( + [ + {"name": "Model1", "Enabled": -1}, + {"name": "Model2", "Enabled": -1}, + ], + object_class=ClassEnum.Model, + ) + + db.delete_attribute("Enabled", object_name="Model1", object_class=ClassEnum.Model) + + rows = db._db.fetchall( + """ + SELECT obj.name, attr.name, data.value + FROM t_attribute_data AS data + JOIN t_attribute AS attr ON attr.attribute_id = data.attribute_id + JOIN t_object AS obj ON obj.object_id = data.object_id + ORDER BY obj.name, attr.name + """ + ) + + assert rows == [("Model2", "Enabled", -1.0)] + + +def test_delete_attribute_fails_with_nonexistent_object(db_with_model_attributes: PlexosDB): + """Raise NotFoundError when deleting an attribute from a missing object.""" + from plexosdb import ClassEnum + from plexosdb.exceptions import NotFoundError + + db = db_with_model_attributes + + with pytest.raises(NotFoundError, match="Object = `MissingModel` does not exist"): + db.delete_attribute("Enabled", object_name="MissingModel", object_class=ClassEnum.Model) + + +def test_delete_attribute_fails_with_nonexistent_attribute_value( + db_with_model_attributes: PlexosDB, +): + """Raise NotFoundError when the object has no value for the attribute.""" + from plexosdb import ClassEnum + from plexosdb.exceptions import NotFoundError + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "Model1") + + with pytest.raises(NotFoundError, match="Attribute 'Enabled' not found for object 'Model1'"): + db.delete_attribute("Enabled", object_name="Model1", object_class=ClassEnum.Model) diff --git a/tests/test_plexosdb_copy_object.py b/tests/test_plexosdb_copy_object.py index d72a889..7c9e007 100644 --- a/tests/test_plexosdb_copy_object.py +++ b/tests/test_plexosdb_copy_object.py @@ -68,3 +68,81 @@ def test_copy_object_with_memberships(db_base: PlexosDB): new_child_membership = db.get_membership_id(new_object_name, child_object_name, collection) assert membership_id_child in membership_mapping assert membership_mapping[membership_id_child] == new_child_membership + + +def test_copy_object_copies_attributes(db_base: PlexosDB): + from plexosdb import ClassEnum + + db = db_base + object_class = ClassEnum.Generator + + original_object_name = "TestGenWithAttribute" + new_object_name = "TestGenWithAttributeCopy" + + original_object_id = db.add_object(object_class, original_object_name) + + db.add_attributes_from_records( + [ + {"name": original_object_name, "attribute": "Latitude", "value": 42.0}, + {"name": original_object_name, "attribute": "Longitude", "value": -105.0}, + ], + object_class=object_class, + ) + + new_object_id = db.copy_object(object_class, original_object_name, new_object_name) + + old_rows = db._db.fetchall( + """ + SELECT attr.name, data.value, data.state + FROM t_attribute_data AS data + JOIN t_attribute AS attr ON attr.attribute_id = data.attribute_id + WHERE data.object_id = ? + ORDER BY attr.name + """, + (original_object_id,), + ) + + new_rows = db._db.fetchall( + """ + SELECT attr.name, data.value, data.state + FROM t_attribute_data AS data + JOIN t_attribute AS attr ON attr.attribute_id = data.attribute_id + WHERE data.object_id = ? + ORDER BY attr.name + """, + (new_object_id,), + ) + + assert old_rows == [ + ("Latitude", 42.0, None), + ("Longitude", -105.0, None), + ] + assert new_rows == old_rows + + +def test_copy_object_without_attributes(db_base: PlexosDB): + from plexosdb import ClassEnum + + db = db_base + object_class = ClassEnum.Generator + + original_object_name = "TestGenNoAttr" + new_object_name = "TestGenNoAttrCopy" + + original_object_id = db.add_object(object_class, original_object_name) + new_object_id = db.copy_object(object_class, original_object_name, new_object_name) + + assert ( + db._db.fetchall( + "SELECT * FROM t_attribute_data WHERE object_id = ?", + (original_object_id,), + ) + == [] + ) + assert ( + db._db.fetchall( + "SELECT * FROM t_attribute_data WHERE object_id = ?", + (new_object_id,), + ) + == [] + ) diff --git a/tests/test_plexosdb_from_records.py b/tests/test_plexosdb_from_records.py index 51291e3..9f56398 100644 --- a/tests/test_plexosdb_from_records.py +++ b/tests/test_plexosdb_from_records.py @@ -375,3 +375,193 @@ def test_get_memberships_system_chunks_over_900_names(db_base: PlexosDB): result = db.get_memberships_system(names, object_class=ClassEnum.Generator) assert len(result) == 950 assert {r["name"] for r in result} == set(names) + + +def test_add_attributes_from_records_explicit_format(db_with_model_attributes: PlexosDB): + """Insert attribute records using explicit attribute/value format.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "AttrModel") + + records = [ + {"name": "AttrModel", "attribute": "Enabled", "value": -1}, + {"name": "AttrModel", "attribute": "Random Number Seed", "value": 1000}, + ] + + db.add_attributes_from_records(records, object_class=ClassEnum.Model) + + rows = db._db.fetchall( + """ + SELECT attr.name, data.value, data.state + FROM t_attribute_data AS data + JOIN t_attribute AS attr ON attr.attribute_id = data.attribute_id + JOIN t_object AS obj ON obj.object_id = data.object_id + WHERE obj.name = ? + ORDER BY attr.name + """, + ("AttrModel",), + ) + + assert rows == [ + ("Enabled", -1.0, None), + ("Random Number Seed", 1000.0, None), + ] + + +def test_add_attributes_from_records_wide_format(db_with_model_attributes: PlexosDB): + """Insert attribute records using wide-column attribute format.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "WideAttrModel") + + db.add_attributes_from_records( + [{"name": "WideAttrModel", "Enabled": -1, "Random Number Seed": 1000}], + object_class=ClassEnum.Model, + ) + + rows = db._db.fetchall( + """ + SELECT attr.name, data.value + FROM t_attribute_data AS data + JOIN t_attribute AS attr ON attr.attribute_id = data.attribute_id + JOIN t_object AS obj ON obj.object_id = data.object_id + WHERE obj.name = ? + ORDER BY attr.name + """, + ("WideAttrModel",), + ) + + assert rows == [ + ("Enabled", -1.0), + ("Random Number Seed", 1000.0), + ] + + +def test_add_attributes_from_records_rejects_duplicates(db_with_model_attributes: PlexosDB): + """Reject duplicate object/attribute pairs in the same batch.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "DuplicateAttrModel") + + records = [ + {"name": "DuplicateAttrModel", "Enabled": -1}, + {"name": "DuplicateAttrModel", "Enabled": 0}, + ] + + with pytest.raises(ValueError, match="Duplicate attribute record"): + db.add_attributes_from_records(records, object_class=ClassEnum.Model) + + assert db._db.fetchone("SELECT COUNT(*) FROM t_attribute_data")[0] == 0 + + +def test_add_attributes_from_records_unknown_attribute(db_with_model_attributes: PlexosDB): + """Raise an error when inserting attributes not defined for the class.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "BadAttrModel") + + with pytest.raises(KeyError, match="Invalid attribute record"): + db.add_attributes_from_records( + [{"name": "BadAttrModel", "Fake Attribute": 123}], + object_class=ClassEnum.Model, + ) + + assert db._db.fetchone("SELECT COUNT(*) FROM t_attribute_data")[0] == 0 + + +def test_add_attributes_from_records_respects_chunksize( + db_with_model_attributes: PlexosDB, + monkeypatch: pytest.MonkeyPatch, +): + """Split attribute inserts into batches according to chunksize.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + names = [f"ChunkAttrModel_{idx}" for idx in range(5)] + db.add_objects(ClassEnum.Model, *names) + + records = [{"name": name, "Enabled": -1} for name in names] + + observed_batch_sizes: list[int] = [] + original_executemany = db._db.executemany + + def spy_executemany(query, params_seq): + observed_batch_sizes.append(len(params_seq)) + return original_executemany(query, params_seq) + + monkeypatch.setattr(db._db, "executemany", spy_executemany) + db.add_attributes_from_records(records, object_class=ClassEnum.Model, chunksize=2) + + assert observed_batch_sizes == [2, 2, 1] + + +def test_add_attributes_from_records_rejects_non_positive_chunksize( + db_with_model_attributes: PlexosDB, +): + """Reject non-positive chunksize values.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "BadChunkAttrModel") + + with pytest.raises(ValueError, match="chunksize must be >= 1"): + db.add_attributes_from_records( + [{"name": "BadChunkAttrModel", "Enabled": -1}], + object_class=ClassEnum.Model, + chunksize=0, + ) + + +def test_add_attributes_from_records_no_records(db_instance_with_schema: PlexosDB, caplog): + """Gracefully handle empty attribute payloads.""" + from plexosdb import ClassEnum + + db = db_instance_with_schema + + db.add_attributes_from_records([], object_class=ClassEnum.Model) + + assert "No records provided" in caplog.text + assert db._db.fetchone("SELECT COUNT(*) FROM t_attribute_data")[0] == 0 + + +def test_add_attributes_from_records_missing_name_explicit_format( + db_instance_with_schema: PlexosDB, +): + """Require explicit-format attribute records to include object names.""" + from plexosdb import ClassEnum + + db = db_instance_with_schema + + with pytest.raises(KeyError, match="missing required 'name'"): + db.add_attributes_from_records( + [{"attribute": "Enabled", "value": -1}], + object_class=ClassEnum.Model, + ) + + +def test_add_attributes_from_records_raises_runtime_error_on_insert_failure( + db_with_model_attributes: PlexosDB, + monkeypatch: pytest.MonkeyPatch, +): + """Raise RuntimeError when bulk attribute insertion fails.""" + from plexosdb import ClassEnum + + db = db_with_model_attributes + db.add_object(ClassEnum.Model, "InsertFailAttrModel") + + def fail_executemany(query, params_seq): + return False + + monkeypatch.setattr(db._db, "executemany", fail_executemany) + + with pytest.raises(RuntimeError, match="Failed to add attribute values"): + db.add_attributes_from_records( + [{"name": "InsertFailAttrModel", "Enabled": -1}], + object_class=ClassEnum.Model, + ) + + assert db._db.fetchone("SELECT COUNT(*) FROM t_attribute_data")[0] == 0