feat: expand Data Fabric entities service [DS-8360]#1616
Conversation
github-actions
Bot
added
test:uipath-langchain
avichalsri24
changed the title
There was a problem hiding this comment.
Pull request overview
This PR expands the Data Fabric EntitiesService surface area by adding missing Data Fabric entity operations (single-record CRUD, structured query, attachments, schema management, bulk import) and by exposing more backend parameters/metadata on existing batch and list endpoints.
Changes:
- Added new Data Fabric APIs to
EntitiesService: single-record CRUD, structured query (V1/V2 routing), attachments, entity schema create/update/delete, and CSV bulk import (sync + async). - Extended existing record list/batch operations to support additional backend parameters and richer pagination metadata (
EntityRecordsListResponse), plus improved batch error recovery for per-record failures on HTTP 400. - Added/updated entity/query/schema models and comprehensive tests for the new behaviors.
Reviewed changes
Copilot reviewed 5 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/uipath/uv.lock | Bumps editable uipath-platform version reference. |
| packages/uipath-platform/uv.lock | Bumps uipath-platform version to 0.1.47. |
| packages/uipath-platform/pyproject.toml | Version bump to 0.1.47. |
| packages/uipath-platform/src/uipath/platform/entities/entities.py | Adds new response/query/schema models (batch failure shape, list response w/ metadata, structured query types). |
| packages/uipath-platform/src/uipath/platform/entities/_entities_service.py | Implements the new/extended EntitiesService methods and request/response helpers. |
| packages/uipath-platform/src/uipath/platform/entities/init.py | Exposes the newly added entities/query/schema symbols via package exports. |
| packages/uipath-platform/tests/services/test_entities_service.py | Adds test coverage for new methods, query routing, attachments, schema creation defaults, and validation/error recovery. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if isinstance(metadata, EntityMetadataUpdateOptions): | ||
| body = metadata.model_dump(by_alias=True, exclude_none=True) | ||
| else: | ||
| body = dict(metadata) | ||
| return RequestSpec( |
There was a problem hiding this comment.
dict inputs now route through EntityMetadataUpdateOptions.model_validate(...) then model_dump(by_alias=True, exclude_none=True), so display_name/is_rbac_enabled get serialized as displayName/isRbacEnabled on the wire. New test test_update_entity_metadata_normalizes_snake_case_dict_keys locks this in.
| "fields": [self._build_schema_field_payload(f) for f in fields], | ||
| "folderId": opts.folder_key or DATA_FABRIC_TENANT_FOLDER_ID, | ||
| "isRbacEnabled": bool(opts.is_rbac_enabled or False), | ||
| "isInsightsEnabled": bool(opts.is_analytics_enabled or False), |
There was a problem hiding this comment.
is_analytics_enabled → isInsightsEnabled — added a one-line comment at the payload site explaining the legacy wire-name divergence.
| def __getitem__(self, index: int) -> EntityRecord: | ||
| """Index records by position (delegates to ``self.items``).""" | ||
| return self.items[index] |
There was a problem hiding this comment.
added @overload for int and slice on EntityQueryRecordsResponse, so slicing returns List[EntityRecord] and indexing returns EntityRecord with proper type narrowing.
| ) -> EntityRecordsListResponse: | ||
| """Asynchronously list records from an entity with optional pagination and schema validation. | ||
|
|
||
| The schema parameter enables type-safe access to entity records by validating the |
There was a problem hiding this comment.
docstring now mirrors the sync version: Args list includes expansion_level, filter, orderby, select, expand; Returns describes EntityRecordsListResponse with pagination metadata; Examples include an OData filter/orderby/select/expand sample.
| FileContent = Union[bytes, bytearray, memoryview] | ||
| """Acceptable raw bytes types for attachment/CSV uploads.""" | ||
|
|
||
| _NAME_RE = re.compile(r"^[a-zA-Z]\w*$") |
There was a problem hiding this comment.
Is this consistent with backend?
There was a problem hiding this comment.
Verified against the UI's create-form validators. Tightened _NAME_RE to ^[a-zA-Z][a-zA-Z0-9]*$ (no underscores) and split lengths by context: entity 1–30, field 3–100. Anything accepted by the SDK can now round-trip through the UI.
| ) | ||
|
|
||
| return self.validate_entity_batch(response, schema) | ||
| async def _do() -> Response: |
| ), | ||
| ) | ||
|
|
||
| def _query_records_spec( |
There was a problem hiding this comment.
!!! You are swapping out the query_records_spec which is catering to FQS query endpoint from DataService. This breaks multiple APIs for agents. Right strategy to avoid this is:
- Isolate the changes for schema into EntitySchemaService
- Isolate the changes for data into EntityDataService.
- EntityService is a facade that doesnt change the external contract.
- Carefully understand the APIs for DS and FQS and then make these changes.
There was a problem hiding this comment.
As discussed, this is a new method for odata query and not impacting sql endpoint consumed by agents (query_entity_records_spec vs query_records_spec).
Regarding refactor, updated the code, split into EntitySchemaService (entity/choiceset CRUD) and EntityDataService (records, attachments, queries — both FQS query_entity_records and DS query). EntitiesService is now a thin facade that preserves the existing flat sdk.entities.* contract via explicit delegation. No external API change. The new DS structured-query method is named query (not query_records) so it can't be confused with the existing FQS query_entity_records, which is fully intact. Sub-services are kept internal (not exported in init.py).
🚨 Heads up:
|
Adds the missing Data Fabric methods on EntitiesService and exposes the
full backend parameter set on existing batch methods.
Internally splits the service into a facade pattern for clearer
separation between schema- and data-side operations while keeping the
public sdk.entities.* surface unchanged:
- EntitySchemaService (internal): entity / choiceset CRUD against
datafabric_/api/Entity.
- EntityDataService (internal): record CRUD (single + batch), structured
query, federated SQL query, attachments, bulk import, choiceset
values, against datafabric_/api/EntityService/... +
datafabric_/api/Attachment + datafabric_/api/v1/query/execute.
- EntitiesService (public): thin facade that delegates to the two
sub-services and owns cross-cutting concerns (agent entity-set
resolution).
New methods (sync + async on the facade):
- Single-record ops: insert_record, get_record, update_record,
delete_record — fire trigger events on each mutation.
- query — structured query with filter_group, sort_options,
selected_fields, expansions, expansion_level, aggregates, group_by,
joins, binnings, start, limit. Routes to V2 endpoint when binnings
is supplied.
- Attachments: upload_attachment (bytes or path), download_attachment,
delete_attachment.
- Schema: create_entity (with SQL-type mapping and per-type constraint
defaults), delete_entity, update_entity_metadata.
- import_records for CSV bulk upload.
Existing methods extended (ticket §B):
- insert_records / update_records / delete_records accept
expansion_level and fail_on_first.
- list_records accepts OData filter / orderby / select / expand /
expansion_level and returns EntityRecordsListResponse (a list
subclass with total_count / has_next_page / next_cursor).
Bug fixes (ticket §C):
- Batch operations recover per-record failures from HTTP 400 responses
that carry successRecords / failureRecords lists; other non-2xx
statuses propagate.
- Record input is normalized — accepts dicts, Pydantic models,
EntityRecord, or any object with __dict__.
Client-side validation in create_entity uses the UI's create-form rules:
^[a-zA-Z][a-zA-Z0-9]*$ (no underscores), entity 1-30 chars, field 3-100
chars, plus reserved-name and per-field constraint range checks. Anything
accepted by the SDK round-trips cleanly through the Data Service UI.
Backward compatibility: public method signatures only gained optional
kwargs. EntityRecord.id stays required. list_records return type
subclasses list, so iteration / indexing / len() / isinstance(records,
list) continue to work. Existing method docstrings preserved verbatim
from main; only ticket-mandated additions appear in their docs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
avichalsri24
force-pushed
the
task/sync_entities_sdk
branch
from
ff1d809 to
bcabb27
Compare
- Add 13 async-variant tests covering retrieve_async, retrieve_by_name (sync + async), list_entities_async, list_records_async, update_record_async, batch async (insert / update / delete), import_records_async, plus validate_entity_batch and 5xx-shape edge cases. - Convert remaining ``Union[A, B]`` annotations to ``A | B`` (PEP 604) across entities source files; drop now-unused ``Union`` imports. Coverage on new code: 87.6% -> ~95% across the four new files, clearing the SonarCloud 90% quality gate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@codex review |
|
To use Codex here, create a Codex account and connect to github. |
|
UIPath-Harshit
left a comment
UIPath-Harshit
left a comment
There was a problem hiding this comment.
Initial comments for data aspects.
|
|
||
| Backend target: ``datafabric_/api/EntityService/...`` plus | ||
| ``datafabric_/api/Attachment/...`` for file attachments, and | ||
| ``datafabric_/api/v1/query/execute`` for legacy SQL queries. |
There was a problem hiding this comment.
not legacy :) rather thats the new api that will be used in the future.
| # Structured query (POST /entity/{id}/query) | ||
| # ------------------------------------------------------------------ | ||
|
|
||
| def query( |
There was a problem hiding this comment.
lets mark this as retrieve_records to ensure that query endpoint is tied back to sql
| binnings: Optional[List[EntityBinning]] = None, | ||
| start: Optional[int] = None, | ||
| limit: Optional[int] = None, | ||
| ) -> EntityQueryRecordsResponse: |
There was a problem hiding this comment.
RetrieveEntityRecordsResponse.
| return self._parse_query_response(response, start=start, limit=limit) | ||
|
|
||
| # ------------------------------------------------------------------ | ||
| # Federated SQL query (legacy escape hatch) |
There was a problem hiding this comment.
Why are we calling this legacy ?
| with self._open_file(file, file_path) as handle: | ||
| response = self.request( | ||
| "POST", | ||
| Endpoint( |
| return response.json().get("results", []) | ||
|
|
||
| @staticmethod | ||
| def _list_records_spec( |
There was a problem hiding this comment.
Can we make the params pydantic using builder method?
| return params | ||
|
|
||
| @staticmethod | ||
| def _query_spec( |
There was a problem hiding this comment.
Same here. Can we pass pydantic models here also
| is_encrypted: Optional[bool] = Field(default=None, alias="isEncrypted") | ||
| default_value: Optional[str] = Field(default=None, alias="defaultValue") | ||
| length_limit: Optional[int] = Field(default=None, alias="lengthLimit") | ||
| max_value: Optional[float] = Field(default=None, alias="maxValue") |
There was a problem hiding this comment.
EntityCreateFieldOptions.max_value / min_value are typed as float for every field kind, and _validate_field_constraints() only checks range. That means EntityCreateFieldOptions(type=INTEGER, max_value=3.5) passes client-side validation and gets serialized as 3.5 for an integer column. Integer-backed field types should reject fractional bounds locally, or use a narrower type for those cases.
| is_encrypted: Optional[bool] = Field(default=None, alias="isEncrypted") | ||
| default_value: Optional[str] = Field(default=None, alias="defaultValue") | ||
| length_limit: Optional[int] = Field(default=None, alias="lengthLimit") | ||
| max_value: Optional[float] = Field(default=None, alias="maxValue") |
There was a problem hiding this comment.
create_entity() now allows DECIMAL/FLOAT/DOUBLE fields to send fractional maxValue / minValue, but the retrieve-side FieldDataType.max_value / min_value model is still Optional[int]. If the backend returns something like 3.14 for one of those constraints, retrieve() / list_entities() will reject or coerce it incorrectly. The response model needs to accept non-integer numeric bounds if the request model can create them.
| try: | ||
| items.append(EntityRecord.from_data(data=raw)) | ||
| except ValueError: | ||
| items.append(EntityRecord.model_construct(_fields_set=set(raw), **raw)) |
There was a problem hiding this comment.
This makes aggregate/binning rows look like EntityRecords even though they may not have an Id. The parser gets the request through, but the declared return type is now lying: result.items is typed as List[EntityRecord], while row.id can still raise AttributeError on these constructed rows. It would be safer to widen the item type here or introduce a separate response model for aggregate/binning queries instead of fabricating invalid EntityRecord instances.
| max_value: Optional[float] = Field(default=None, alias="maxValue") | ||
| min_value: Optional[float] = Field(default=None, alias="minValue") | ||
| decimal_precision: Optional[int] = Field(default=None, alias="decimalPrecision") | ||
| choice_set_id: Optional[str] = Field(default=None, alias="choiceSetId") |
There was a problem hiding this comment.
Type-dependent required fields are not enforced here. create_entity() currently accepts a RELATIONSHIP field without referenceEntityName / referenceFieldName, and CHOICE_SET_SINGLE / CHOICE_SET_MULTIPLE fields without choiceSetId, then serializes those incomplete payloads to the backend. This needs validation in the new schema surface so callers get a local error instead of sending an invalid field definition.
|
|
||
| field_name: str = Field(alias="fieldName") | ||
| operator: QueryFilterOperator | ||
| value: Optional[str] = None |
There was a problem hiding this comment.
EntityQueryFilter currently accepts any combination of value, valueList, or neither, regardless of operator. That means malformed shapes like In/NotIn with only value, Equals with only valueList, or filters with no operand at all all validate locally and are serialized. The structured-query model should enforce scalar-vs-list operands by operator so invalid queries are rejected before the network call.
Summary
Adds the missing Data Fabric methods on
EntitiesServiceand exposes the full backend parameter set on existing batch methods. Internally splits the service for clearer separation between schema- and data-side operations while keeping the publicsdk.entities.*surface unchanged.Jira: https://uipath.atlassian.net/browse/DS-8360
Internal structure
EntitiesServiceis now a thin facade that delegates to two internal sub-services:EntitySchemaService(in_entity_schema_service.py, not exported)datafabric_/api/EntityEntityDataService(in_entity_data_service.py, not exported)datafabric_/api/EntityService/...+Attachment+v1/query/executeSub-services are deliberately not exported via
__init__.py—sdk.entities.*remains the only sanctioned entry point.EntityDataService.queryis the new structured-query method (namedqueryso it can't be confused with the existing FQSquery_entity_records, which is fully intact).New methods (sync + async on the facade)
insert_record,get_record,update_record,delete_record— fire trigger events on each mutation.query— structured query withfilter_group,sort_options,selected_fields,expansions,expansion_level,aggregates,group_by,joins,binnings,start,limit. Routes to V2 endpoint whenbinningsis supplied.upload_attachment(bytes or path),download_attachment,delete_attachment.create_entity(with full SQL-type mapping and per-type constraint defaults),delete_entity,update_entity_metadata.import_recordsfor CSV bulk upload.Existing methods extended
insert_records/update_records/delete_recordsacceptexpansion_levelandfail_on_first.list_recordsaccepts ODatafilter/orderby/select/expand/expansion_leveland returnsEntityRecordsListResponse(alistsubclass withtotal_count/has_next_page/next_cursor).Bug fixes
successRecords/failureRecordslists; other non-2xx statuses propagate.EntityRecord, or any object with__dict__.Client-side validation
create_entityvalidates entity / field names client-side using the UI's create-form rules:^[a-zA-Z][a-zA-Z0-9]*$(no underscores), entity 1–30 chars, field 3–100 chars, plus reserved-name and per-field constraint range checks. Anything accepted by the SDK round-trips cleanly through the Data Service UI.Backward compatibility
Public method signatures only gained optional kwargs.
EntityRecord.idstays required.list_recordsreturn type subclasseslist, so iteration / indexing /len()/isinstance(records, list)continue to work. Existing docstrings are preserved verbatim frommain; only ticket-mandated additions (new param entries, new Examples, new return types) appear in their docs.Test plan
pytest packages/uipath-platform/tests/services/test_entities_service.py— 112 passpytest packages/uipath-platform/tests/— 1158 passpytest packages/uipath/tests/— 1840 passruff check,ruff format --check,mypy src tests, customlint_httpx_client.py— clean across both packages🤖 Generated with Claude Code