Skip to content

feat: Lazy DAO fields#2713

Open
obabichevjb wants to merge 1 commit intomainfrom
obabichev/exposed-969-lazy-entity-field
Open

feat: Lazy DAO fields#2713
obabichevjb wants to merge 1 commit intomainfrom
obabichev/exposed-969-lazy-entity-field

Conversation

@obabichevjb
Copy link
Copy Markdown
Collaborator

Description

Summary of the change: Added lazy loading support for DAO entity fields, allowing developers to mark large or rarely-accessed columns (e.g., BLOB, TEXT) as lazy-loaded to improve query performance and reduce memory usage.

  • What:

    • Added a new LazyEntityField class that implements property delegation for lazy-loaded columns
    • Added the .lazy() extension function to mark entity properties as lazy-loaded
    • Implemented a dedicated cache (lazyFieldCache) in the Entity class to store lazy field values
  • How:

    • LazyEntityField.kt: New class implementing getValue()/setValue() operators for lazy property delegation. Maintains a global registry of lazy columns per table. On first access, executes a targeted SELECT query for that specific column and caches the result.
    • Entity.kt: Added lazyFieldCache HashMap to store lazy field values independently from readValues. Modified storeWrittenValues() to cache lazy fields during flush, preventing unnecessary database queries. Added refresh() support to clear lazy field cache.

Type of Change

Please mark the relevant options with an "X":

  • New feature
  • Documentation update

Affected databases:

  • All

Related Issues

Can DAO API support conditional retrieve some columns?

* Maps each table to a set of its lazy columns.
* This allows EntityClass to efficiently filter out lazy columns for its specific table.
*/
internal val lazyColumns = mutableMapOf<Table, MutableSet<Column<*>>>()
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In terms of API I didn't want to extend DSL layer, because it doesn't look like DSL need lazy column, so I wanted the method like lazy() exists only in Entities, but in this case it's a bit complicated to track which fields are lazy and which are not, because columns in DAO are delegates.

So I stopped at the variant of creating global cache for lazy columns assuming that in general it's not a big set. Every column is registered in the moment of call lazy().

Initially I hoped that in the moment of call lazy() the column could be stored inside entity, but at that moment it's not initialized yet.

Copy link
Copy Markdown
Collaborator

@vnikolova vnikolova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs look good, thanks for the addition! 👍
Suggested some small edits below.

<chapter id="lazy-fields" title="Lazy fields">
<p>
By default, all entity properties are loaded immediately when an entity is retrieved from the database.
However, for large columns like BLOBs or TEXT fields, or for rarely accessed data, this can negatively
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
However, for large columns like BLOBs or TEXT fields, or for rarely accessed data, this can negatively
However, for large columns like BLOB or TEXT fields, or for rarely accessed data, this behavior can negatively

<p>
Exposed provides the
<a href="%BASE_API_URL%/exposed-dao/org.jetbrains.exposed.dao/-entity/lazy.html"><code>.lazy()</code></a>
modifier to mark columns as lazy-loaded. Lazy fields are:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
modifier to mark columns as lazy-loaded. Lazy fields are:
modifier, which allows you to mark columns as lazy-loaded. Lazy fields are:

modifier to mark columns as lazy-loaded. Lazy fields are:
</p>
<list>
<li>Excluded from the default entity query</li>
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<li>Excluded from the default entity query</li>
<li>Excluded from the default entity query.</li>

</p>
<list>
<li>Excluded from the default entity query</li>
<li>Loaded on-demand when first accessed</li>
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<li>Loaded on-demand when first accessed</li>
<li>Loaded on-demand when first accessed.</li>

<list>
<li>Excluded from the default entity query</li>
<li>Loaded on-demand when first accessed</li>
<li>Cached after the first access to avoid redundant queries</li>
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<li>Cached after the first access to avoid redundant queries</li>
<li>Cached after the first access to avoid redundant queries.</li>

*
* On first access:
* 1. Checks for active transaction
* 2. Flushes entity if it has pending writes
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* 2. Flushes entity if it has pending writes
* 2. Flushes the entity if it has pending writes.

* On first access:
* 1. Checks for active transaction
* 2. Flushes entity if it has pending writes
* 3. Loads this lazy column value from the database
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* 3. Loads this lazy column value from the database
* 3. Loads the lazy column value from the database.

* 1. Checks for active transaction
* 2. Flushes entity if it has pending writes
* 3. Loads this lazy column value from the database
* 4. Caches the lazy value for subsequent accesses
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* 4. Caches the lazy value for subsequent accesses
* 4. Caches the lazy column value for subsequent accesses.

/**
* Sets the value of this lazy field for the given entity.
*
* The value is stored in the entity's write values and will be persisted on flush.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* The value is stored in the entity's write values and will be persisted on flush.
* The value is stored in the entity's write values and is persisted on flush.

import org.jetbrains.exposed.v1.core.Column

/**
* An exception that provides information about a lazy [column] that was accessed
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* An exception that provides information about a lazy [column] that was accessed
* An exception that provides information about a lazy [column] accessed

Copy link
Copy Markdown
Member

@e5l e5l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants