Add SQLite JSON querying guide #292
Conversation
|
this is great! 🔥 One suggestion I have is to link to this reference from other places in the docs where this use case would come up, e.g. two that come to mind are https://docs.powersync.com/usage/use-case-examples/custom-types-arrays-and-json (maybe even consolidate these 2 pages as there seems to be a lot of overlap), and where we mention JSON on https://docs.powersync.com/usage/sync-rules/types |
…ate text column reference and add links for further information on JSON handling and database type mapping.
michaelbarnes
left a comment
michaelbarnes
left a comment
There was a problem hiding this comment.
Overall, this is a great addition. I've left a few comments around wording and a few links back to SQLite docs for easy reference.
… links to relevant SQLite functions for better user guidance.
michaelbarnes
left a comment
michaelbarnes
left a comment
There was a problem hiding this comment.
A few small changes to be consistent.
…ON functions, corrected example references to 'tasks', and added a link to the SQLite JSON functions for enhanced user guidance.
|
|
||
| Your backend database might store structured data in various ways: | ||
| - **MongoDB**: Documents with nested documents and arrays | ||
| - **PostgreSQL**: JSONB or JSON column types |
There was a problem hiding this comment.
Some additional Postgres types are also represented as JSON, notably arrays and custom types.
| - `->` returns JSON (preserves type information, quotes strings) | ||
| - `->>` returns TEXT (unquoted, for final values) |
There was a problem hiding this comment.
Covered in the examples below, but this is not TEXT for numbers / booleans.
| assignee.value -> '$.hours_allocated' AS hours | ||
| FROM tasks t, | ||
| json_each(t.assignees) AS assignee | ||
| WHERE assignee.value ->> '$.role' = 'developer'; |
There was a problem hiding this comment.
Maybe worth making this WHERE (assignee.value ->> '$.role') = 'developer' to make the order of operations clear (same in other examples below).
| ``` | ||
|
|
||
| <Info> | ||
| **Note the CAST**: JSON numbers are extracted as text. When doing numeric comparisons or sorting, cast to the appropriate type. |
There was a problem hiding this comment.
Why not just use ->> here, instead of the CAST?
| ```sql | ||
| SELECT | ||
| assignee.value ->> '$.role' AS role, | ||
| SUM(CAST(assignee.value -> '$.hours_allocated' AS INTEGER)) AS total_hours, |
There was a problem hiding this comment.
Similarly, use ->> here instead of the CAST
| t.title, | ||
| assignee.value ->> '$.user_id' AS user_id, | ||
| assignee.value ->> '$.role' AS role, | ||
| CAST(assignee.value -> '$.hours_allocated' AS INTEGER) AS hours |
There was a problem hiding this comment.
Another one that should be ->>.
| -- ✅ GOOD - Filter by title first, then expand JSON | ||
| SELECT t.*, assignee.value | ||
| FROM tasks t, | ||
| json_each(t.assignees) AS assignee | ||
| WHERE t.title LIKE '%homepage%' | ||
| AND assignee.value ->> '$.role' = 'developer'; | ||
|
|
||
| -- ❌ LESS EFFICIENT - Expands all rows first | ||
| SELECT t.*, assignee.value | ||
| FROM tasks t, | ||
| json_each(t.assignees) AS assignee | ||
| WHERE assignee.value ->> '$.role' = 'developer' | ||
| AND t.title LIKE '%homepage%'; | ||
| ``` |
There was a problem hiding this comment.
Are you sure these two are different? the SQLite query optimizer is pretty good at re-ordering filters to efficiently execute the query.
| -- vs joining which creates all row combinations | ||
| ``` | ||
|
|
||
| 5. **Cache extracted values in CTEs**: Extract once, use multiple times: |
There was a problem hiding this comment.
What is being used multiple times in this example? It doesn't look like the CTE would make any difference in performance here.
| 2. **Type mismatches**: JSON numbers are returned as text with `->`. Always cast for numeric operations: | ||
| ```sql | ||
| -- ❌ String comparison (wrong!) | ||
| WHERE metadata -> '$.priority' > 5 | ||
|
|
||
| -- ✅ Numeric comparison (correct!) | ||
| WHERE CAST(metadata -> '$.priority' AS INTEGER) > 5 | ||
| ``` |
There was a problem hiding this comment.
Same here - should just use ->>
Added comprehensive guide for querying JSON data in the client SQLite database
->vs->>operators with clear examples showing quoted vs unquoted outputjson_extract()andjson_each()for flattening and querying nested arrays/objectsinstr()pattern