REDB backend

[danielabrozzoni]

Summer of Bitcoin Project Proposal

Description

Wallets built with the bdk_wallet library currently have two implementations of the WalletPersister trait: sqlite and file_store. It would be useful to add an alternative storage back-end implementation using redb, a pure-Rust key-value store.

Expected Outcomes

• Create a new bdk_redb crate similar to the bdk_file_store crate.
• Add an implementation of the WalletPersister trait for the new bdk_redb data store.
• Add tests in bdk_redb and bdk_wallet to confirm data is accurately stored and retrieved.
• Add a new bdk_wallet feature called redb that enables the the bdk_redb implementation of WalletPersister.
• Create an example similar to example_wallet_esplora_blocking that demonstrates using bdk_redb instead of file_store.

Resources

• bdk_wallet
• bdk_file_store
• rusqlite_impl.rs
• redb
• WalletPersister Trait
• example_wallet_esplora_blocking
• mempool.space version of electrs

Skills Required

• Experience with git. Guide
• Experience with rust. First seven chapters of the book
• Able to setup a local core node in regtest and testnet mode.
• Familiarity with database technologies like SQL and SQLite.

Mentor(s)
@notmandatory

Difficulty
Hard

Competency Test

• Install rust, compile and run all bdk examples and tests.
• Setup a local Bitcoin Core node daemon in regtest mode.
• Setup a local regtest esplora API server using the mempool.space version of electrs.
• Make a wallet with example_wallet_esplora_blocking example wallet and receive and send regtest bitcoin.

Original Description

Discussed in #691

^{Originally posted by casey July 30, 2022}
redb is a pure-Rust key-value store, and would be a good candidate for an alternative back-end. It is secure against data loss in some cases which sled probably isn't. It is still alpha quality, and the on-disk format has not been finalized, so it might not be quite ready yet, but could be good to watch for the future.

I mention this because I had issues building a binary on a Linux box due to the SQLite backend, and I don't personally believe that sled is mature enough to be used for production workloads, so another option would be nice.

[hrik2001]

Hi I am riksucks#3292 from discord, you might remember me recently texting in the #introductions channel. I am eager to take up this issue after grokking the codebase for a while. Seems like it's something that I can take up with my novice rust skills.

Here is how I would go forward implementing this feature

• Make redb.rs at bdk/src/database and write needed functions there
• Update impl_inner_method macro at bdk/src/database/any.rs to include attribute for redb

PS: if you do assign me, I might be a bit slow because of examinations around the corner at my uni, but apart from that, very eager to take this up. Thanks :)

[danielabrozzoni]

Hey @hrik2001, thanks for your interest! Sadly, at the moment we're heavily refactoring the bdk's architecture, and we think that at the end of the refactor we won't really need databases anymore - or, if we do, the code for handling them will be radically different. You can read more about the refactoring here: https://bitcoindevkit.org/blog/road-to-bdk-1/

For this reason, I can't really let you work on this issue at the moment, as I'm afraid it might be wasted work. However, I'm sending you a DM on discord so that we can discuss small issues you could work on :)

[110CodingP]

This might help close this issue.

[notmandatory]

I added this to the bdk_wallet 3.0 milestone to remind me to to talk to @110CodingP and the team about bringing bdk_redb into the bitcoindevkit org. :-)