feat: implement snapshot-based persistence (closes #39)#69
Open
ithustle wants to merge 1 commit intoAlexJuca:mainfrom
Open
feat: implement snapshot-based persistence (closes #39)#69ithustle wants to merge 1 commit intoAlexJuca:mainfrom
ithustle wants to merge 1 commit intoAlexJuca:mainfrom
Conversation
Serialize the hashtable to disk periodically and on shutdown using an atomic write strategy (tmp + fsync + rename + fsync dir). On startup the snapshot is loaded back, discarding any expired keys. Enabled by default (every 5 min if >= 1 change). Configurable via save-interval and snapshot-path in server.conf, or disabled with save-interval 0 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
to a binary RDB file with CRC32 integrity checking
calls shutdown_server)
dirty >= threshold and enough ticks have elapsed
save-interval 300 1(5 min, 1 change minimum)save-interval,snapshot-pathin server.conf;disable with
save-interval 0 0Files added (5)
src/persistence/crc32.h— CRC32 lookup table and update functionsrc/persistence/persistence.h— save/load APIsrc/persistence/persistence.c— serialization, atomic write, CRC verificationsrc/fkvs_time.h— standalone fkvs_now_ms() to decouple persistence from utils.htests/test_persistence.c— 8 test cases (round-trip, encoding, TTL, corruption)Files modified (8)
src/server.h— new fields: save_interval, dirty, shutdown_requested, snapshot_pathsrc/config.c— defaults and parsing for save-interval, snapshot-pathsrc/server.c— shutdown_server(), load on startup, simplified SIGINT handlersrc/networking/networking.c— dirty counter for write commandssrc/io/event_dispatcher_{kqueue,epoll,io_uring}.c— shutdown check + periodic savesrc/utils.h— delegates fkvs_now_ms() to fkvs_time.hCMakeLists.txt— persistence.c in server targets, test_persistence targetserver.conf— documented save-interval and snapshot-path optionsRDB format
HEADER: [4B magic "FKVS"] [1B version] [8B timestamp BE]
STORE: [1B marker 0x01] [8B count BE] [entries: key_len|key|val_len|val|encoding]
EXPIRES: [1B marker 0x02] [8B count BE] [entries: key_len|key|deadline_ms BE]
FOOTER: [4B CRC32 BE] [1B EOF 0xFF]
Test plan
test_persistence— 8 unit tests covering round-trip, encoding, TTL,expired discard, empty DB, missing file, corrupted header, corrupted CRC
Ctrl+C, verify fkvs-dump.rdb created, restart and GET keys back
save-interval 0 0, verify no snapshot createdIssue reference: #39