Updated concurrency

EntityFrameworkCore.Sqlite.Concurrency/EFCore.Sqlite.Concurrency.csproj

@@ -11,7 +11,7 @@
 
         <!-- Core Package Identity -->
         <PackageId>EntityFrameworkCore.Sqlite.Concurrency</PackageId>
-        <Version>10.0.2</Version>
+        <Version>10.0.3</Version>
 
         <!-- Core Metadata for Trust & Recognition -->
         <Authors>Mike Gotfryd</Authors>
@@ -34,29 +34,39 @@
         <!-- 3. STRUCTURED RELEASE NOTES -->
         <PackageReleaseNotes>
             <![CDATA[
-🚀 **v10.0.2 - Initial Stable Release: Production-Ready SQLite Concurrency & Performance**
-
-This first major release transforms SQLite into a robust database for concurrent .NET applications by fixing core limitations of the standard provider.
-
-**✅ SOLVES: Concurrency & Locking Errors**
-• **Eliminates `SQLITE_BUSY` / "database is locked" errors** with automatic, application-level write serialization.
-• **Guarantees 100% write reliability** under any multi-threaded load.
-
-**⚡ DELIVERS: Exceptional Performance**
-• **Achieves up to 10x faster bulk inserts** vs. standard `SaveChanges()` through intelligent batching.
-• **Enables true parallel read scaling** with non-blocking connections.
-• **Optimizes all interactions** (connections, transactions, WAL mode) for maximum throughput.
-
-**🧩 PROVIDES: Seamless Developer Experience**
-• **Drop-in replacement** – change `UseSqlite()` to `UseSqliteWithConcurrency()`.
-• **Full EF Core compatibility** – all existing DbContexts, models, and LINQ queries work unchanged.
-• **Simplifies complex logic** – abstracts retry patterns, lock management, and connection pooling.
-
-**🏗️ ENSURES: Enterprise-Grade Robustness**
-• Built-in production resilience with exponential backoff retry and crash-safe transactions.
-• Targets the modern .NET ecosystem with first-class support for **.NET 10** and **Entity Framework Core 10**.
-
-Get started in one line. Stop compromising on SQLite reliability and speed.
+v10.0.3 — SQLITE_BUSY_SNAPSHOT fix, IDbContextFactory support, structured logging
+
+BUGS FIXED
+• SQLITE_BUSY_SNAPSHOT (extended code 517) now correctly restarts the full operation lambda
+  instead of retrying the same statement — the only correct fix for a stale WAL read snapshot.
+• Exponential backoff now uses full jitter ([baseDelay, 2×baseDelay]) to prevent thundering
+  herd when multiple threads contend simultaneously.
+• Cache=Shared in the connection string now throws ArgumentException at startup — it silently
+  broke WAL mode semantics in prior versions.
+• Invalid SqliteConcurrencyOptions values (MaxRetryAttempts ≤ 0, negative BusyTimeout, etc.)
+  now throw ArgumentOutOfRangeException at startup instead of silently misbehaving.
+
+NEW FEATURES
+• AddConcurrentSqliteDbContextFactory<T> — registers IDbContextFactory<T> with all concurrency
+  settings. Use this for Task.WhenAll, background services, Channel<T> consumers, and any
+  workload that creates concurrent database operations. DbContext is not thread-safe; the factory
+  pattern gives each concurrent flow its own independent instance.
+• Structured logging: pass ILoggerFactory (or let DI resolve it) to get Warning logs for
+  SQLITE_BUSY/SQLITE_BUSY_SNAPSHOT events, Error logs for SQLITE_LOCKED, and Debug logs for
+  BEGIN IMMEDIATE upgrades — all through your existing logging pipeline.
+• GetWalCheckpointStatusAsync — runs PRAGMA wal_checkpoint(PASSIVE) and returns a typed
+  WalCheckpointStatus with IsBusy, TotalWalFrames, CheckpointedFrames, and CheckpointProgress.
+  Call periodically to detect long-running readers blocking WAL reclamation before it degrades
+  read performance.
+• TryReleaseMigrationLockAsync — detects and optionally clears a stale __EFMigrationsLock
+  row left behind by a crashed migration process. Prevents indefinite blocking on Database.Migrate()
+  in multi-instance deployments.
+• SynchronousMode option — configures PRAGMA synchronous (Off / Normal / Full / Extra).
+  Default remains Normal (recommended for WAL: safe after app crash, fast writes).
+• UpgradeTransactionsToImmediate option — opt out of the BEGIN → BEGIN IMMEDIATE rewrite
+  if you manage write transactions explicitly yourself. Default remains true.
+
+NO BREAKING CHANGES — all existing call sites compile and behave correctly without modification.
 ]]>
         </PackageReleaseNotes>
         <!-- =============================================== -->
@@ -93,9 +103,9 @@ Get started in one line. Stop compromising on SQLite reliability and speed.
         <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="10.0.2" />
         <PackageReference Include="Microsoft.Data.Sqlite" Version="10.0.2" />
         <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.2" />
-        <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="10.0.2" />
+        <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="10.0.5" />
         <!-- SourceLink for debugging support -->
-        <PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0" PrivateAssets="All" />
+        <PackageReference Include="Microsoft.SourceLink.GitHub" Version="10.0.201" PrivateAssets="All" />
     </ItemGroup>
 
     <!-- Optional Dependencies (Conditional) -->

EntityFrameworkCore.Sqlite.Concurrency/doc/QUICKSTART.md

@@ -34,30 +34,62 @@ public class BlogDbContext : DbContext
 }
 ```
 
-### 2. Configure with One Line of Code
+### 2. Configure in Program.cs
 
-In your `Program.cs` or startup configuration:
+#### Single-threaded / request-scoped use (ASP.NET Core controllers, Razor Pages, Blazor Server)
+
+One context is created per HTTP request through the DI scope. ASP.NET Core processes requests one thread at a time per scope, so sharing a context here is safe.
 
 ```csharp
-// Simple configuration
-builder.Services.AddDbContext<BlogDbContext>(options =>
-    options.UseSqliteWithConcurrency("Data Source=blog.db"));
+builder.Services.AddConcurrentSqliteDbContext<BlogDbContext>("Data Source=blog.db");
 ```
 
 Or with custom options:
 
 ```csharp
-builder.Services.AddDbContext<BlogDbContext>(options =>
-    options.UseSqliteWithConcurrency(
-        "Data Source=blog.db",
-        sqliteOptions =>
+builder.Services.AddConcurrentSqliteDbContext<BlogDbContext>(
+    "Data Source=blog.db",
+    options =>
+    {
+        options.BusyTimeout = TimeSpan.FromSeconds(30);
+        options.MaxRetryAttempts = 5;
+    });
+```
+
+#### Concurrent use (background workers, Task.WhenAll, channels, hosted services)
+
+A `DbContext` is **not thread-safe** — it must not be shared across concurrent operations. Use `IDbContextFactory<T>` instead. Each concurrent flow calls `CreateDbContext()` to get its own independent instance.
+
+```csharp
+builder.Services.AddConcurrentSqliteDbContextFactory<BlogDbContext>("Data Source=blog.db");
+```
+
+Then inject and use the factory:
+
+```csharp
+public class PostImportService
+{
+    private readonly IDbContextFactory<BlogDbContext> _factory;
+
+    public PostImportService(IDbContextFactory<BlogDbContext> factory)
+        => _factory = factory;
+
+    public async Task ImportPostsAsync(IEnumerable<Post> posts, CancellationToken ct)
+    {
+        var tasks = posts.Select(async post =>
         {
-            sqliteOptions.UseWriteQueue = true;     // Enable write serialization
-            sqliteOptions.BusyTimeout = TimeSpan.FromSeconds(30);
-            sqliteOptions.MaxRetryAttempts = 5;
-        }));
+            await using var db = _factory.CreateDbContext();
+            db.Posts.Add(post);
+            await db.SaveChangesAsync(ct);
+        });
+
+        await Task.WhenAll(tasks); // ✅ Each task has its own context — no EF thread-safety violation
+    }
+}
 ```
 
+> **Note:** `Cache=Shared` in the connection string is incompatible with WAL mode and will throw an `ArgumentException` at startup. Use the default connection string format (`Data Source=blog.db`) — connection pooling is enabled automatically.
+
 ## Basic Usage Examples
 
 ### Writing Data (Automatically Thread-Safe)
@@ -168,45 +200,60 @@ public class ImportService
 Imagine a scenario where multiple background workers are processing tasks:
 
 ```csharp
-// WITHOUT ThreadSafeEFCore.SQLite - This would fail with "database is locked"
+// ❌ WRONG — sharing one DbContext across concurrent tasks
+//    EF Core will throw InvalidOperationException about concurrent usage,
+//    and SQLite returns "database is locked" for simultaneous writers.
 public class TaskProcessor
 {
+    private readonly AppDbContext _context; // shared — unsafe for concurrent use
+
     public async Task ProcessTasksConcurrently()
     {
         var tasks = Enumerable.Range(1, 10)
             .Select(i => ProcessSingleTaskAsync(i));
-
-        await Task.WhenAll(tasks); // 💥 Database locked errors!
+
+        await Task.WhenAll(tasks); // 💥 EF thread-safety violation + database locked
+    }
+
+    private async Task ProcessSingleTaskAsync(int taskId)
+    {
+        _context.TaskResults.Add(new TaskResult { TaskId = taskId });
+        await _context.SaveChangesAsync(); // 💥 concurrent SaveChanges on one context
     }
 }
 
-// WITH ThreadSafeEFCore.SQLite - Just works!
+// ✅ CORRECT — one context per concurrent flow via IDbContextFactory
+//    Register with: builder.Services.AddConcurrentSqliteDbContextFactory<AppDbContext>("Data Source=app.db");
 public class TaskProcessor
 {
-    private readonly AppDbContext _context;
-
+    private readonly IDbContextFactory<AppDbContext> _factory;
+
+    public TaskProcessor(IDbContextFactory<AppDbContext> factory)
+        => _factory = factory;
+
     public async Task ProcessTasksConcurrently()
     {
         var tasks = Enumerable.Range(1, 10)
             .Select(i => ProcessSingleTaskAsync(i));
-            
+
         await Task.WhenAll(tasks); // ✅ All tasks complete successfully
     }
-    
+
     private async Task ProcessSingleTaskAsync(int taskId)
     {
-        // Each task writes to the database
         var result = await PerformWorkAsync(taskId);
-
-        // The package automatically queues these writes
-        _context.TaskResults.Add(new TaskResult
+
+        // Each concurrent flow creates and disposes its own context.
+        // ThreadSafeEFCore.SQLite serializes the actual writes at the SQLite level.
+        await using var db = _factory.CreateDbContext();
+        db.TaskResults.Add(new TaskResult
         {
             TaskId = taskId,
             Result = result,
             CompletedAt = DateTime.UtcNow
         });
-        
-        await _context.SaveChangesAsync();
+
+        await db.SaveChangesAsync(); // ✅ Thread-safe — no shared context, writes queued automatically
     }
 }
 ```
@@ -216,8 +263,7 @@ public class TaskProcessor
 ```csharp
 // Create contexts manually when needed
 var dbContext = ThreadSafeFactory.CreateContext<BlogDbContext>(
-    "Data Source=blog.db",
-    options => options.UseWriteQueue = true);
+    "Data Source=blog.db");
 
 // Use it
 await dbContext.Posts.AddAsync(new Post { Title = "Hello World" });
@@ -254,19 +300,47 @@ public async Task UpdatePostWithRetryAsync(int postId, string newContent)
 
 | Option | Default | Description |
 |--------|---------|-------------|
-| `UseWriteQueue` | `true` | Automatically queue write operations |
-| `BusyTimeout` | 30 seconds | How long to wait if database is busy |
-| `MaxRetryAttempts` | 3 | Number of retries for busy errors |
-| `CommandTimeout` | 300 seconds | SQL command timeout |
-| `EnableWalCheckpointManagement` | `true` | Automatically manage WAL checkpoints |
+| `BusyTimeout` | 30 seconds | Per-connection `PRAGMA busy_timeout`. First layer of busy handling; SQLite retries lock acquisition internally for up to this duration. |
+| `MaxRetryAttempts` | 3 | Application-level retry attempts for `SQLITE_BUSY*` errors, with exponential backoff and jitter. |
+| `CommandTimeout` | 300 seconds | EF Core SQL command timeout in seconds. |
+| `WalAutoCheckpoint` | 1000 pages | WAL auto-checkpoint interval (`PRAGMA wal_autocheckpoint`). Each page is 4 096 bytes by default (~4 MB). Set to `0` to disable. |
+| `SynchronousMode` | `Normal` | Durability vs. performance trade-off (`PRAGMA synchronous`). `Normal` is recommended for WAL mode: safe against application crashes; a power loss or OS crash may roll back the last commit(s) not yet checkpointed. Use `Full` or `Extra` for stronger durability guarantees. |
+| `UpgradeTransactionsToImmediate` | `true` | Rewrites `BEGIN`/`BEGIN TRANSACTION` to `BEGIN IMMEDIATE` to prevent `SQLITE_BUSY_SNAPSHOT` mid-transaction. Disable only if you manage write transactions explicitly yourself. |
+
+## Multi-Instance Deployments and Migration Locks
+
+EF Core uses a `__EFMigrationsLock` table to serialize concurrent migrations. If a migration process crashes after acquiring the lock but before releasing it, subsequent calls to `Database.Migrate()` will block indefinitely.
+
+**Recommended approach:** run migrations once as a controlled startup step rather than calling `Database.Migrate()` from every app instance simultaneously.
+
+If a stale lock does occur, use the built-in helper to detect and clear it:
+
+```csharp
+// In your startup or migration runner:
+using var db = factory.CreateDbContext();
+var connection = db.Database.GetDbConnection();
+await connection.OpenAsync();
+
+var wasStale = await SqliteConnectionEnhancer.TryReleaseMigrationLockAsync(connection);
+if (wasStale)
+    logger.LogWarning("Stale EF migration lock found and released. Proceeding with migration.");
+
+await db.Database.MigrateAsync();
+```
+
+Pass `release: false` to check for a stale lock without removing it (useful for diagnostics).
+
+> **Network filesystem warning:** SQLite WAL mode requires all connections to be on the **same physical host**. Do not point the database at an NFS, SMB, or other network-mounted path. If your app runs across multiple machines or containers, use a client/server database instead.
 
 ## Best Practices
 
-1. **Use Dependency Injection** when possible for automatic context management
-2. **Keep write transactions short** - queue your data and write quickly
-3. **Use `BulkInsertOptimizedAsync`** for importing large amounts of data
-4. **Enable WAL mode** (already done by default) for better concurrency
-5. **Monitor performance** with the built-in diagnostics when needed
+1. **Use `IDbContextFactory<T>` for concurrent workloads** — inject the factory and call `CreateDbContext()` per concurrent operation; never share a single `DbContext` instance across concurrent tasks
+2. **Use `AddConcurrentSqliteDbContext<T>` for request-scoped workloads** — standard ASP.NET Core controllers and Razor Pages where one request = one thread = one context
+3. **Keep write transactions short** — acquire the write slot, write, commit; long-held write transactions block all other writers
+4. **Use `BulkInsertOptimizedAsync`** for importing large amounts of data
+5. **WAL mode is enabled automatically** — do not add `Cache=Shared` to the connection string; it is incompatible with WAL
+6. **Run migrations from a single process** — avoid calling `Database.Migrate()` concurrently from multiple instances; use `TryReleaseMigrationLockAsync` if a stale lock occurs
+7. **Stay on local disk** — WAL mode does not work over network filesystems (NFS, SMB); use a client/server database for multi-host deployments
 
 ## What Makes It Different
 

EntityFrameworkCore.Sqlite.Concurrency/doc/v10_0_0.md

@@ -19,7 +19,7 @@ options.UseSqlite("Data Source=app.db");
 // With this:
 options.UseSqliteWithConcurrency("Data Source=app.db");
 ```
-Guaranteed 100% write reliability and up to 10x faster bulk operations.
+Eliminates write contention errors and provides up to 10x faster bulk operations.
 
 ---
 
@@ -70,7 +70,7 @@ Next, explore high-performance bulk inserts or fine-tune the configuration.
 | **Mixed Read/Write Workload** | ~15.3 seconds | ~3.8 seconds | **4.0x faster** |
 | **Memory Usage (100k operations)** | ~425 MB | ~285 MB | **33% less memory** |
 
-*Benchmark environment: .NET 10, Windows 11, Intel i7-13700K, 32GB RAM*
+**Benchmark environment:** .NET 10, Windows 11, Intel i7-13700K, 32GB RAM*
 
 ---
 
@@ -82,13 +82,7 @@ public async Task PerformDataMigrationAsync(List<LegacyData> legacyRecords)
 {
     var modernRecords = legacyRecords.Select(ConvertToModernFormat);
 
-    await _context.BulkInsertSafeAsync(modernRecords, new BulkConfig
-    {
-        BatchSize = 5000,
-        PreserveInsertOrder = true,
-        EnableStreaming = true,
-        UseOptimalTransactionSize = true
-    });
+    await _context.BulkInsertSafeAsyncmodernRecords);
 }
 ```
 
@@ -114,8 +108,7 @@ public async Task<TResult> ExecuteHighPerformanceOperationAsync<TResult>(
     Func<DbContext, Task<TResult>> operation)
 {
     using var context = ThreadSafeFactory.CreateContext<AppDbContext>(
-        "Data Source=app.db",
-        options => options.EnablePerformanceOptimizations = true);
+        "Data Source=app.db");
 
     return await context.ExecuteWithRetryAsync(operation, maxRetries: 2);
 }
@@ -136,7 +129,6 @@ services.AddDbContext<AppDbContext>(options =>
             concurrencyOptions.BusyTimeout = TimeSpan.FromSeconds(30);
             concurrencyOptions.MaxRetryAttempts = 3;          // Performance-focused retry logic
             concurrencyOptions.CommandTimeout = 180;          // 3-minute timeout for large operations
-            concurrencyOptions.EnablePerformanceOptimizations = true; // Additional speed boosts
         }));
 ```