2.0.0b1 - Beta Release for Spark Connect and a new DuckDB engine

PyDeequ 2.0.0b1 - Beta Release

We're excited to announce the first beta release of PyDeequ 2.0, a major architectural overhaul that introduces Spark Connect support!

What's New

• PyDeequ 2.0 moves away from the Py4J-based JVM bridge to a modern client-server architecture using Spark Connect (introduced in Spark 3.4+). See #254 for the code changes.
• Multi-engine support with DuckDB backend. This is #255.

Call for Testing

This is a beta release intended for getting early feedback and is NOT production ready.

• Please follow the testing setup for local testing.

• End-to-end examples:

  • https://github.com/awslabs/python-deequ/blob/v2_engine/tutorials/data_quality_example_duckdb.py
  • https://github.com/awslabs/python-deequ/blob/v2_engine/tutorials/data_quality_example_v2.py

• Please create issues using the v2_beta label.