Version 0.4.7

Features

New Connectors

• Native Delta Connector: Added native support for reading Delta Lake tables.
• Microsoft Fabric KQL Connector: Introduced a new Spark-based connector for executing KQL queries on Microsoft Fabric.

Connector Enhancements

• CSV Schema Support: Ability to define a strict schema for the CSV connector, supporting int, float, string, bool, and datetime types.

Compare Engine

• Join Comparison Mode: Added support for data comparison via joins using specific keys (join_keys) in Spark mode, providing an alternative to the ORDER mode.
• Numeric Tolerance: Improved handling of numeric comparisons in Spark by applying a tolerance threshold.

Exporters

• Executed Action Tracking: Exporters now include the specific action executed (e.g., file path or KQL query) for each test case.
• Execution ID: Added execution_id to the export process to improve the traceability of test runs.

Bug Fixes

• Column Name Sanitization: Column names are now automatically trimmed before comparison to prevent failures caused by leading or trailing whitespace.

Technical Enhancements

• Dry Run Workflow: Implemented a "dry run" step in GitHub Actions to validate the package and its execution before publishing.
• Linting & Quality: Enhanced the codebase maintainability by improving Pylint scores and updating configurations.
• Unit Testing: Expanded unit test coverage, particularly for new connectors and comparison engine logic.