Data Quality with Airflow

Airflow
1

Hello all,
From which version of Airflow can I use SQLColumnCheckOperator and SQLTableCheckOperator?
Is there also SQLQueryCheckOperator or something like that I can execute a query and then check it result (empty or not/ the value itself equal to X)?

2

These operators appear to exist since at least 1.10.15

Consider using the SQL Check Operator and changing your SQL to return a boolean value for the check, e.g.

SELECT my_column = 'x' FROM my_table WHERE ...;
3

I am not sure how it can help me.
As far as I know the SQL Column Check Operator checks if a column of a table matches a certain pattern.
Can I execute a query and checks over the result if a pattern matches? If so, How?

4

Hi @tzachi_next!

I encourage you to watch a quick 15-min deep dive into how to use SQLColumnCheckOperator.

There are a couple of templated checks available (like null check) and a few qualifiers (like equal_to) that should allow you run the checks you described.