Another day another odd error, my colleague and I were alerted to an issue on one of our MWAA Airflow 2.5.1 environments.
|
1 |
ImportError: cannot import name 'is_s3express_bucket' |
The above error was happening when a new DAG tried to instantiate S3 via botocore/boto3. We traced it down to the S3transfer package. Apparently in this commit:
https://github.com/boto/s3transfer/commit/3b50c31bb608188cdfb0fc7fd8e8cd03b6b7b187
Support was added for S3express, and from what we gathered we needed to upgrade 3 packages in our constraints.txt to solve the import issue.
|
1 2 3 4 5 6 7 8 9 10 |
========================================= --constraint "/usr/local/airflow/dags/constraints.txt" airflow==2.5.1 s3transfer==0.10.3 boto3==1.35.52 botocore==1.35.52 ................................... # Other existing packages ========================================= |
After making these package version changes we were back in business.