DuckDB has the ability to query S3 Tables, when this was first announced in beta I was unable to get it to work and subsequently gave up. A little over a year later I decided to give this a go again and now that it’s generally available, to my surprise, it just works. There’s still guides on how to set this up with nightly releases and other feature flags but nothing that shows the generally available path, so here’s my steps to get started.
Read more...
S3
I’ve been exploring AWS analytics services again so it’s time for another TIL. This post goes over accessing and querying S3 tables from QuickSight, via Athena, as S3 tables don’t natively work with QuickSight.
Here’s the TL;DR on what you need to do:
Create your S3 Tables bucket, namespace and table. Ensure that your S3 Tables bucket has AWS analytics service integration enabled. Grant your QuickSight user permission to the S3 table via LakeFormation and your QuickSight service role permisisons through IAM. Add Athena as a data source in QuickSight. Add a dataset from the previously created data source and use custom SQL to query your S3 Tables. Verify you can use your table dataset from an analysis. Those are a lot of steps and here are some AWS resources that explain them:
Read more...