🐰#21 SQLFluff, Data Orchestrators, Decoupling BIs; ThDPTh #21 🐰

Finally, a good SQL linter, how to choose a data orchestrator, and why decoupling makes the BI world better.

Data will power every piece of our existence in the near future. I collect “Data Points” to help understand & shape this future.

If you want to support this, please share it on Twitter, LinkedIn, or Facebook.

(1)🔥 SQLFluff, Finally a Good SQLLinter

Finally, there’s a SQL linter that actually might make the cut. Almost all SQL linters fail on two problems, the variety of SQL dialects, say hello to RedShift & Snowflake, and the ability to lint templated, usually Jinja, SQL.

SQLFluff at least got these two problems nailed and seems to be in a good shape. So if you haven’t found your SQL linter yet, I suggest you check out SQLFluff. They even got an online linter you can try out.

GitHub - sqlfluff/sqlfluff: A SQL linter and auto-formatter for Humans

A SQL linter and auto-formatter for Humans

github.com  •  Share

(2)🎁 Comparing Data Orchestrators

I find it pretty hard to navigate between all these data orchestrators/processing frameworks like airflow, prefect, dagster, and all the others. So I really enjoyed the article by Kapernikov which takes a step back, evaluates these tools, and above all provides a good framing around them.

Even though all of these frameworks already evolved quite a bit since the article was written last year, it still helped me take the bird’s eye view on all these tools.

Oh and to add just one thing: There’s really only one thing all these tools have in common, they build & manage DAGs. And then some.

A comparison of data processing frameworks – Kapernikov

Data, industry 4.0 & prototyping

kapernikov.com  •  Share

(3)😍 Decoupling In the BI World

I’ve already touched on this topic in my newsletter on building evolving data architectures, so I really enjoyed another company taking up the topic of decoupling in the BI world.

I really enjoyed the ideas the company GoodData shares with respect to bringing more software engineering to this world, so go take a look at the article.

Here is What Happens If You Decouple Your BI Stack | Medium

The field of Business Intelligence has significantly evolved over the last decades.

medium.com  •  Share

🎄 In Other News & Thanks

Thanks for reading this far! I’d also love it if you shared this newsletter with people whom you think might be interested in it.

P.S.: I share things that matter, not the most recent ones. I share books, research papers, and tools. I try to provide a simple way of understanding all these things. I tend to be opinionated. You can always hit the unsubscribe button!

By Sven Balnojan

Data; Business Intelligence; Machine Learning, Artificial Intelligence; Everything about what powers our future.

Tweet     Share

In order to unsubscribe, click here.

If you were forwarded this newsletter and you like it, you can subscribe here.

Powered by Revue