DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Bad data costs real money. Here is the HubSpot to Supabase workflow that fixed it.

Bad data costs real money. Here is the HubSpot to Supabase workflow that fixed it.

Comments
4 min read
Cosmos Semantic: an open-source semantic layer that makes AI agents get the number right, and prove it

Cosmos Semantic: an open-source semantic layer that makes AI agents get the number right, and prove it

Comments
6 min read
I Built SchemaCourt at the Flip the Switch Hackathon

I Built SchemaCourt at the Flip the Switch Hackathon

Comments
2 min read
Moving Big Data from AWS to GCP: Redshift, EMR and Glue

Moving Big Data from AWS to GCP: Redshift, EMR and Glue

Comments
21 min read
Power BI on Databricks Metric Views: a live connection, with the measures kept in Databricks

Power BI on Databricks Metric Views: a live connection, with the measures kept in Databricks

Comments
12 min read
Building a Near Real-Time Stock Analytics Pipeline on AWS (and the Silent Bug That Hid My Data)

Building a Near Real-Time Stock Analytics Pipeline on AWS (and the Silent Bug That Hid My Data)

Comments
9 min read
NVIDIA's auto-ontology: A Semantic Layer for Better AI Context

NVIDIA's auto-ontology: A Semantic Layer for Better AI Context

Comments
3 min read
Why Access Control Cannot Enforce Purpose Limitation in Data Pipelines

Why Access Control Cannot Enforce Purpose Limitation in Data Pipelines

Comments
4 min read
Microsoft Fabric: How to Prove the Business Case Before You Scale

Microsoft Fabric: How to Prove the Business Case Before You Scale

Comments
8 min read
NIH RePORTER codes Quebec as two different states, and asking for the valid one returns 2 institutions out of 21

NIH RePORTER codes Quebec as two different states, and asking for the valid one returns 2 institutions out of 21

Comments
6 min read
Track SEC comment letters for any ticker in 5 minutes (EDGAR UPLOAD/CORRESP, no API key)

Track SEC comment letters for any ticker in 5 minutes (EDGAR UPLOAD/CORRESP, no API key)

1
Comments
2 min read
How I Extracted Data from 783 PDFs with 97% Verified Accuracy (A Data Engineering Case Study)

How I Extracted Data from 783 PDFs with 97% Verified Accuracy (A Data Engineering Case Study)

1
Comments
2 min read
Building a Hadoop + Spark Data Warehouse with a Text-to-SQL AI Assistant

Building a Hadoop + Spark Data Warehouse with a Text-to-SQL AI Assistant

1
Comments
2 min read
Designing Airflow DAGs with Retries, SLAs, and Idempotency

Designing Airflow DAGs with Retries, SLAs, and Idempotency

1
Comments
7 min read
From YAML validator to event-driven projects: 14 releases of Hydra ETL in six weeks

From YAML validator to event-driven projects: 14 releases of Hydra ETL in six weeks

Comments
4 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.