Hacker News

JULY 4, 2023

The second section will delve more deeply into the various approaches that can be used to handle recursive schema definitions. The following Go Arrow schema definition provides an example of such a schema, instrumented with a collection of annotations. The depth of this definition cannot be predetermined.

Data Modeling 123

Data Modeling Data Models Data Observability 123

Alation

JANUARY 20, 2022

This starts by determining the critical data elements for the enterprise. These items become in scope for the data quality program. Step 2: Data Definitions. Here each critical data element is described so there are no inconsistencies between users or data stakeholders. Step 4: Data Sources.

Data Quality 94

Data Quality Data Governance Data Profiling Clean Data 94

Precisely

NOVEMBER 13, 2024

The Importance of a System of Record MDM’s role in your data landscape is closely tied to the concept of a system of record: a centralized repository where critical business data is stored and managed. It can also link with most commonly-used systems like your CRM, ERP, and marketing platforms.

Data Silos 64

Data Silos Data Observability Data Governance Data Quality 64

Alation

MAY 24, 2022

This has created many different data quality tools and offerings in the market today and we’re thrilled to see the innovation. People will need high-quality data to trust information and make decisions. A business glossary is critical to aligning an organization around the definition of business terms.

Data Quality 98

Data Quality Data Governance ETL Data Observability 98

The MLOps Blog

AUGUST 3, 2023

And then, we’re trying to boot out features of the platform and the open-source to be able to take Hamilton data flow definitions and help you auto-generate the Airflow tasks. To a junior data scientist, it doesn’t matter if you’re using Airflow, Prefect , Dexter. I term it as a feature definition store.

ML 52

ML ML Data Scientist Machine Learning 52

Pickl AI

JULY 8, 2024

These pipelines automate collecting, transforming, and delivering data, crucial for informed decision-making and operational efficiency across industries. Robust validation and monitoring frameworks enhance pipeline reliability and trustworthiness, safeguarding against data-driven decision-making risks.

Data Pipeline 52

Data Pipeline Data Quality Database Apache Kafka 52

Alation

FEBRUARY 8, 2022

Signals around the quality and integrity of the data are essential if people are to understand and trust it. Data provenance and lineage, for example, clarify an asset’s origin and past usages, important details for a newcomer to understand and trust that asset. Readers may notice these attributes echo other data management frameworks.

Data Governance 52

Data Governance Data Engineering Data Engineering Data Engineering 52