ODSC - Open Data Science

MARCH 20, 2025

30% Off ODSC East, Fan-Favorite Speakers, Foundation Models for Times Series, and ETL Pipeline Orchestration The ODSC East 2025 Schedule isLIVE! 15 Fan-Favorite Speakers & Instructors Returning for ODSC East2025 Over the years, weve had hundreds of speakers present at ODSC events. Register by Friday for 30%off.

ETL 52

ETL Data Science Machine Learning Machine Learning 52

Hacker News

JULY 18, 2024

ABOUT EVENTUAL Eventual is a data platform that helps data scientists and engineers build data applications across ETL, analytics and ML/AI. OUR PRODUCT IS OPEN-SOURCE AND USED AT ENTERPRISE SCALE Our distributed data engine Daft [link] is open-sourced and runs on 800k CPU cores daily.

ML 126

ML ML Python ETL 126

Smart Data Collective

MAY 20, 2019

We’re well past the point of realization that big data and advanced analytics solutions are valuable — just about everyone knows this by now. The entire process is also achieved much faster, boosting not just general efficiency but an organization’s reaction time to certain events, as well.

Analytics 111

Analytics Analytics Data Analyst Machine Learning 111

Smart Data Collective

AUGUST 17, 2022

Kafka And ETL Processing: You might be using Apache Kafka for high-performance data pipelines, stream various analytics data, or run company critical assets using Kafka, but did you know that you can also use Kafka clusters to move data between multiple systems. A three-step ETL framework job should do the trick. Conclusion.

Apache Kafka 130

Apache Kafka ETL Data Lakes AWS 130

Data Science Blog

JUNE 27, 2023

Big Data Analytics stands apart from conventional data processing in its fundamental nature. In this representation, there is a separate store for events within the speed layer and another store for data loaded during batch processing. This architectural concept relies on event streaming as the core element of data delivery.

Big Data 130

Big Data Big Data Apache Kafka Database 130

Dataconomy

JULY 29, 2024

Hosted at one of Mindspace’s coworking locations, the event was a convergence of insightful talks and professional networking. Mindspace , a global coworking and flexible office provider with over 45 locations worldwide, including 13 in Germany, offered a conducive environment for this knowledge-sharing event.

Apache Kafka 91

Apache Kafka Data Pipeline Data Warehouse ETL 91

Dataconomy

MAY 16, 2025

Define behavioral events, latency targets, and compliance guardrails up front. Keep immutable raw events and a queryready warehouse or lakehouse side by side. Common pitfalls and how to avoid them Tomlein highlights five recurring traps: Data leakage Partition feature calcs strictly by event time. Schemafirst design.

AI 103

AI AI ETL ML 103

Smart Data Collective

SEPTEMBER 19, 2022

billion on financial analytics by 2030. Fintech analytics helps businesses in the financial and banking industry offer satisfactory services by: Enhancing View Of Customer Profiling. Data analytics fintech provides crucial information financial institutions need to build a robust risk assessment strategy.

Big Data 145

Big Data Big Data Big Data Analytics Big Data Analytics 145

AWS Machine Learning Blog

MARCH 1, 2023

This post is co-written with Suhyoung Kim, General Manager at KakaoGames Data Analytics Lab. The result of these events can be evaluated afterwards so that they make better decisions in the future. With this proactive approach, Kakao Games can launch the right events at the right time. However, this approach is reactive.

AWS 102

AWS ML ML ETL 102

Smart Data Collective

JULY 20, 2021

Whether it is closing more sales deals, getting leads, offering vital customer services, marketing automation, analytics or application development, Salesforce CRM provides a bucket of comprehensive solutions. This tool is designed to connect various data sources, enterprise applications and perform analytics and ETL processes.

Big Data 142

Big Data Big Data ETL Analytics 142

Data Science Dojo

SEPTEMBER 6, 2023

This proactive approach helps maintain optimal system performance, ensuring users execute analytical queries efficiently and deliver insights without delay. In case of security breaches or data anomalies, auditing logs provide a trail of events that led to the incident.

Data Warehouse 195

Data Warehouse Data Governance Data Quality ETL 195

AWS Machine Learning Blog

JANUARY 10, 2024

ML Engineer at Tiger Analytics. EventBridge monitors status change events to automatically take actions with simple rules. The EventBridge model registration event rule invokes a Lambda function that constructs an email with a link to approve or reject the registered model. This post is co-written with Jayadeep Pabbisetty, Sr.

ML 133

ML ML AWS Machine Learning 133

AWS Machine Learning Blog

FEBRUARY 21, 2025

If the question was Whats the schedule for AWS events in December?, AWS usually announces the dates for their upcoming # re:Invent event around 6-9 months in advance. Previously, Karam developed big-data analytics applications and SOX compliance solutions for Amazons Fintech and Merchant Technologies divisions.

AWS 128

AWS Natural Language Processing Machine Learning Machine Learning 128

Flipboard

NOVEMBER 27, 2024

Gi Kim is a Data & ML Engineer with the AWS Professional Services team, helping customers build data analytics solutions and AI/ML applications. View the execution status and details of the workflow by fetching the state machine Amazon Resource Name (ARN) from the CloudFormation stack.

AWS 149

AWS Database ML ML 149

phData

MAY 3, 2024

Leveraging real-time analytics to make informed decisions is the golden standard for virtually every business that collects data. If you have the Snowflake Data Cloud (or are considering migrating to Snowflake ), you’re a blog away from taking a step closer to real-time analytics. Why Pursue Real-Time Analytics for Your Organization?

Apache Kafka 52

Apache Kafka Analytics Analytics ETL 52

AWS Machine Learning Blog

FEBRUARY 21, 2025

Thats why we use advanced technology and data analytics to streamline every step of the homeownership experience, from application to closing. Data refinement: Raw data is refined into consumable layers (raw, processed, conformed, and analytical) using a combination of AWS Glue extract, transform, and load (ETL) jobs and EMR jobs.

Data Science 83

Data Science AWS Hadoop Data Scientist 83

Smart Data Collective

APRIL 29, 2020

Extraction, Transform, Load (ETL). Data analytics and visualisation. This involves the processing of selecting data from data warehouses, data analytics and presentation in dashboards and visualisations. Redshift is the product for data warehousing, and Athena provides SQL data analytics. Data transformation.

Data Warehouse 98

Data Warehouse SQL Azure ETL 98

phData

FEBRUARY 3, 2023

With Snowpipe’s feature of automated data loading, it also leverages event notification for the purpose of cloud storage. Automated Snowpipe utilizes the event notifications for determining the time of arrival of the new files in the cloud storage that is being monitored. Snowpipe enables copying these files into a long queue.

ETL 59

ETL Data Warehouse Analytics Analytics 59

Data Science Dojo

JULY 3, 2024

Data Engineering : Building and maintaining data pipelines, ETL (Extract, Transform, Load) processes, and data warehousing. Career Support Some bootcamps include job placement services like resume assistance, mock interviews, networking events, and partnerships with employers to aid in job placement.

Data Science 150

Data Science Machine Learning Machine Learning Data Visualization 150

Data Science Blog

JUNE 21, 2023

What makes the difference is a smart ETL design capturing the nature of process mining data. By utilizing these services, organizations can store large volumes of event data without incurring substantial expenses. Depending the organization situation and data strategy, on premises or hybrid approaches should be also considered.

Big Data 130

Big Data Big Data Data Engineer Data Engineering 130

AWS Machine Learning Blog

JANUARY 17, 2024

It can represent a geographical area as a whole or it can represent an event associated with a geographical area. To obtain such insights, the incoming raw data goes through an extract, transform, and load (ETL) process to identify activities or engagements from the continuous stream of device location pings.

Clustering 136

Clustering AWS ML ML 136

ODSC - Open Data Science

APRIL 4, 2024

Data Analytics in the Age of AI, When to Use RAG, Examples of Data Visualization with D3 and Vega, and ODSC East Selling Out Soon Data Analytics in the Age of AI Let’s explore the multifaceted ways in which AI is revolutionizing data analytics, making it more accessible, efficient, and insightful than ever before.

Data Visualization 52

Data Visualization Analytics Analytics Big Data Analytics 52

IBM Journey to AI blog

AUGUST 29, 2024

Event-driven businesses across all industries thrive on real-time data, enabling companies to act on events as they happen rather than after the fact. This is where Apache Flink shines, offering a powerful solution to harness the full potential of an event-driven business model through efficient computing and processing capabilities.

Apache Kafka 45

Apache Kafka Hadoop ETL Data Pipeline 45

AWS Machine Learning Blog

MAY 5, 2023

The following figure shows an example diagram that illustrates an orchestrated extract, transform, and load (ETL) architecture solution. For example, searching for the terms “How to orchestrate ETL pipeline” returns results of architecture diagrams built with AWS Glue and AWS Step Functions.

AWS 106

AWS ETL ML ML 106

Mlearning.ai

MAY 16, 2023

ETL Design Pattern The ETL (Extract, Transform, Load) design pattern is a commonly used pattern in data engineering. ETL Design Pattern Here is an example of how the ETL design pattern can be used in a real-world scenario: A healthcare organization wants to analyze patient data to improve patient outcomes and operational efficiency.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

ODSC - Open Data Science

JUNE 19, 2023

As the name suggests, real-time operating systems (RTOS) handle real-time applications that undertake data and event processing under a strict deadline. Advanced analytics and AI  — It is virtually impossible to extract insights from big data through conventional evaluation and analysis, let alone manually.

Big Data 96

Big Data Big Data Artificial Intelligence Artificial Intelligence 96

AWS Machine Learning Blog

JANUARY 5, 2024

Whenever drift is detected, an event is launched to notify the respective teams to take action or initiate model retraining. Event-driven architecture – The pipelines for model training, model deployment, and model monitoring are well integrated by use Amazon EventBridge , a serverless event bus.

AWS 136

AWS Data Science ML ML 136

AWS Machine Learning Blog

JANUARY 6, 2023

TR used AWS Glue DataBrew and AWS Batch jobs to perform the extract, transform, and load (ETL) jobs in the ML pipelines, and SageMaker along with Amazon Personalize to tailor the recommendations. As the users are interacting with TR’s applications, they generate clickstream events, which are published into Amazon Kinesis Data Streams.

AWS 97

AWS Data Warehouse ML ML 97

Pickl AI

JULY 30, 2024

Guaranteed Delivery : NiFi ensures that data delivered reliably, even in the event of failures. It maintains a write-ahead log to ensure that the state of FlowFiles preserved, even in the event of a failure. Provenance Repository : This repository records all provenance events related to FlowFiles.

ETL 52

ETL Data Lakes Big Data Big Data 52

ODSC - Open Data Science

SEPTEMBER 27, 2023

Data Warehouses Some key characteristics of data warehouses are as follows: Data Type: Data warehouses primarily store structured data that has undergone ETL (Extract, Transform, Load) processing to conform to a specific schema. They are optimized for complex analytical queries and reporting. Interested in attending an ODSC event?

Data Lakes 98

Data Lakes Data Warehouse Database Big Data 98

phData

APRIL 29, 2024

Now, let’s cover the healthcare industry, which also has a surging demand for data and analytics, along with the underlying processes to make it happen. Some even provide a relational layer specifically designed for analytics, while others expose APIs. and delivers them to analytics platforms downstream.

SQL 52

SQL Data Warehouse Azure Cloud Data 52

Pickl AI

MARCH 19, 2025

Apache Kafka Apache Kafka is a distributed event streaming platform used for real-time data processing. It is commonly used for analytics and business intelligence, helping organisations make data-driven decisions. Google BigQuery Google BigQuery is a fully managed data warehouse that enables real-time analytics on large datasets.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

Tableau

AUGUST 21, 2023

Accelerator Instant access to the best analytic solutions Tableau Accelerators are ready-to-use dashboards that you can combine with your data and customize to fit your needs to help you get to data-driven insights faster. You can also use this to monitor events such as extract data source refresh failure and flow run failure.

Tableau 97

Tableau Database Analytics Analytics 97

ODSC - Open Data Science

JANUARY 18, 2024

EVENT — ODSC East 2024 In-Person and Virtual Conference April 23rd to 25th, 2024 Join us for a deep dive into the latest data science and AI trends, tools, and techniques, from LLMs to data analytics and from machine learning to responsible AI. Interested in attending an ODSC event? Learn more about our upcoming events here.

Data Engineering 52

Data Engineering Data Engineering Data Engineering Data Engineer 52

Dataconomy

MARCH 24, 2023

BI developer:  A BI developer is responsible for designing and implementing BI solutions, including data warehouses, ETL processes, and reports. Database management:  A BI professional should be able to design and manage databases, including data modeling, ETL processes, and data integration.

Business Intelligence 70

Business Intelligence Business Intelligence Data Analysis Data Analysis 70

Dataconomy

MARCH 24, 2023

BI developer:  A BI developer is responsible for designing and implementing BI solutions, including data warehouses, ETL processes, and reports. Database management:  A BI professional should be able to design and manage databases, including data modeling, ETL processes, and data integration.

Business Intelligence 70

Business Intelligence Business Intelligence Data Analysis Data Analysis 70

The MLOps Blog

MARCH 15, 2023

The system used advanced analytics and mostly classic machine learning algorithms to identify patterns and anomalies in claims data that may indicate fraudulent activity. If you aren’t aware already, let’s introduce the concept of ETL. We primarily used ETL services offered by AWS.

AWS 52

AWS ETL ML ML 52