Analytics Vidhya

APRIL 28, 2023

Introduction Apache Kafka is an open-source publish-subscribe messaging application initially developed by LinkedIn in early 2011. It is a famous Scala-coded data processing tool that offers low latency, extensive throughput, and a unified platform to handle the data in real-time.

Apache Kafka 289

Apache Kafka Analytics Analytics Hadoop 289

Analytics Vidhya

NOVEMBER 2, 2020

Overview Learn about viewing data as streams of immutable events in contrast to mutable containers Understand how Apache Kafka captures real-time data through event. The post Apache Kafka: A Metaphorical Introduction to Event Streaming for Data Scientists and Data Engineers appeared first on Analytics Vidhya.

Apache Kafka 291

Apache Kafka Data Scientist Data Engineering Data Engineering 291

KDnuggets

APRIL 5, 2023

Learn about Apache Kafka architecture and its implementation using a real-world use case of a taxi booking app.

Apache Kafka 271

Apache Kafka Data Engineering Data Engineering Data Engineer 271

KDnuggets

APRIL 1, 2025

This article explains how to create a system that processes data in real time using Apache Kafka and Spark.

Apache Kafka 237

Apache Kafka Data Science Analytics Analytics 237

Flipboard

MAY 14, 2025

Well explain what it is, why it matters, and how to use tools like Apache Kafka, Apache Flink, and PyFlink to build real-time pipelines. This guide introduces data streaming from a data science perspective.

Apache Kafka 108

Apache Kafka Data Science 108

Dataconomy

MAY 26, 2017

The post Amazon Kinesis vs. Apache Kafka For Big Data Analysis appeared first on Dataconomy. Data processing today is done in form of pipelines which include various steps like aggregation, sanitization, filtering and finally generating insights by applying various statistical models. Parts of the Kinesis platform are.

Apache Kafka 155

Apache Kafka Big Data Big Data Data Analysis 155

databricks

AUGUST 12, 2024

The blog explores data streams from NASA satellites using Apache Kafka and Databricks. It demonstrates ingestion and transformation with Delta Live Tables in SQL and AI/BI-powered analysis of supernova events.

Apache Kafka 290

Apache Kafka SQL AI AI 290

Analytics Vidhya

JULY 12, 2023

Best Big Data Softwares - Apache Hadoop, Apache Spark, apache Kafka, Apache Storm, Apache Cassandra, Apache Hive, zoho & more.

Apache Kafka 266

Apache Kafka Apache Hadoop Big Data Big Data 266

Analytics Vidhya

SEPTEMBER 22, 2022

Dale Carnegie” Apache Kafka is a Software Framework for storing, reading, and analyzing streaming data. This article was published as a part of the Data Science Blogathon. Introduction “Learning is an active process. We learn by doing. Only knowledge that is used sticks in your mind.-

Data Pipeline 270

Data Pipeline Apache Kafka Internet of Things Data Science 270

Hacker News

FEBRUARY 8, 2023

Learn what windowing is, the difference between the four types of windows (hopping and tumbling, or session and sliding), and how to create them.

Apache Kafka 175

Apache Kafka 175

Hacker News

FEBRUARY 5, 2024

A desktop client for Apache Kafka. Contribute to Bogdanp/Franz development by creating an account on GitHub.

Apache Kafka 109

Apache Kafka 109

KDnuggets

APRIL 12, 2023

How to Build a Scalable Data Architecture with Apache Kafka Top 19 Skills You Need to Know in 2023 to Be a Data Scientist • 8 Open-Source Alternative to ChatGPT and Bard • Free eBook: 10 Practical Python Programming Tricks • DataLang: A New Programming Language for Data Scientists… Created by ChatGPT? •

Apache Kafka 241

Apache Kafka Data Scientist Python 241

IBM Journey to AI blog

FEBRUARY 12, 2024

At the forefront of this event-driven revolution is Apache Kafka, the widely recognized and dominant open-source technology for event streaming. While most enterprises have already recognized how Apache Kafka provides a strong foundation for EDA, they often fall behind in unlocking its true potential.

Apache Kafka 100

Apache Kafka EDA SQL Database 100

Hacker News

JULY 13, 2023

While playing Factorio the other day, I was struck by the many similarities with Apache Kafka.

Apache Kafka 179

Apache Kafka 179

IBM Journey to AI blog

NOVEMBER 3, 2023

Apache Kafka and Apache Flink working together Anyone who is familiar with the stream processing ecosystem is familiar with Apache Kafka: the de-facto enterprise standard for open-source event streaming. With Apache Kafka, you get a raw stream of events from everything that is happening within your business.

Apache Kafka 79

Apache Kafka Data Warehouse Data Pipeline Big Data 79

Hacker News

OCTOBER 22, 2023

The choice between OpenTelemetry Collector and Apache Kafka isn't a zero-sum game. Each has its unique strengths and can even complement each other in certain architectures.

Apache Kafka 179

Apache Kafka Data Quality 179

Hacker News

AUGUST 30, 2024

This is a guest article by Stanislav Kozlovski, an Apache Kafka Committer. If you would like to connect with Stanislav, you can do so on Twitter and LinkedIn. AWS S3 is a service every engineer is familiar with. It’s the service that popularized the notion of cold-storage to the

Apache Kafka 156

Apache Kafka AWS 156

IBM Journey to AI blog

SEPTEMBER 4, 2024

Apache Kafka is an open-source , distributed streaming platform that allows developers to build real-time, event-driven applications. With Apache Kafka, developers can build applications that continuously use streaming data records and deliver real-time experiences to users. How does Apache Kafka work?

Apache Kafka 58

Apache Kafka Internet of Things Data Pipeline Clustering 58

Hacker News

APRIL 7, 2024

A cloud native implementation for Apache Kafka, reducing your cloud infrastructure bill by up to 90%. AutoMQ/automq

Apache Kafka 134

Apache Kafka 134

Smart Data Collective

AUGUST 17, 2022

You can safely use an Apache Kafka cluster for seamless data movement from the on-premise hardware solution to the data lake using various cloud services like Amazon’s S3 and others. 5 Key Comparisons in Different Apache Kafka Architectures. 5 Key Comparisons in Different Apache Kafka Architectures.

Apache Kafka 130

Apache Kafka ETL Data Lakes AWS 130

Hacker News

DECEMBER 18, 2023

Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs. Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs. A multi-protocol, event-native proxy. GitHub - aklivity/zilla: ?

Apache Kafka 109

Apache Kafka 109

Hacker News

MAY 8, 2024

Recently I wanted to learn a bit about Apache Kafka. It is often used as a way to do event sourcing (or similar message-driven architectures). An “add-on” to.

Apache Kafka 116

Apache Kafka 116

Hacker News

APRIL 24, 2025

The last few days I spent some time digging into the recently announced KIP-1150 ("Diskless Kafka"), as well AutoMQs Kafka fork, tightly integrating Apache Kafka and object storage, such as S3.

Apache Kafka 120

Apache Kafka ETL 120

Hacker News

JUNE 15, 2023

Comments (..)

Apache Kafka 100

Apache Kafka 100

ODSC - Open Data Science

MAY 31, 2023

Be sure to check out his talk, “ Apache Kafka for Real-Time Machine Learning Without a Data Lake ,” there! The combination of data streaming and machine learning (ML) enables you to build one scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem.

Data Lakes 131

Data Lakes Machine Learning Machine Learning Apache Kafka 131

Towards AI

FEBRUARY 29, 2024

Within this article, we will explore the significance of these pipelines and utilise robust tools such as Apache Kafka and Spark to manage vast streams of data efficiently. Apache Kafka Apache Kafka is a distributed event streaming platform used for building real-time data pipelines and streaming applications.

Apache Kafka 140

Apache Kafka SQL Clustering Data Pipeline 140

Dataconomy

JULY 29, 2024

Talks and insights Mikhail Epikhin: Navigating the processor landscape for Apache Kafka Mikhail Epikhin began the session by sharing his team’s research on optimizing Managed Service for Apache Kafka. His presentation focused on the performance and efficiency of different instance types and processor architectures.

Apache Kafka 91

Apache Kafka Data Pipeline Data Warehouse ETL 91

Dataconomy

MARCH 11, 2025

Apache Flink: A powerful open-source framework for distributed stream processing with an emphasis on event-driven applications. Apache Kafka: Vital for creating real-time data pipelines and streaming applications. StreamAnalytix: A user-friendly interface that allows for intuitive application management across various domains.

Apache Kafka 91

Apache Kafka Machine Learning Machine Learning Data Mining 91

IBM Journey to AI blog

APRIL 2, 2024

However, IBM MQ and Apache Kafka can sometimes be viewed as competitors, taking each other on in terms of speed, availability, cost and skills. MQ and Apache Kafka: Teammates Simply put, they are different technologies with different strengths, albeit often perceived to be quite similar. Interested in learning more?

Apache Kafka 75

Apache Kafka Clustering SQL AI 75

IBM Journey to AI blog

NOVEMBER 21, 2023

Apache Kafka is a well-known open-source event store and stream processing platform and has grown to become the de facto standard for data streaming. Apache Kafka transfers data without validating the information in the messages.

Apache Kafka 116

Apache Kafka Clustering Data Quality Data Governance 116

IBM Journey to AI blog

MAY 9, 2023

IBM Event Automation provides an intuitive and integrated experience for distributing, discovering and processing business events across the organization: Event distribution: Collect raw streams of real-time business events with enterprise-grade Apache Kafka.

Apache Kafka 110

Apache Kafka Business Intelligence Business Intelligence 110

ODSC - Open Data Science

APRIL 23, 2025

Confluent Confluent provides a robust data streaming platform built around Apache Kafka. With AI credits, teams can streamline the annotation process using intelligent suggestions and quality control mechanisms. Amazon Web Services(AWS) AWS offers one of the most extensive AI and ML infrastructures in the world.

Data Scientist 52

Data Scientist Azure Apache Kafka ML 52

IBM Journey to AI blog

JANUARY 8, 2024

They often use Apache Kafka as an open technology and the de facto standard for accessing events from a various core systems and applications. IBM provides an Event Streams capability build on Apache Kafka that makes events manageable across an entire enterprise.

EDA 92

EDA Apache Kafka Clustering Data Governance 92

Analytics Vidhya

JUNE 27, 2020

Overview There are a plethora of data science tools out there – which one should you pick up? Here’s a list of over 20. The post 22 Widely Used Data Science and Machine Learning Tools in 2020 appeared first on Analytics Vidhya.

Data Science 400

Data Science Machine Learning Machine Learning Analytics 400

Twilio Segment

SEPTEMBER 7, 2021

Event streaming platforms such as Apache Kafka are gaining in importance across all industries. In this article we'll discuss the benefits Apache Kafka implementations can gain from pairing it with a CDP.

Apache Kafka 40

Apache Kafka 40

IBM Journey to AI blog

AUGUST 29, 2024

The unique advantages of Apache Flink Apache Flink augments event streaming technologies like Apache Kafka to enable businesses to respond to events more effectively in real time. Integration: Integrates seamlessly with other data systems and platforms, including Apache Kafka, Spark, Hadoop and various databases.

Apache Kafka 45

Apache Kafka Hadoop ETL Data Pipeline 45

Women in Big Data

OCTOBER 9, 2024

Data Ingestion: Data is collected and funneled into the pipeline using batch or real-time methods, leveraging tools like Apache Kafka, AWS Kinesis, or custom ETL scripts. This phase ensures quality and consistency using frameworks like Apache Spark or AWS Glue.

Big Data 59

Big Data Big Data Apache Kafka Data Pipeline 59