Analytics Vidhya

APRIL 28, 2023

Introduction Apache Kafka is an open-source publish-subscribe messaging application initially developed by LinkedIn in early 2011. It is a famous Scala-coded data processing tool that offers low latency, extensive throughput, and a unified platform to handle the data in real-time.

Apache Kafka 235

Apache Kafka Analytics Analytics Hadoop 235

KDnuggets

APRIL 5, 2023

Learn about Apache Kafka architecture and its implementation using a real-world use case of a taxi booking app.

Apache Kafka 249

Apache Kafka Data Engineering Data Engineering Data Engineer 249

Dataconomy

MAY 26, 2017

The post Amazon Kinesis vs. Apache Kafka For Big Data Analysis appeared first on Dataconomy. Data processing today is done in form of pipelines which include various steps like aggregation, sanitization, filtering and finally generating insights by applying various statistical models. Parts of the Kinesis platform are.

Apache Kafka 155

Apache Kafka Data Analysis Big Data Data Analysis 155

databricks

AUGUST 12, 2024

The blog explores data streams from NASA satellites using Apache Kafka and Databricks. It demonstrates ingestion and transformation with Delta Live Tables in SQL and AI/BI-powered analysis of supernova events.

Apache Kafka 252

Apache Kafka SQL AI AI 252

Hacker News

FEBRUARY 8, 2023

Learn what windowing is, the difference between the four types of windows (hopping and tumbling, or session and sliding), and how to create them.

Apache Kafka 168

Apache Kafka 168

Hacker News

FEBRUARY 5, 2024

A desktop client for Apache Kafka. Contribute to Bogdanp/Franz development by creating an account on GitHub.

Apache Kafka 100

Apache Kafka 100

Analytics Vidhya

SEPTEMBER 22, 2022

Dale Carnegie” Apache Kafka is a Software Framework for storing, reading, and analyzing streaming data. This article was published as a part of the Data Science Blogathon. Introduction “Learning is an active process. We learn by doing. Only knowledge that is used sticks in your mind.-

Data Pipeline 224

Data Pipeline Apache Kafka Internet of Things Data Science 224

IBM Journey to AI blog

FEBRUARY 12, 2024

At the forefront of this event-driven revolution is Apache Kafka, the widely recognized and dominant open-source technology for event streaming. While most enterprises have already recognized how Apache Kafka provides a strong foundation for EDA, they often fall behind in unlocking its true potential.

Apache Kafka 100

Apache Kafka EDA SQL Database 100

Analytics Vidhya

JULY 12, 2023

Best Big Data Softwares - Apache Hadoop, Apache Spark, apache Kafka, Apache Storm, Apache Cassandra, Apache Hive, zoho & more.

Apache Kafka 201

Apache Kafka Apache Hadoop Big Data Big Data 201

Hacker News

JULY 13, 2023

While playing Factorio the other day, I was struck by the many similarities with Apache Kafka.

Apache Kafka 177

Apache Kafka 177

IBM Journey to AI blog

NOVEMBER 3, 2023

Apache Kafka and Apache Flink working together Anyone who is familiar with the stream processing ecosystem is familiar with Apache Kafka: the de-facto enterprise standard for open-source event streaming. With Apache Kafka, you get a raw stream of events from everything that is happening within your business.

Apache Kafka 79

Apache Kafka Data Warehouse Data Pipeline Big Data 79

KDnuggets

APRIL 12, 2023

How to Build a Scalable Data Architecture with Apache Kafka Top 19 Skills You Need to Know in 2023 to Be a Data Scientist • 8 Open-Source Alternative to ChatGPT and Bard • Free eBook: 10 Practical Python Programming Tricks • DataLang: A New Programming Language for Data Scientists… Created by ChatGPT? •

Apache Kafka 179

Apache Kafka Data Scientist Python 179

Hacker News

OCTOBER 22, 2023

The choice between OpenTelemetry Collector and Apache Kafka isn't a zero-sum game. Each has its unique strengths and can even complement each other in certain architectures.

Apache Kafka 177

Apache Kafka Data Quality 177

Hacker News

AUGUST 30, 2024

This is a guest article by Stanislav Kozlovski, an Apache Kafka Committer. If you would like to connect with Stanislav, you can do so on Twitter and LinkedIn. AWS S3 is a service every engineer is familiar with. It’s the service that popularized the notion of cold-storage to the

Apache Kafka 141

Apache Kafka AWS 141

Hacker News

APRIL 7, 2024

A cloud native implementation for Apache Kafka, reducing your cloud infrastructure bill by up to 90%. AutoMQ/automq

Apache Kafka 128

Apache Kafka 128

IBM Journey to AI blog

SEPTEMBER 4, 2024

Apache Kafka is an open-source , distributed streaming platform that allows developers to build real-time, event-driven applications. With Apache Kafka, developers can build applications that continuously use streaming data records and deliver real-time experiences to users. How does Apache Kafka work?

Apache Kafka 58

Apache Kafka Internet of Things Data Pipeline Clustering 58

Smart Data Collective

AUGUST 17, 2022

You can safely use an Apache Kafka cluster for seamless data movement from the on-premise hardware solution to the data lake using various cloud services like Amazon’s S3 and others. 5 Key Comparisons in Different Apache Kafka Architectures. 5 Key Comparisons in Different Apache Kafka Architectures.

Apache Kafka 113

Apache Kafka ETL Data Lakes AWS 113

Hacker News

MAY 8, 2024

Recently I wanted to learn a bit about Apache Kafka. It is often used as a way to do event sourcing (or similar message-driven architectures). An “add-on” to.

Apache Kafka 104

Apache Kafka 104

Hacker News

DECEMBER 18, 2023

Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs. Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs. A multi-protocol, event-native proxy. GitHub - aklivity/zilla: ?

Apache Kafka 98

Apache Kafka 98

Hacker News

JUNE 15, 2023

Comments (..)

Apache Kafka 97

Apache Kafka 97

ODSC - Open Data Science

MAY 31, 2023

Be sure to check out his talk, “ Apache Kafka for Real-Time Machine Learning Without a Data Lake ,” there! The combination of data streaming and machine learning (ML) enables you to build one scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem.

Data Lakes 131

Data Lakes Machine Learning Machine Learning Apache Kafka 131

Towards AI

FEBRUARY 29, 2024

Within this article, we will explore the significance of these pipelines and utilise robust tools such as Apache Kafka and Spark to manage vast streams of data efficiently. Apache Kafka Apache Kafka is a distributed event streaming platform used for building real-time data pipelines and streaming applications.

Apache Kafka 131

Apache Kafka SQL Clustering Data Pipeline 131

Dataconomy

JULY 29, 2024

Talks and insights Mikhail Epikhin: Navigating the processor landscape for Apache Kafka Mikhail Epikhin began the session by sharing his team’s research on optimizing Managed Service for Apache Kafka. His presentation focused on the performance and efficiency of different instance types and processor architectures.

Apache Kafka 91

Apache Kafka Data Pipeline Data Warehouse ETL 91

Data Science Dojo

MARCH 9, 2023

Challenges for individuals Traditional messaging brokers, such as Apache Kafka, RabbitMQ, and ActiveMQ, have been widely used to enable communication between applications and services. Handling too many data sources can become overwhelming, especially with complex schemas. Debugging and troubleshooting can also be challenging.

Apache Kafka 208

Apache Kafka Azure Data Science Data Pipeline 208

IBM Journey to AI blog

APRIL 2, 2024

However, IBM MQ and Apache Kafka can sometimes be viewed as competitors, taking each other on in terms of speed, availability, cost and skills. MQ and Apache Kafka: Teammates Simply put, they are different technologies with different strengths, albeit often perceived to be quite similar. Interested in learning more?

Apache Kafka 75

Apache Kafka Clustering SQL AI 75

IBM Journey to AI blog

NOVEMBER 21, 2023

Apache Kafka is a well-known open-source event store and stream processing platform and has grown to become the de facto standard for data streaming. Apache Kafka transfers data without validating the information in the messages.

Apache Kafka 116

Apache Kafka Clustering Data Quality Data Governance 116

IBM Journey to AI blog

MAY 9, 2023

IBM Event Automation provides an intuitive and integrated experience for distributing, discovering and processing business events across the organization: Event distribution: Collect raw streams of real-time business events with enterprise-grade Apache Kafka.

Apache Kafka 110

Apache Kafka Business Intelligence Business Intelligence 110

IBM Journey to AI blog

JANUARY 8, 2024

They often use Apache Kafka as an open technology and the de facto standard for accessing events from a various core systems and applications. IBM provides an Event Streams capability build on Apache Kafka that makes events manageable across an entire enterprise.

EDA 92

EDA Apache Kafka Clustering Data Governance 92

Twilio Segment

SEPTEMBER 7, 2021

Event streaming platforms such as Apache Kafka are gaining in importance across all industries. In this article we'll discuss the benefits Apache Kafka implementations can gain from pairing it with a CDP.

Apache Kafka 40

Apache Kafka 40

Analytics Vidhya

JUNE 27, 2020

Overview There are a plethora of data science tools out there – which one should you pick up? Here’s a list of over 20. The post 22 Widely Used Data Science and Machine Learning Tools in 2020 appeared first on Analytics Vidhya.

Machine Learning 400

Machine Learning Data Science Machine Learning Analytics 400

IBM Journey to AI blog

AUGUST 29, 2024

The unique advantages of Apache Flink Apache Flink augments event streaming technologies like Apache Kafka to enable businesses to respond to events more effectively in real time. Integration: Integrates seamlessly with other data systems and platforms, including Apache Kafka, Spark, Hadoop and various databases.

Apache Kafka 45

Apache Kafka Hadoop ETL Data Pipeline 45

Women in Big Data

OCTOBER 9, 2024

Data Ingestion: Data is collected and funneled into the pipeline using batch or real-time methods, leveraging tools like Apache Kafka, AWS Kinesis, or custom ETL scripts. This phase ensures quality and consistency using frameworks like Apache Spark or AWS Glue.

Big Data 59

Big Data Big Data Apache Kafka Data Pipeline 59

Data Science Dojo

JULY 24, 2023

Apache Flink for stream processing: Wrapping up In conclusion, stream processing with distributed systems like Apache Kafka, Apache Flink, and Apache Spark Streaming empowers organizations to harness real-time data insights, enabling timely decision-making and enhanced user experiences.

Big Data 195

Big Data Big Data Data Engineering Data Engineering 195

phData

MAY 3, 2024

How Snowflake Helps Achieve Real-Time Analytics Snowflake is the ideal platform to achieve real-time analytics for several reasons, but two of the biggest are its ability to manage concurrency due to the multi-cluster architecture of Snowflake and its robust connections to 3rd party tools like Kafka. Looking for additional help?

Apache Kafka 52

Apache Kafka Analytics Analytics ETL 52

Data Science Blog

JUNE 27, 2023

In practical implementation, the Kappa architecture is commonly deployed using Apache Kafka or Kafka-based tools. Applications can directly read from and write to Kafka or an alternative message queue tool. This architectural concept relies on event streaming as the core element of data delivery.

Big Data 130

Big Data Big Data Apache Kafka Database 130

insideBIGDATA

JUNE 25, 2024

In this contributed article, Sijie Guo, Founder and CEO of Streamnative, believes that with remote work entrenched in the post-pandemic enterprise, organizations are restructuring their technology stack and software strategy for a new, distributed workforce.

Apache Kafka 243

Apache Kafka Big Data Big Data Analytics 243

IBM Journey to AI blog

MARCH 14, 2024

Apache Kafka stands as a widely recognized open source event store and stream processing platform. One key advantage of opting for managed Kafka services is the delegation of responsibility for broker and operational metrics, allowing users to focus solely on metrics specific to applications.

Apache Kafka 106

Apache Kafka Data Pipeline 106