KDnuggets

APRIL 5, 2023

Learn about Apache Kafka architecture and its implementation using a real-world use case of a taxi booking app.

Apache Kafka 262

Apache Kafka Data Engineering Data Engineer Data Engineering 262

Analytics Vidhya

NOVEMBER 2, 2020

Overview Learn about viewing data as streams of immutable events in contrast to mutable containers Understand how Apache Kafka captures real-time data through event. The post Apache Kafka: A Metaphorical Introduction to Event Streaming for Data Scientists and Data Engineers appeared first on Analytics Vidhya.

Apache Kafka 284

Apache Kafka Data Scientist Data Engineering Data Engineer 284

Hacker News

FEBRUARY 5, 2024

A desktop client for Apache Kafka. Contribute to Bogdanp/Franz development by creating an account on GitHub.

Apache Kafka 104

Apache Kafka 104

IBM Journey to AI blog

FEBRUARY 12, 2024

At the forefront of this event-driven revolution is Apache Kafka, the widely recognized and dominant open-source technology for event streaming. While most enterprises have already recognized how Apache Kafka provides a strong foundation for EDA, they often fall behind in unlocking its true potential.

Apache Kafka 86

Apache Kafka EDA SQL Database 86

IBM Journey to AI blog

NOVEMBER 3, 2023

Apache Kafka and Apache Flink working together Anyone who is familiar with the stream processing ecosystem is familiar with Apache Kafka: the de-facto enterprise standard for open-source event streaming. With Apache Kafka, you get a raw stream of events from everything that is happening within your business.

Apache Kafka 78

Apache Kafka Data Warehouse Data Pipeline SQL 78

Analytics Vidhya

JULY 12, 2023

Best Big Data Softwares - Apache Hadoop, Apache Spark, apache Kafka, Apache Storm, Apache Cassandra, Apache Hive, zoho & more.

Apache Kafka 266

Apache Kafka Apache Hadoop Big Data Big Data 266

Hacker News

FEBRUARY 8, 2023

Learn what windowing is, the difference between the four types of windows (hopping and tumbling, or session and sliding), and how to create them.

Apache Kafka 174

Apache Kafka 174

Dataconomy

MAY 26, 2017

The post Amazon Kinesis vs. Apache Kafka For Big Data Analysis appeared first on Dataconomy. Data processing today is done in form of pipelines which include various steps like aggregation, sanitization, filtering and finally generating insights by applying various statistical models. Parts of the Kinesis platform are.

Apache Kafka 155

Apache Kafka Data Analysis Data Analysis Big Data 155

Analytics Vidhya

SEPTEMBER 22, 2022

Dale Carnegie” Apache Kafka is a Software Framework for storing, reading, and analyzing streaming data. This article was published as a part of the Data Science Blogathon. Introduction “Learning is an active process. We learn by doing. Only knowledge that is used sticks in your mind.-

Data Pipeline 270

Data Pipeline Apache Kafka Internet of Things Data Science 270

Hacker News

APRIL 7, 2024

A cloud native implementation for Apache Kafka, reducing your cloud infrastructure bill by up to 90%. AutoMQ/automq

Apache Kafka 133

Apache Kafka 133

KDnuggets

APRIL 12, 2023

How to Build a Scalable Data Architecture with Apache Kafka Top 19 Skills You Need to Know in 2023 to Be a Data Scientist • 8 Open-Source Alternative to ChatGPT and Bard • Free eBook: 10 Practical Python Programming Tricks • DataLang: A New Programming Language for Data Scientists… Created by ChatGPT? •

Apache Kafka 193

Apache Kafka Data Scientist Python 193

Hacker News

OCTOBER 22, 2023

The choice between OpenTelemetry Collector and Apache Kafka isn't a zero-sum game. Each has its unique strengths and can even complement each other in certain architectures.

Apache Kafka 179

Apache Kafka Data Quality 179

Hacker News

DECEMBER 18, 2023

Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs. Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs. A multi-protocol, event-native proxy. GitHub - aklivity/zilla: ?

Apache Kafka 104

Apache Kafka 104

Hacker News

JULY 13, 2023

While playing Factorio the other day, I was struck by the many similarities with Apache Kafka.

Apache Kafka 179

Apache Kafka 179

IBM Journey to AI blog

APRIL 2, 2024

However, IBM MQ and Apache Kafka can sometimes be viewed as competitors, taking each other on in terms of speed, availability, cost and skills. MQ and Apache Kafka: Teammates Simply put, they are different technologies with different strengths, albeit often perceived to be quite similar. Interested in learning more?

Apache Kafka 65

Apache Kafka Clustering SQL AI 65

Towards AI

FEBRUARY 29, 2024

Within this article, we will explore the significance of these pipelines and utilise robust tools such as Apache Kafka and Spark to manage vast streams of data efficiently. Apache Kafka Apache Kafka is a distributed event streaming platform used for building real-time data pipelines and streaming applications.

Apache Kafka 133

Apache Kafka SQL Clustering Data Pipeline 133

Smart Data Collective

AUGUST 17, 2022

You can safely use an Apache Kafka cluster for seamless data movement from the on-premise hardware solution to the data lake using various cloud services like Amazon’s S3 and others. 5 Key Comparisons in Different Apache Kafka Architectures. 5 Key Comparisons in Different Apache Kafka Architectures.

Apache Kafka 109

Apache Kafka ETL Data Lakes AWS 109

Hacker News

JUNE 15, 2023

Comments (..)

Apache Kafka 98

Apache Kafka 98

Twilio Segment

SEPTEMBER 7, 2021

Event streaming platforms such as Apache Kafka are gaining in importance across all industries. In this article we'll discuss the benefits Apache Kafka implementations can gain from pairing it with a CDP.

Apache Kafka 40

Apache Kafka 40

IBM Journey to AI blog

JANUARY 8, 2024

They often use Apache Kafka as an open technology and the de facto standard for accessing events from a various core systems and applications. IBM provides an Event Streams capability build on Apache Kafka that makes events manageable across an entire enterprise.

EDA 80

EDA Apache Kafka Clustering Data Governance 80

IBM Journey to AI blog

NOVEMBER 21, 2023

Apache Kafka is a well-known open-source event store and stream processing platform and has grown to become the de facto standard for data streaming. Apache Kafka transfers data without validating the information in the messages.

Apache Kafka 107

Apache Kafka Clustering Data Quality Data Governance 107

IBM Journey to AI blog

MAY 9, 2023

IBM Event Automation provides an intuitive and integrated experience for distributing, discovering and processing business events across the organization: Event distribution: Collect raw streams of real-time business events with enterprise-grade Apache Kafka.

Apache Kafka 106

Apache Kafka Business Intelligence Business Intelligence 106

ODSC - Open Data Science

FEBRUARY 6, 2024

Apache Kafka For data engineers dealing with real-time data, Apache Kafka is a game-changer. Spark offers a versatile range of functionalities, from batch processing to stream processing, making it a comprehensive solution for complex data challenges.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

Precisely

DECEMBER 28, 2023

One very popular platform is Apache Kafka , a powerful open-source tool used by thousands of companies. But in all likelihood, Kafka doesn’t natively connect with the applications that contain your data. It also allows for applications, analytics, and reporting to process information as it happens.

Data Pipeline 91

Data Pipeline Apache Kafka Big Data Big Data 91

Precisely

JULY 21, 2023

Used by more than 75% of the Fortune 500, Apache Kafka has emerged as a powerful open source data streaming platform to meet these challenges. But harnessing and integrating Kafka’s full potential into enterprise environments can be complex. This is where Confluent steps in.

Apache Kafka 52

Apache Kafka Data Silos Data Pipeline Analytics 52

ODSC - Open Data Science

JUNE 7, 2023

Bilokon | Visiting Lecturer, CEO and Founder | Imperial College London, Thalesians Ltd Apache Kafka for Real-Time Machine Learning Without a Data Lake: Kai Waehner | Global Field CTO, Author, International Speaker Semantic Analysis and Procedural Language Understanding in the Era of Large Language Models: Dr.

Apache Kafka 52

Apache Kafka Machine Learning Machine Learning Data Science 52

IBM Journey to AI blog

MARCH 14, 2024

Apache Kafka stands as a widely recognized open source event store and stream processing platform. One key advantage of opting for managed Kafka services is the delegation of responsibility for broker and operational metrics, allowing users to focus solely on metrics specific to applications.

Apache Kafka 93

Apache Kafka Data Pipeline 93

Precisely

SEPTEMBER 12, 2023

Precisely data integrity solutions fuel your Confluent and Apache Kafka streaming data pipelines with trusted data that has maximum accuracy, consistency, and context and we’re ready to share more with you at the upcoming Current 2023. Let’s cover some additional information to know before attending.

Data Silos 52

Data Silos Apache Kafka Data Pipeline Data Quality 52

ODSC - Open Data Science

JULY 22, 2023

Leverage Compound Sparsity to Achieve the Fastest Inference Performance on CPUs: Damian Bogunowicz | Neural Magic and Konstantin Gulin | Machine Learning Engineer | Neural Magic Apache Kafka for Real-Time Machine Learning Without a Data Lake: Kai Waehner | Global Field CTO | Author, International Speaker Time Series Forecasting for Managers — All Forecasts (..)

Apache Kafka 52

Apache Kafka Machine Learning Machine Learning Data Science 52

ODSC - Open Data Science

JUNE 1, 2023

We’re going to assume that the pizza service already captures orders in Apache Kafka and is also keeping a record of its customers and the products that they sell in MySQL. Apache Pinot is a real-time OLAP database built at LinkedIn to deliver scalable real-time analytics with low latency.

Analytics 98

Analytics Analytics Apache Kafka Data Science 98

ODSC - Open Data Science

MAY 24, 2023

Streaming Machine Learning Without a Data Lake The combination of data streaming and ML enables you to build one scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem.

Data Lakes 52

Data Lakes ML ML Analytics 52

ODSC - Open Data Science

JULY 14, 2023

The session participants will learn the theory behind compound sparsity, state-of-the-art techniques, and how to apply it in practice using the Neural Magic platform.

Machine Learning 52

Machine Learning Machine Learning Apache Kafka Data Science 52

IBM Journey to AI blog

NOVEMBER 29, 2023

IBM Event Automation is a fully composable solution, built on open technologies, with capabilities for: Event streaming : Collect and distribute raw streams of real-time business events with enterprise-grade Apache Kafka. Event endpoint management : Describe and document events easily according to the Async API specification.

Artificial Intelligence 93

Artificial Intelligence Artificial Intelligence Apache Kafka AI 93

AWS Machine Learning Blog

MARCH 10, 2023

The same architecture applies if you use Amazon Managed Streaming for Apache Kafka (Amazon MSK) as a data streaming service. This pattern can be useful for real-time fraud detection, notification, and potential prevention. Example use cases for this could be payment processing or high-volume account creation.

Machine Learning 115

Machine Learning Machine Learning AWS Apache Kafka 115

ODSC - Open Data Science

MAY 31, 2023

Be sure to check out his talk, “ Apache Kafka for Real-Time Machine Learning Without a Data Lake ,” there! The combination of data streaming and machine learning (ML) enables you to build one scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem.

Data Lakes 131

Data Lakes Machine Learning Machine Learning Apache Kafka 131

Data Science Dojo

MARCH 9, 2023

Challenges for individuals Traditional messaging brokers, such as Apache Kafka, RabbitMQ, and ActiveMQ, have been widely used to enable communication between applications and services. Handling too many data sources can become overwhelming, especially with complex schemas. Debugging and troubleshooting can also be challenging.

Apache Kafka 326

Apache Kafka Azure Data Science Data Pipeline 326

Mlearning.ai

AUGUST 4, 2023

Apache Kafka and R abbitMQ are particularly popular in LEs. Graph 7: Percentage of Programming Languages MiscTech Tools In Both LEs and SMEs: ‘. NET (5+) ’, ‘ pandas ’, ‘ numpy ’, and ‘. NET Framework (1.0–4.8)’ 4.8)’ are widely used.

Database 52

Database Apache Kafka SQL AI 52