IBM Data Science in Practice

JANUARY 9, 2024

How Maximo Visual Inspection empowers Image Segmentation Image segmentation (IS) is the process of locating objects and their boundaries in computer vision, more specifically, it assigns every pixel in an image to a class. IS can be broken down into two distinct types: Semantic segmentation: this type of segmentation groups together multiple objects of the same class as a single instance.

Data Science 130

Data Science 130

IBM Data Science in Practice

DECEMBER 21, 2023

Many consumers prefer to listen to digital text rather than read it in today’s digital environment, and Text-to-speech (TTS) technology can help you turn digital text into audio. IBM Watson Text to Speech is a service that converts text into natural-sounding speech in a range of languages , dialects, and voices using speech synthesis capabilities. This service also includes a number of settings for modifying the characteristics of speech synthesis such as speech rate and voice pitch.

Exploratory Data Analysis 130

Exploratory Data Analysis EDA Python Clustering 130

IBM Data Science in Practice

DECEMBER 4, 2023

Generative AI is dominating the headlines, and with it comes questions about AI Governance. The new and amplified risks of generative AI along with new regulations have meant enterprises have been looking for solutions to these challenges. And that is where watsonx.governance comes in. watsonx.governance is the next step in our watsonx story and brings aspects of trust and explainability, with new features to govern LLMs. watsonx.governance is the introduction of one unified platform to govern b

Data Scientist 130

Data Scientist ML ML AI 130

IBM Data Science in Practice

NOVEMBER 24, 2023

Part 1: built-in metrics from basic UI navigation Overview In this article, we share our experience with IBM Turbonomic in Cloud Pak for Data (CP4D) environments. Our experience focuses on using monitoring capabilities to help our development teams to analyze resource usage and identify performance and resource optimizations. What we have learned should be helpful to you as well.

Clustering 130

Clustering Data Science AI AI 130

IBM Data Science in Practice

NOVEMBER 24, 2023

Part 2: Advanced report-based monitoring Overview In part 1 of the article, we introduce some basic UI navigation to show the built-in metrics. We share our experience with IBM Turbonomic in Cloud Pak for Data (CP4D) environments. Our experience focuses on using monitoring capabilities to help our development teams to analyze resource usage and identify performance and resource optimizations.

Clustering 130

Clustering Data Science 130

IBM Data Science in Practice

NOVEMBER 15, 2023

This article is co-authored with Annalise Sumpon. What is Image Segmentation? Image segmentation (IS) is the process of locating objects and their boundaries in computer vision, more specifically, it assigns every pixel in an image to a class. IS can be broken down into two distinct types: Semantic segmentation: this type of segmentation groups together multiple objects of the same class as a single instance.

Data Science 130

Data Science AI AI 130

IBM Data Science in Practice

NOVEMBER 3, 2023

Picture this: a world where every movement, gesture, and moment is meticulously observed and understood. In the realm of action detection, this futuristic dream is fast becoming a reality. As technology continues to evolve at an unprecedented pace, we find ourselves on the brink of a revolution that promises to transform the way we interact with the digital world.

Deep Learning 130

Deep Learning Deep Learning Data Science Artificial Intelligence 130

IBM Data Science in Practice

OCTOBER 18, 2023

By Carolyn Saplicki, Data Scientist I recently worked on a project that utilized panel data. Currently, many packages have been developed to handle panel data such as plm, fixest, and brms, among others. However, dynamite stands out by offering unique features that make it a unique for handling complex panel data. As dynamite is relatively new, there were few examples of its use outside of documentation.

Data Analysis 130

Data Analysis Data Analysis Data Scientist Data Science 130

IBM Data Science in Practice

AUGUST 23, 2023

Improve Cluster Balance with the CPD Scheduler — Part 1 The default Kubernetes (“k8s”) scheduler can be thought of as a sort of “greedy” scheduler, in that it always tries to place pods on the nodes that have the most free resources. This can lead to problems, as it can cause one or more nodes to become overloaded while others are underutilized. This can lead to performance problems and even outages.

Clustering 130

Clustering Algorithm Data Preparation Data Science 130

IBM Data Science in Practice

JULY 12, 2023

In the dynamic realm of data science, having access to advanced tools and platforms is crucial for AI engineers to harness the evolving potential of AI. IBM has recently released watsonx , an enterprise-ready AI and data platform. With this news, it’s hard not to think about how watsonx can revolutionize data science workflows by providing AI engineers with a suite of robust capabilities.

Data Science 130

Data Science AI AI Database 130

IBM Data Science in Practice

JULY 5, 2023

Improve Cluster Balance with CPD Scheduler — Part 2 The default Kubernetes scheduler has some limitations that cause unbalanced clusters. You can see more details in part 1 on this topic. In an unbalanced cluster, some of the worker nodes are overloaded and others are under-utilized. Such clusters are not good for resource usage efficiency and a consistent performance experience.

Clustering 130

Clustering Data Science Algorithm 130

IBM Data Science in Practice

JULY 5, 2023

Did you know that otters are able to learn from each other? [1] And that otters frequently hold hands while they sleep to keep themselves from drifting apart? [2] This way, they can create a “network of otters” ensuring all of them stay together, and they can keep learning from each other. This can be extrapolated to several other aspects of life, in which each part of a whole is important and contributes, in some way, to the final goal.

Database 130

Database Python Algorithm Deep Learning 130

IBM Data Science in Practice

JUNE 16, 2023

By Carolyn Saplicki and Erika Agostinelli As industries begin to utilize AI in Sustainability practices, the focus, thus far, has been on Asset Performance Management (APM). In this process, statistical or machine learning (ML) models have been created to aid with predictive maintenance. This means predictive maintenance depends on the health of your models.

Machine Learning 130

Machine Learning Machine Learning ML ML 130

IBM Data Science in Practice

MAY 26, 2023

By Erika Agostinelli (IBM Senior Data Scientist), Stefan van der Stockt (IBM Lead Data Scientist, Master Inventor) and Carolyn Saplicki (IBM Data Scientist) Various industries are utilizing AI in Sustainability practices more to create models that aid with critical asset performance management (APM) initiatives. These models are applied to anything from condition-based monitoring, predictive maintenance, asset failure prediction, detecting anomalies in daily usage patterns, to predicting the end

AI 130

AI AI Data Scientist Data Science 130

IBM Data Science in Practice

APRIL 4, 2023

Introduction This article shows how to monitor a model deployed on AWS Sagemaker for quality, bias and explainability, using IBM Watson OpenScale on the IBM Cloud Pak for Data platform. You can leverage both AWS Sagemaker and IBM Cloud Pak for Data platforms to bring a machine learning model from development to production with maximum flexibility, and then monitor the health and performance of the deployed models with Watson OpenScale.

AWS 130

AWS Machine Learning Machine Learning Data Science 130

IBM Data Science in Practice

MARCH 21, 2023

By Courtney Branson and Carolyn Saplicki Today, many businesses create AI models to aid in prediction. However, not all businesses have the tools or understanding of Machine Learning Operations (MLOps) to productionize an AI model. In fact, VentureBeat found that 87% of data science projects never make it in production. Often, research teams do not generally consider the operationalization of models during creation.

Deep Learning 130

Deep Learning Deep Learning Machine Learning Machine Learning 130

IBM Data Science in Practice

MARCH 21, 2023

Deep learning models built using Maximo Visual Inspection (MVI) are used for a wide range of applications, including image classification and object detection. These models train on large datasets and learn complex patterns that are difficult for humans to recognize. This may be a daunting task for a non-data scientist or a data scientist with little to no experience.

ML 130

ML ML Deep Learning Deep Learning 130

IBM Data Science in Practice

MARCH 13, 2023

With IBM Watson NLP, IBM introduced a common library for natural language processing, document understanding, translation, and trust. IBM Watson NLP brings everything under one umbrella for consistency and ease of development and deployment. This tutorial walks you through the steps to serve pretrained Watson NLP models using Knative Serving in a Red Hat OpenShift cluster.

Clustering 130

Clustering Natural Language Processing Data Science AI 130

IBM Data Science in Practice

MARCH 13, 2023

Co-authored by Carolyn Saplicki , Data Scientist with Expert Labs, and Mitali Bante , Data Scientist with Expert Labs. In our previous blog, Fairness Explained: Definitions and Metrics , we discuss fairness definitions and fairness metrics through a real-world example. This sets the stage for how bias can be identified in machine learning. Specifically, we utilize tools such as the Fairness Compass and AI Fairness 360 to understand the existence of bias and quantify its effects.

Machine Learning 130

Machine Learning Machine Learning Algorithm Data Scientist 130

IBM Data Science in Practice

MARCH 13, 2023

This is Part 3 of a series of blogs on IBM AI Governance. We are in a unique phase right now, where everyone is excited about AI and its potential impact on humanity. Along with bright future prospects, there is also a recognition of significant risks associated with growing AI innovations and adoption. There is a significant increase in AI rules, regulations, and the need for AI Governance to manage this risk.

AI 130

AI AI Data Science Machine Learning 130

IBM Data Science in Practice

MARCH 8, 2023

Feature Platforms — A New Paradigm in Machine Learning Operations (MLOps) Operationalizing Machine Learning is Still Hard OpenAI introduced ChatGPT. DeepMind launched AlphaFold , which can accurately predict 3D models of protein structures, accelerating research in nearly every field of biology. The United States published a Blueprint for the AI Bill of Rights.

Machine Learning 264

Machine Learning Machine Learning ML ML 264

IBM Data Science in Practice

MARCH 8, 2023

This is Part 2 of a series of blogs on IBM AI Governance. As a first step to better understand where AI Governance fits in, its useful to think of common applications and use cases in these 3 categories: AI lifecycle governance refers to processes and practices to help ensure transparent and explainable AI from model development, testing, validation, and deployment to retirement.

AI 130

AI AI ML ML 130

IBM Data Science in Practice

MARCH 8, 2023

Introduction This article shows how to build a machine learning model on a Watson Studio platform running on IBM Cloud Pak for Data (CPD). You can then export the model and deploy it on Amazon Sagemaker on Amazon Web Server (AWS). You can leverage both platforms to bring a machine learning model from development to production with maximum flexibility.

AWS 130

AWS Machine Learning Machine Learning Algorithm 130

IBM Data Science in Practice

MARCH 7, 2023

Introducing IBM AI Governance to unlock full potential of AI investments while mitigating risks I am a big fan on Thomas Friedman. He is a New York Times columnist and has written groundbreaking books like The World is Flat and The Lexus and Olive Tree. He focuses on revolutionary changes that happen to humans globally. His latest book is Thank You for Being Late.

AI 242

AI AI Data Science Artificial Intelligence 242

IBM Data Science in Practice

FEBRUARY 21, 2023

Just for AI Titans — Autonomous & Continuous AI Training —  MLOPS on steroids. Your company created multiple models, and the maintenance and redeployment workload are exploding, taking up the majority of your time? Continuous training is the solution. Photo by Jeroen den Otter on Unsplash Who should read this article: Machine and Deep Learning Engineers, Solution Architects, Data Scientist, AI Enthusiast, AI Founders What is covered in this article?

AI 130

AI AI Machine Learning Machine Learning 130

IBM Data Science in Practice

FEBRUARY 21, 2023

IBM Analytics Engine — Serverless Spark — End to End Demo Short Video Demos — Run Spark workloads, Troubleshoot using Spark UI/ Spark History in combination with Logs DEMO START This video blog is a series of quick, short demos on various concepts of IBM Analytics Engine Serverless plan — right from creating and configuring an “instance”, running spark workloads, troubleshooting the run through a combination of the application run logs and through Spark UI / Spark History P.S These demos do no h

Analytics 130

Analytics Analytics Data Science 130

IBM Data Science in Practice

FEBRUARY 5, 2023

This article gives an overview of the code experience for data scientists in Watson Studio on Cloud Pak for Data 4.6. We cover the new improved VS Code Desktop integration, JupyterLab- and RStudio- integration. VS Code desktop integration lets data scientists use a familiar IDE to run and debug code that runs on the Cloud Pak for Data cluster. This enforces permissions and privacy rules defined in Cloud Pak for Data, like when using built-in tools.

Python 130

Python Clustering Data Scientist Data Science 130

IBM Data Science in Practice

FEBRUARY 2, 2023

Energy is the biggest contributor to environmental pollution. With most companies having the objective of becoming carbon-free, they are making extra efforts to reduce the amount of energy usage in many aspects. According to the International Energy Agency[1] (IEA), the share of energy comes from different sources: 31% Oil, 24% Natural Gas, 27% Coal, 4% Nuclear Energy, 7% Hydroelectricity, and 7% Renewables.

Machine Learning 130

Machine Learning Machine Learning Algorithm Data Science 130

IBM Data Science in Practice

JANUARY 26, 2023

Co-authored by Carolyn Saplicki and Courtney Branson Many businesses today use the R programming language to build their machine learning models. Currently, R model deployments are not natively supported by WML. To use the R model on WML, you have to wrap your R script within a Python wrapper function. This function can then utilize the R script to evaluate new data on your R model and return the output to the Python function.

Machine Learning 130

Machine Learning Machine Learning Python Data Science 130

IBM Data Science in Practice

NOVEMBER 11, 2022

Unstructured content is constantly rising in volume these days. Handling this data and converting it into a structured manner is time-consuming and labor-intensive for employees. Using the topic modeling approach, a machine can sift through unlimited lists of unstructured content into similar documents. It will save businesses time and money. So everyone wants the best Topic Modeling tool which they can use directly with their unstructured data & got the best results.

Clustering 130

Clustering Algorithm AI AI 130

IBM Data Science in Practice

NOVEMBER 21, 2022

Leverage the Watson NLP library to build the best classification models by combining the power of classic ML, Deep Learning, and Transformed based models. [link] Text classification is one of the most used NLP tasks for several use cases like email spam filtering, tagging, and classifying content, blogs, metadata, etc. Some of the more specific use cases are Sentiment, Emotion, and Tone classification.

Deep Learning 130

Deep Learning Deep Learning Exploratory Data Analysis ML 130

IBM Data Science in Practice

NOVEMBER 28, 2022

As organizations steer their business strategies to become data-driven decision-making organizations, data and analytics are more crucial than ever before. Insights from data gathered across business units improve business outcomes, but having heterogeneous data from disparate applications and storages makes it difficult for organizations to paint a big picture.

Data Pipeline 130

Data Pipeline Data Quality Data Preparation Data Governance 130

IBM Data Science in Practice

DECEMBER 7, 2022

The IBM Watson NLP Library for Embed is a containerized library intended to be embedded into partner applications in order to provide natural language capabilities. This article will take you through the steps to start serving Watson NLP models using standalone containers. The idea is to build a container image that includes both the models to be served, as well as the runtime that serves the models.

Clustering 100

Clustering AI AI AWS 100

IBM Data Science in Practice

DECEMBER 7, 2022

One of the key elements that builds a data fabric architecture is to weave integrated data from many different sources, transform and enrich data, and deliver it to downstream data consumers. IBM Multicloud Data Integration helps organizations connect data from disparate sources, build data pipelines, remediate data issues, enrich data, and deliver integrated data to multicloud platforms where it can easily accessed by data consumers or built into a data product.

Data Pipeline 130

Data Pipeline Data Quality Data Preparation ETL 130

IBM Data Science in Practice

DECEMBER 8, 2022

IBM Federated Learning enables you to train a machine learning model across multiple decentralized parties holding local data sets, without sharing the local data sets. Such parties can be for example within an enterprise, within a consortium of enterprises, within multiple data centers or multiple clouds, or on edge devices. This allows to build a collective machine learning model without sharing data between the parties, therefore addressing data security, privacy, and regulatory compliance re

Machine Learning 130

Machine Learning Machine Learning Data Science 130

IBM Data Science in Practice

DECEMBER 12, 2022

Co-authored with: Matan Orbach, Yoav Katz, Noam Slonim Customer reviews of businesses and products have become a powerful tool for measuring customer satisfaction. They provide useful insights that help companies improve their services and assist consumers in decision-making. Sentiment analysis plays a key role in analyzing these reviews. It identifies the sentiment expressed in the text within an entire review document or even in a single sentence.

Deep Learning 130

Deep Learning Deep Learning Data Science Python 130

IBM Data Science in Practice

JANUARY 4, 2023

Context My team manages historical weather data at IBM. We have multiple systems that ingest, store and retrieve data. Every hour, the system takes a snapshot of global weather conditions and ingests it in parquet format (where each column is a weather variable) into a COS bucket. That means an hourly snapshot of data is stored as a separate set of files and to retrieve data for 24 hours the system has to read 24 different sets of hourly files.

Analytics 130

Analytics Analytics Python Clustering 130