Fundamentals of Scalable Data Science

The value of IoT can be found within the analysis of data gathered from the system under observation, where insights gained can have direct impact on business and operational transformation.Through analysis data correlation, patterns, trends, and other insight are discovered. Insight leads to better communication between stakeholders, or actionable insights, which can be used to raise alerts or send commands, back to IoT devices. With a focus on the topic of Exploratory Data Analysis, the course provides an in-depth look at mathematical foundations of basic statistical measures, and how they can be used in conjunction with advanced charting libraries to make use of the world’s best pattern recognition system – the human brain.Learn how to work with the data, and depict it in ways that support visual inspections, and derive to inferences about the data. Identify interesting characteristics, patterns, trends, deviations or inconsistencies, and potential outliers. The goal is that you are able to implement end-to-end analytic workflows at scale, from data acquisition to actionable insights.Through a series of lectures and exercises students get the needed skills to perform such analysis on any data, although we clearly focus on IoT Sensor Event Data. If you choose to take this course and earn the Coursera course certificate, you will also earn an IBM digital badge.

Objectives

  • Describe how basic statistical measures, are used to reveal patterns within the data
  • Recognize data characteristics, patterns, trends, deviations or inconsistencies, and potential outliers.
  • Identify useful techniques for working with big data such as dimension reduction and feature selection methods
  • Use advanced tools and charting libraries to: o Automatically store data from IoT device(s)
  • Improve efficiency of analysis of big-data with partitioning and parallel analysis
  • Visualize the data in an number of 2D and 3D formats (Box Plot, Run Chart, Scatter Plot, Pareto Chart, and Multidimensional Scaling)

Outline

Introduction to exploratory analysis Tools that support IoT solutions

  • Data storage solutions
  • ApacheSpark and how it supports the data scientist
  • Programming language options on ApacheSpark
  • Functional programming basics
  • Introduction of Cloudant
  • ApacheSparkSQL
  • Overview of end-to-end scenario
  • IBM Watson Studio (formerly Data Science Experience

 

Mathematical Foundations on Exploratory Data Analysis

  • Averages
  • Standard deviation
  • Skewness
  • Kurtosis
  • Covariance, Covariance matrices, correlation
  • Multidimensional vector spaces

Data Visualization

  • Plotting with ApacheSpark and python’s matplotlib
  • Dimensionality reduction
  • PCA

Audience

Computer Science and Engineering

Duration

5 Days
X