Towards Data Science

Recently I had a chance to work with a really cool road crack detection dataset as part of my research. A company (lets call it Ministry of Road Cracks and Other Important Stuff (MRCOIS for short) 😑) was seeking an autonomous system to localize the road cracks and classify them according to 3 crack severity … Read moreApplication Of Deep Learning In Identifying Road Cracks

It’s the final frontier in artificial intelligence. It’s the star of countless films and novels, both the greatest villain and hero of modern-day fantasies. I’m talking about true “intelligent” machines, sometimes called “hard” AI, or “general intelligence”. That is, an artificial intelligence that is intelligent “like us”, that is “conscious” or “self aware”. This is … Read moreHow Computers Think

I am currently part of the Ryerson Vision Lab, where I am working towards achieving my master’s. Over time, I have made a lot of friends in the lab, such as Jason and Matthew. Image by Sasin Tipchai from Pixabay From time to time, I’ll catch a glimpse of their monitors and see what they’re … Read moreHow I Got Better at Machine Learning

Dueling Network Architectures for Deep Reinforcement Learning Let’s go over some important definitions before going through the Dueling DQN paper. Most of these should be familiar. Given the agent’s policy π, the action value and state value are defined as, respectively: The above Q function can also be written as: The Advantage is a quantity … Read moreDueling Deep Q Networks

Photo by Ahmed Hasan on Unsplash Wikipedia tells us that Machine learning is, “a field of computer science that gives computers the ability to learn without being explicitly programmed.” It goes on to say, “machine learning explores the study and construction of algorithms that can learn from and make predictions on data — such algorithms … Read moreMachine learning: introduction, monumental failure, and hope

I will be sharing my perspective on which is actually more sought after in the current industry. Let me ask you one question. If you were the tech lead of data science, and there already has a lot of Ph.D. people working for you, at the same time, you would like to expand your team. … Read moreMath Vs Coding: Data Science

Why treating analytics like a second-class citizen will hurt you This article is an extended 2-in-1 remix of my HBR article and TDS article about analysts. Be careful which skills you put on a pedestal, since the effects of unwise choices can be devastating. In addition to mismanaged teams and unnecessary hires, you’ll see the … Read moreData Science’s Most Misunderstood Hero

Machine Learning is the crown of Data Science; Supervised learning is the crown jewel of Machine Learning. Supervised learning is the machine learning task or process of producing a function that predicts output variables. It has been adopted widely in the industry. For example, banks apply supervised models to detect credit card fraud. Quantitative traders … Read moreClassifying Rare Events Using Five Machine Learning Techniques

After 2 years of being a self-learner who taught himself everything he wanted and another year of being lost, unmotivated and kind of depressed, I have come to realize that — Teaching yourself anything can be difficult, draining and quite a lonely process But when you care about what you are learning, you change it … Read moreDifficulties with self-learning and 3 systems to solve them

What is Data, How can It be Used in Machine Learning and How is It Gathered? After having made a post asking for topics to be explored on a group on Facebook about artificial intelligence and deep learning one of the most interesting ones that emerged was training data. The question was: “Can you try … Read moreArtificial Intelligence and Training Data

A more advanced method of doing EDA with ‘ggplot2’ and ‘tidyverse’ ggplot2 In my previous article, ‘Exploratory Data Analysis in R for beginners (Part 1)’, I have introduced a basic step-by-step approach from data importing to cleaning and visualization. Here is a quick summary of Part 1: Import data appropriately with fileEncoding and na.strings arguments. … Read moreExploratory Data Analysis in R for beginners (Part 2)

Running algorithms to get the most accurate results This article is the last of my series, on the Housing dataset. For the uninitiated, I had already covered EDA and Feature Engineering in previous two articles. Summarising the work so far- we covered the awfully mundane work of data munging in EDA and meticulous re-engineering of … Read moreMachine Learning- Predicting House prices with Regression

Classification and object detection are the main parts of computer vision. Classification is finding what is in an image and object detection and localisation is finding where is that object in that image. Detection is a more complex problem to solve as we need to find the coordinates of the object in an image. To … Read moreStep-by-Step R-CNN Implementation From Scratch In Python

Computer Vision for Sports Straight-to-the-point guide to building a computer vision model to detect players and ball in overhead camera images. We will be using the RetinaNet model as describe in the Focal Loss for Dense Object Detection paper by Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár. We will also use Fizyr’s … Read moreSoccer Player Detection in Overhead Images using RetinaNet

Harnessing the most recent advances in NLP for time series forecasting and classification Transformers (specifically self-attention) have powered significant recent progress in NLP. They have enabled models like BERT, GPT-2, and XLNet to form powerful language models that can be used to generate text, translate text, answer questions, classify documents, summarize text, and much more. … Read moreAttention for time series classification and forecasting

An explanation of Gated Recurrent Units (GRUs) with the math behind how the loss backpropagates through time. Read this story without paywall Source : xkcd The GRU Network, or Gated Recurrent Units were initially proposed Cho et. al, 2014 and is a very interesting type of recurrent neural network. It improved on the simple RNN … Read moreForward and Backpropagation in GRUs — Derived | Deep Learning

The holy grail of AI, the pinnacle of human achievement, the ultimate weapon, our future master, and the end of the world as we know it. Is that plain enough? In the words of Douglas Adams, Don’t Panic. Siri, Cortana, Alexa and Google Assistant are not plotting to take over the human race. Let’s be … Read moreArtificial General Intelligence in plain English

Data Preprocessing First, patients under 16 years old and those who stayed in the ICU less than 7 hours are removed from the dataset. Different cleaning steps were needed for each type of dataset used: diagnoses, treatments, past medical history, periodic vital signs, aperiodic vitals, and lab results. Each was loaded and pre-processed independently according … Read moreDeep Prognosis: Predicting Mortality in the ICU

Python (and soon JavaScript with TensorFlow.js) is a dominant language for Machine Learning. What about SQL? There is a way to build/run Machine Learning models in SQL. There could be a benefit to run model training close to the database, where data stays. With SQL we can leverage strong data analysis out of the box … Read moreMachine Learning with SQL

How to automatically prune nodes in neural networks? Paper summary of Group Sparse Regularization for Deep Neural Networks With the advancement in hardware technologies and thereby reduced costs, training large neural networks has become a lot easier. Due to this, it has become common to throw in large networks for even simple tasks. Having too … Read moreGroup Sparse Regularization for Deep Neural Networks

First, you’ll need a RaspberryPi (or any similar clone) with platypush. I assume that you’ve already got platypush installed and configured. If not, please head to my previous article on how to get started with platypush. You’ll also need a relational database installed on your device. The example in this article will rely on PostgreSQL, … Read moreHow to build your home infrastructure for data collection and visualization and be the real owner…

Customer lifetime values (CLV) are a critical metric considered to be a company’s north star metric and all in-compassing KPI, used to inform: marketing campaigns — how much should we spend to acquire customers? customer segmentation — who are our most valuable customers and what are their demographic and behavioral traits? overall health of the … Read moreCalculating Customer Lifetime Values using a Shifted-Beta-Geometric model

As a data scientist, a big part of your job is obtaining the data that you need to operate on. In many companies, that data will be stored in one or more relational databases. The goal of this article is to introduce relational databases — what they are and why they’ve historically been so popular. … Read moreAn Introduction to the Relational Database

PREDICTION is an essential component to successful inventory management in retail. Identifying and understanding past trends in sales transactions can be used for defining a better marketing strategy. Here, I will show how to combine forecasts using the concepts of Clustering. Clusters of items are identified based on the similarity in their sales forecasts and … Read moreClustering & Machine Learning Combination in Sales Prediction

Making sense of recommendations provided by a collaborative filter based recommendation system in medical context With the rise of emerging analytical techniques, an effective recommendation system has become a game changer in the world of data science. Companies like Netflix and Amazon have surpassed their competitors by building some reliable recommendation systems. But these black … Read moreDecoding the output of a hybrid recommendation system

Almost 72 years ago, acclaimed American writer Kurt Vonnegut came up with a novel method for graphing the plot lines of stories as part of his master’s thesis in anthropology. Although his work was ultimately rejected by the University of Chicago “because it was so simple and looked like too much fun,” according to Vonnegut, … Read moreVisualizing the Emotional Arcs of Movie Scripts Using Rule-Based Sentiment Analysis

Using the dataset by Expedia on Kaggle, I will attempt to build a machine learning model to predict the group of hotels that a customer will make a reservation. The dataset contains a log of customer behaviour including what they searched for, if any reservation were made, and if the search was a travel package. … Read morePredicting Hotel Bookings on Expedia

A classifier trained on 320,000 classes and 90 million images yields surprisingly accurate results Credit: Gado Images AI and Machine Learning are usually all about computers and cutting-edge tech. Self-driving cars! Smart speakers! Robots stealing your job! But what if there was an AI-driven system that took you out of the world of tech and … Read morePlantsnap and Imagga Use Machine Learning to Put a Botanist in Your Pocket

Feodora Chiosea/iStock Gender equality is a highly debated and an important topic, and it deserves to be one. As our society evolves, women’s role in our society also evolves. However, not all the time norms in our society are keeping up with the change. In this article, we will look at a subtree of gender … Read morePlease mind the Gender Pay Gap

Introducing his students to the study of the human brain Jeff Lichtman, a Harvard Professor of Molecular and Cellular Biology, once asked: “If understanding everything you need to know about the brain was a mile, how far have we walked?”. He received answers like ‘three-quarters of a mile’, ‘half a mile’, and ‘a quarter of … Read moreSilicon Valley’s Brain-Meddling: A New Frontier For Tech Gadgetry

Check out part 1 (here)and part 2 (here) of this series Author: Dhruv Nair, data scientist, Comet.ml In the last part of our series on uncertainty estimation, we addressed the limitations of approaches like bootstrapping for large models, and demonstrated how we might estimate uncertainty in the predictions of a neural network using MC Dropout. … Read moreEstimating Uncertainty in Machine Learning Models — Part 3

Data Science from the ground up Building an intuition for how KNN models work Data science or applied statistics courses typically start with linear models, but in its way, K-nearest neighbors is probably the simplest widely used model conceptually. KNN models are really just technical implementations of a common intuition, that things that share similar … Read moreThe Basics: KNN for classification and regression

I love seeing data scientists using advanced packages, creating dazzling exhibits and experimenting with different algorithms. Data scientists can keep the computer burning for a whole day. A cool T-shirt, a cup of coffee and a laptop — that’s all he or she needs. While as crazy as their titles sound, some novice data scientists … Read moreAvoid These Deadly Modeling Mistakes that May Cost You a Career

Kaplan Meier analysis is often misused when analyzing survival data. This article highlights common Kaplan Meier errors and provides suggestions to improve prognostic factor research. Photo Credit: Louis Reed A patient’s prognosis refers to the risk of future medical outcomes such as surgery complications, tumor recurrence, or death. Prognostic factor research is the study of … Read moreKaplan Meier Mistakes

Data scientists generally work in the back end. Building an interactive app is not a strength of mine, but it can be a great way to show off the work that you can do. In this post, I will discuss some tips for creating a Flask App that I have learned while building a website … Read moreFlask App for Data Science

4 different sources, 2 different methods Air pollution is a big issue during the winter months in Ulaanbaatar, Mongolia. The effects of air pollution are quite severe, with those living in the most polluted areas suffering from lowered lung function, increased rates of respiratory infection, and shortened lifespans. In an attempt to solve this issue, … Read moreHow air pollution is measured in Ulaanbaatar

A word of advice for the Data Science aspirants who have just started or wish to start their Data Science journey? Shivam: Data science is all about ideas and experiments. It is all about trying those ideas and experiments, and re-iterating again and again until a successful stage is reached. It’s about developing a mindset … Read moreThe Data Scientist who rules the ‘Data Science for Good’ competitions on Kaggle.

“Methods that scale with computation are the future of AI” [1],“The two (general purpose) methods that .. scale …are search and learning.” [2] Prof Rich Sutton, Father of Reinforcement Learning in “The Bitter Lesson” TLDR; Hopsworks uses PySpark to parallelize machine learning applications across lots of containers, containing GPUs if you need them. PySpark’s stage-based … Read moreSay Hello to Asynchronous Search for PySpark

Common to all scientific theories is the ambition of deriving observable quantities starting from some abstract model. The parameters of the theory are usually assumed to be known, for instance, based on first principles, direct measurement or something more sophisticated like symmetry considerations. On the other hand, in the Big Data era, growing interest is … Read moreWhy the Gaussian distribution is a “natural” choice (Part 1)

The x-bar and R-chart are quality control charts used to monitor the mean and variation of a process based on samples taken in a given time. The control limits on both chats are used to monitor the mean and variation of the process going forward. If a point is out of the control limits, it … Read moreQuality Control Charts: x-bar chart, s-chart and Process Capability Analysis