A word of advice for the Data Science aspirants who have just started or wish to start their Data Science journey? Shivam: Data science is all about ideas and experiments. It is all about trying those ideas and experiments, and re-iterating again and again until a successful stage is reached. It’s about developing a mindset … Read moreThe Data Scientist who rules the ‘Data Science for Good’ competitions on Kaggle.

“Methods that scale with computation are the future of AI” [1],“The two (general purpose) methods that .. scale …are search and learning.” [2] Prof Rich Sutton, Father of Reinforcement Learning in “The Bitter Lesson” TLDR; Hopsworks uses PySpark to parallelize machine learning applications across lots of containers, containing GPUs if you need them. PySpark’s stage-based … Read moreSay Hello to Asynchronous Search for PySpark

Common to all scientific theories is the ambition of deriving observable quantities starting from some abstract model. The parameters of the theory are usually assumed to be known, for instance, based on first principles, direct measurement or something more sophisticated like symmetry considerations. On the other hand, in the Big Data era, growing interest is … Read moreWhy the Gaussian distribution is a “natural” choice (Part 1)

The x-bar and R-chart are quality control charts used to monitor the mean and variation of a process based on samples taken in a given time. The control limits on both chats are used to monitor the mean and variation of the process going forward. If a point is out of the control limits, it … Read moreQuality Control Charts: x-bar chart, s-chart and Process Capability Analysis

NLP Chatbot Pattern matching is simple and quick to implement but it can only go so far. It needs a lot of pre-generated templates and is useful only for applications which expect a limited number of questions. xkcd Enter NLP! NLP is a collection of slightly advanced techniques which can understand a broad range of … Read moreHow to Build a Chatbot — A Lesson in NLP

Open Location Code has a library of open source code in different languages that generate and read Plus Codes. When BigQuery scripting launched, I figured creating something that encodes Plus Codes could be a neat way to introduce this new feature. It allows for easily encoding coordinates in bulk, with BigQuery bringing along a number … Read morePlus Codes (Open Location Code) and Scripting in Google BigQuery

In a typical multi-class classification, the neural network outputs scores for each class. Then we apply softmax function on the final scores to convert them into probabilities. And the data is classified into a class that have highest probability value(refer to the following image). Multiclass Classification But in sudoku, the scenario is different. We have … Read moreSolving Sudoku with Convolution Neural Network | Keras

I recently read an article suggesting that crime rates increase on full moons. Curious about the effect, I did a little research and it seems that the question of whether lunar cycles affect human behavior has been hotly debated in academic circles for decades (see Thakur and Sharma, 1984 and Rotton and Kelly, 1985 for … Read moreModeling Lunar Cycles in Tweets and Financial Markets using Facebook Prophet

“All you need is a plan, the road map, and the courage to press on to your destination” — Earl Nightingale In the previous part of this series we talked about how the road to AGI could be divided into perception and control. Within control, navigation and grasping are a crucial part of the roadmap … Read moreFrom Cups to Consciousness (Part 3): Mapping your home with SLAM

I recently went to a Data & Healthcare meetup to see how a renowned cancer research institute, Memorial Sloan Kettering Cancer Center (MSK), applies and uses Data Science. One of the first uses they mentioned was predictive modeling for post-hospital care. Whether it is from being discharged after an acute condition that required a hospital … Read moreData Science and Healthcare

The primary way to incorporate machine learning into applications is by deploying a trained model as a web API on cloud infrastructure. Running inference at scale requires an understanding of several engineering design principles that may have more to do with DevOps than Data Science. Running machine learning in production is not just about deploying … Read moreReal-time inference at scale on AWS

You introduced product recommendations on one section of your e-commerce website a few months ago and want to know if it is ‘working’. One of the simplest comparisons you can do is compare the spending of visitors who clicked on a recommendation with those visitors who didn’t. This is what you find. Website visitors who … Read moreLook Before You Leap —The Careful Use of Observational Data in Business (Part 1 of 2)

The inverse relationship between data appetite and data success Photo by Martin Adams on Unsplash A few years ago, it quickly became a cliche to say that every company was a tech company. Sure, every company is — in a way. But another trend has followed on the heels of that one, and this one … Read moreLet’s Get Back to Basics

One of the most integral parts of predictive modeling is pre-processing. Luckily, Lathe has us covered for this portion. This is what we’ll need to do so in Julia using Lathe: using Lathe.preprocess: StandardScalar, TrainTestSplit Now we need to get our data into a test set and a train set. Lathe makes this super easy … Read moreDeep Sea Correlation

Setting the right course and steering responsibly This article is coauthored by Joy Rimchala and Shir Meir Lador. Rapid adoption of complex machine learning (ML) models in recent years has brought with it a new challenge for today’s companies: how to interpret, understand, and explain the reasoning behind these complex models’ predictions. Treating complex ML … Read moreNavigating the Sea of Explainability

This post is a summary of the theory of nonlinear dynamics and chaos that I have recently learned from an online course by the Santa Fe Institute. All materials are credited to the institute alone. Complexity Explorer About the Course: In this course you’ll learn about the tools used by scientists to understand complex systems. … Read moreNonlinear Dynamics and Chaos

What’s wrong with modern convolutional networks and how can we fix them? In April 2019, a new computer vision paper appeared, titled “Making Convolutional Networks Shift-Invariant Again”. Why are modern convolutional networks not shift-invariant anymore, what does this mean, and how do we make them shift-invariant again? Most modern convolutional networks are not robust against … Read moreMaking Convolutional Networks Shift-Invariant Again

https://openai.com/blog/solving-rubiks-cube/ Meta-Learning describes the abstraction to designing higher level components associated with training Deep Neural Networks. The term “Meta-Learning” is thrown around in Deep Learning literature frequently referencing “AutoML”, “Few-Shot Learning”, or “Neural Architecture Search” when in reference to the automated design of neural network architectures. Emerging from comically titled papers such as “Learning to … Read moreThe Rise of Meta Learning

As You as You Can Be Image by Gerd Altmann from Pixabay What is a mind upload? It’s where a (future) device measures your mental state and uploads that to a computer, which then runs a simulation of your mental state and your brain’s functioning following this mental state. In other words, “you” are uploaded … Read moreOn Mind Uploads

Recently, more and more attention is focussed on the interpretability of the Machine Learning models mainly Deep Learning ones because of their black-box nature. One such important Deep Learning architecture used is Convolutional Neural Networks(CNNs) which has made a breakthrough in Computer Vision including image classification, object detection, semantic segmentation, Instance segmentation, image captioning etc. … Read moreDemystifying Convolutional Neural Networks using ScoreCam

With that said, over the last year or so we’ve spent a considerable amount of time reading, returning to and distilling a favourite field of ours: Reinforcement Learning (RL). For those interested in RL as a branch of AI, we’re open-sourcing an RL course that we built last year as an introduction for engineers & … Read moreFrom Zero to SOTA in Reinforcement Learning