Towards Data Science

I recently talked about deriving attributes from sitcom transcripts and determining the possibilities of learning the essence of making a popular television show at Data Science Salon, Los Angeles. This article would present the investigation which went in and what were the results I got post training on neural networks as well as leveraging pre-trained … Read more Can Data Science help us find what makes a hit television show?

As you could guess, nobody, no data engineer could have all these technologies on his CV. Even if somebody did have one-third of them on his CV, would you think that he is an expert on all of them? and if he is not an expert in all these technologies, how can he be technology-focused? … Read more Data Engineers are not just technology experts

Introduction Technology has evolved tremendously over the years, with the technology access gap shrinking year by year as individuals and businesses are more capable of accessing higher processing and compute capabilities, larger memory capacities, smarter and more efficient data storage technologies, and so on, whether it’s on-premise or in the cloud. Despite this, data volume … Read more Swarm Intelligence — Swarm-Based Dimensionality Reduction

To understand more, let’s have an example, suppose we have a square matrix with a dimension of 3 X 3, as illustrated below. To get the zeros in the lower-left corner of this matrix, we would do the following: What I’ve done here is that I divide 4 over 2 — which is called the … Read more Understanding Matrix Factorization for recommender systems

Now, we have an idea what type of data we are dealing with, let’s explore the core ideas of HDBSCAN and how it excels even when the data has: Arbitrarily shaped clusters Clusters with different sizes and densities Noise HDBSCAN uses a density-based approach which makes few implicit assumptions about the clusters. It is a … Read more Understanding HDBSCAN and Density-Based Clustering

Convolutional neural networks, or CNNs, form the backbone of multiple modern computer vision systems. Image classification, object detection, semantic segmentation — all these tasks can be tackled by CNNs successfully. At first glance, it seems to be counterintuitive to use the same technique for a task as different as Natural Language Processing. This post is … Read more CNN Sentiment Analysis

The purpose of this post is to introduce the concept of Deel Q Learning and use it to solve the CartPole environment from the OpenAI Gym. The post will consist of the following components: Open AI Gym Environment Intro Random Baseline Strategy Deep Q Learning Deep Q Learning with Replay Memory Double Deep Q Learning … Read more Deep Q Learning for the CartPole

Before jumping into the tree traversal algorithms, let’s define Tree as a data structure first. That will help you to grasp the concepts in a meaningful way. Tree is a hierarchical data structure which stores the information naturally in the form of hierarchy unlike linear data structures like, Linked List, Stack, etc. A tree contains … Read more 4 Types of Tree Traversal Algorithms

A Brain-Computer Interface (BCI) is a system that extracts and translates the brain activity patterns of a subject (humans or animals) into messages or commands for an interactive application. The brain activity patterns are signals obtained with Electroencephalography (EEG). The concept of controlling devices solely with our minds is nothing new. Science fiction and Hollywood … Read more Deep Learning in Brain-Computer Interface

The SCperf package from R contains several functions for multiple inventory planning and managing methods. For the following example, let’s consider the following values for the Newsvendor model variables (assuming the demand is normally distributed): D= 100 units, sd= 30 units, p = $4 per unit, and c = $1 per unit, and s = … Read more Newsvendor Inventory Problem with R

We used the code from this repo for knowledge distillation and modified it for training and evaluation on the MS Marco dataset. We initially trained a teacher bert-base-uncased network in PyTorch with the MS Marco training triples set. Then we used it as a teacher to train a smaller student BERT network with only 4 … Read more TinyBERT for Search: 10x faster and 20x smaller than BERT

Happy New Year! If data is the new oil, why do we experience so many backfires in our systems? In a prior 2-part article series titled Serverless Data Integration, the importance of data integration, particularly in the serverless realm was discussed in good detail. We concluded that data integration was a critical business need, that … Read more Architecting Serverless Data Integration Hubs on AWS for Enterprise Data Delivery (2020)

It appears that those with higher levels of education are less likely to respond to an offer. Similar to the first graph, it appears that customers with luxury vehicles are less likely to respond to an offer. How can we further explore the seemingly negative correlation between wealth and response rate? This graph further supports … Read more Predictive Analytics on Customer Behavior with Support Vector Machines (SVM)

In December my girlfriend and I went back to Uruguay and Argentina, from where we are from. Over there, her grandfather, a person I personally find fascinating -in the best possible way-, recommended me a book peppering it telling me it was one of the hardest books he had ever read in his life. Let … Read more It’s official: time doesn’t exist

As an additional benefit, the attack has only one hyperparameter (the maximum step length or trust region radius), and if one simply sets this parameter to its default value one can already reach very good results. I believe that this attack should be part of the standard toolbox of anyone trying to evaluate model robustness. … Read more Accurate, reliable and fast robustness evaluation

Time for Some Code Let’s get into the details. I should emphasize, though, that this is just one way that one can go about such a task using an autoencoder. There are other ways and technics to build autoencoders and you should experiment until you find the architecture that suits your project. These are the … Read more A Keras-Based Autoencoder for Anomaly Detection in Sequences

There are two things that make the posterior calculation expensive. First, we are computing the posterior for every single θ. Why do we have to calculate the posterior for thousands of thetas? Because you are normalizing the posterior (line 21). Even if you choose not to normalize the posterior, the end goal is to find … Read more Conjugate Prior Explained

How to prevent costly failures in high-risk data science applications The intrinsic risk for predicting customer churn on a smartphone plan is not the same as pricing a trillion dollars of mortgage-backed securities or detecting a pedestrian in a crosswalk for a self-driving car. In the first situation a wrong decision has an opportunity cost, … Read more Billion Dollar Data Science

In conclusion, Data Science, Machine Learning and the others, are very promising fields that are exciting, fun , and have endless applications. Despite there being a lot of practitioners, there is a shortage of qualified professionals in these areas. If you become one more name in that list of professionals, you can only expect quality … Read more Why learn Data Science in 2020?

What are the sources and significant examples of implicit bias in artificial intelligence? What can be done to minimize their consequences in establishing acceptable data practices? Image credit: Bayeté Ross Smith When we evoke the implicit bias of artificial intelligence, we are recognizing that machine learning reflects the unconscionable attitudes and stereotypes that influence managerial … Read more Addressing AI’s Hidden Agenda

Now, consider a very basic linear model with two variables. Temperature is the independent variable and we wish to find out the effect of temperature on sales (dependent variable). The twelve data points are in the form of (x,Y). The x-coordinate represents the temperature in degree Celsius and the Y-coordinate represents the sales in dollars. … Read more Introducing Linear Regression (Least Squares) the Easy Way

“Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making”[11] Big data analytics can be time-consuming, complicated, and computationally demanding, without the proper tools, frameworks, and techniques. When the volume of data is too high to process and analyze on a single … Read more Big Data Analytics: Apache Spark vs. Apache Hadoop

In an insightful Medium article which I strongly recommend you read if you’re serious in learning more about data science, Cassie Kozyrkov, the Chief Decision Scientist at Google, gave her so-called pithiest definition. Data science is the discipline of making data useful. Subject of Concern: Data From a personal perspective as a minimalist, I think … Read more A Minimalism Approach to Understand What Data Science Is

5 coding sniffs you must know if you are working in the Data Science industry “It was Friday evening. I clearly remember how excited I was to spend the rest of the day with my family. My parents had traveled to Bangalore for the first time and I already had plans of showing them the … Read more Are your coding skills good enough for a Data Science job?

The sequence of random variables {Yt: t = 0, ±1, ±2, ±3,…} is called a stochastic process and serves as a model for an observed time series.The complete probabilistic structure is defined by the set of all the distributions, but we do not have to deal with them. ( As we move forward, certain key … Read more Time Series Analysis: Basic Concepts

Splice Machine 3.0 adds geo-replication, Kubernetes support, time travel, Jupyter notebooks, in-DB machine learning model deployment & more Splice Machine is part of a relatively new segment of the database management systems (DBMS) market where transactions and analytics functionality converges. In July 2013, Gartner used the term HTAP or Hybrid Transaction Analytical Processing to describe … Read more What’s New in Splice Machine 3.0

A technique to transform a digital image that adopts the style of different image Introduction: Before we begin, let’s go to this website to get some inspiration. On the website, we choose a photo from the local computer (let’s assume the image named Joey.jpg). Let’s call this content image. Then we choose another image, say … Read more Neural Style Transfer using VGG model

Analysis of Generated Text and Model Training The 1558MB version of GPT-2 is able to generate coherent text for a variety of prompts. Interestingly, it is able to describe neural networks, deep learning, machine learning, and data science fairly accurately. However, these language models still lack intrinsic models of the world. For instance, it generates … Read more Interviewing the 1.5B GPT-2 model by OpenAI

How I became a data scientist with a PhD in theoretical physics There are a lot of stories out there of people who explain how they became data scientists but I found it hard when I did the transition to find one that corresponded to my profile. This is why I want to share my … Read more From theoretical physicist to data scientist

I would use three different models (+ baseline) to emulate step-by-step “work” on the task: Mean model (baseline) Random predictions Linear Regression Gradient Boosting over Decision Trees (LightGBM) In real-world problems, it is equivalent to continuously improving model whose changes are pushed to the repository. One should also define metrics that estimate how good the … Read more Continuous quality evaluation for ML projects using GitHub Actions.

“The powers that be no longer have to stifle information. They can now overload us with so much of it, there’s no way to know what’s factual or not. The ability to be an informed public is only going to worsen with advancing deep fake technology.” J. Andrew Schrecker All of us have heard Donald … Read more How Deepfake Technology Can Become More Dangerous Than a Nuclear Weapon

Solving sim2real transfer for specialized object detectors with no budget Deep learning has recently become the favored approach to object detection problems. However, like with many other uses of this technology, annotating training data is cumbersome and time-consuming, especially if you are a small company with a specific use-case. In this article, I present some … Read more Training Object Detectors with No Real Data using Domain Randomization

Girl at tablet: Photo by Hal Gatewood on Unsplash Cutting through the hype surrounding artificial intelligence, François Chollet, an AI researcher at Google, has proposed the Abstract and Reasoning Corpus (ARC), an intelligence test that could shape the course of future AI research. To date, there has been no satisfactory definition of artificial intelligence nor … Read more Who’s smarter? An IQ test for both AI systems and humans

Editor’s note: The Towards Data Science podcast’s “Climbing the Data Science Ladder” series is hosted by Jeremie Harris, Edouard Harris and Russell Pollari. Together, they run a data science mentorship startup called SharpestMinds. You can listen to the podcast below: Getting hired as a data scientist, machine learning engineer or data analyst is hard. And … Read more Mastering the data science job hunt

We can see that the connections are not particularly strong for either group (the diagonal line can be ignored as it shows correlation with itself and, thus, always equals to 1). To better visualize the connections and the differences, we can project these back onto the brain. #Getting the center coordinates from the component decomposition … Read more Using neural networks for a functional connectivity classification of fMRI data

v1.0 Abstract For years I’ve been looking for a theory that will unify my understanding of intelligence and related concepts. Among the ones I’d encountered most share some core ideas, so I’ve been trying to combine and distill them into a more formal and general framework. Here I want to propose PTI in attempt to … Read more Physical Theory of Intelligence

Twitter Data Science Interview Questions Twitter Interviews Twitter is known for its news and debates and holds the title of the SMS of the world. Created in 2006 by Jack Dorsey, Noah Glass, Evan Williams, and Biz Stone, it has grown to have more than 321 million active users per month as well as 1.6 … Read more The Twitter Data Scientist Interview

A Summary and Review of the New Strategy for AI On The Day of Its Launch 14th of January This day is special to me, because I have been covering most of the AI strategies in Europe, and today my home country has released their own national strategy. The Norwegian national strategy was released on … Read more The Norwegian National Strategy for Artificial Intelligence Has Launched!

Running a data science team is hard! We need to take inspiration wherever we can, and the culture of Nelson’s navy is one place to start. Portrait of Nelson by Lemuel Francis Abbott “no captain can do very wrong if he places his ship alongside that of the enemy” Admiral Horatio Nelson, before the battle … Read more Run your data science team like an Admiral …

How a team obtained private data, constructed a fake AI model, and got away with the money from a platform for adopting neglected pets The cheaters stole from Petfinder.my, a platform for adopting homeless and neglected pets. [pixabay image] Kaggle just announced that the 1st Place Team, Bestpetting[1], has been disqualified from the Petfinder.my competition … Read more Kaggle 1st place winner cheated, $10,000 prize declared irrecoverable