Towards Data Science

Enjoy sate-of-the art results in your Earth Observation tasks with simple coding and the Fastai’s deep learning API. Water pixels segmentation example in Jirau reservoir (Brazil), using U-Net architecture and Fastai-v2. Fastai is an open source deep learning library that adds higher level functionalities to PyTorch and makes it easier to achieve state-of-the-art results with … Read more How to create a DataBlock for Multispectral Satellite Image Segmentation with the Fastai-v2…

3. UNIVARIATE ANALYSIS: Univariate analysis is the simplest form of analyzing the data. It is the analysis of only one feature from a set of different features. It normally gives us a description of a particular feature. It doesn’t deal with cause or relationship, it just takes the data, summarizes it, and represents it in … Read more Exploratory Data Analysis

5. Model Implementation Within this section, we will implement the AlexNet CNN architecture from scratch. Through the utilization of Keras Sequential API, we can implement consecutive neural network layers within our models that are stacked against each other. Here are the types of layers the AlexNet CNN architecture is composed of, along with a brief … Read more Implementing AlexNet CNN Architecture Using TensorFlow 2.0+ and Keras

The human resources industry relies heavily on a wide range of assessments to support its functions. In fact, to ensure unbiased and fair hiring practices the US department of labor maintains a set of guidelines (Uniform Guidelines) to aid HR professionals in their assessment development ventures. Personality assessments are often used in selection batteries to … Read more Assessing Statistical Rigor of Employee Surveys

Population density is a crucial concept in urban planning. Theories on how it affects economic growth are divided. Some claim, as Rappaport does, that an economy is a form of “spatial equilibrium”: that net flows of residents and employment gradually move to be balanced with one another. The thought that density has some sort of … Read more Visualizing the Philippines’ Population Density using GeoPandas

Examining why logistic regressions are more suitable for classification problems than linear regressions Image by Trist’n Joseph It is no longer a question as to whether data science has taken over the world. Data has grown to become one of the world’s valuable resource, so much so that almost every decision made within big companies … Read more How Social Media Companies Know What to Show You: Hello Logistic Regressions!

Must Read to Build Good Classification ML Models Photo by Martin Sanchez on Unsplash There are different types of problems in machine learning. Some might fall under regression (having continuous targets) while others might fall under classification (having discrete targets). Some might not have a target at all where you are just trying to learn … Read more Building and Evaluating Classification ML Models

Text Prediction using RNN Problem: Given the phrase “34+17”, the model should predict the next word in the sequence “51”. The input and output is a sequence of characters which in turn an arithmetic expression of two numbers and its result. Thus our data is represented as a sequence of two words expression and result. … Read more Making a RNN model learn Arithmetic Operations

The Heart of Dimensionality Reduction Principal Component Analysis is a powerful method, but it often fails in that it assumes that the data can be modelled linearly. PCA expressed new features as linear combinations of existing ones by multiplying each by a coefficient. To address the limitations of PCA, various techniques have been created by … Read more Manifold Learning [t-SNE, LLE, Isomap, +] Made Easy

A brief history of how I learned about the forgotten core of artificial intelligence. In my early days working as a data scientist in AI, I was taught one thing above all: you need more and better data to feed your learning systems. And I have dedicated myself to finding that data. I was lured … Read more How Machine Learning made AI forget about Knowledge Representation and Reasoning

Figures 2 and 3 present the percentiles of authors’ publications and citations, respectively. We see a similar positively skewed distribution on the CDF’s for citations and publications. Most authors have published less than 200 scientific papers and amassed less than 2500 citations across all of them. This makes intuitive sense; the majority of scientists are … Read more Who is An Expert in Scientific Research?

While adding new users is essential, it is ultimately active users who contribute to revenue by clicking ads or making a purchase. For this reason, it’s essential to focus on these users, separating them from the rest of the user base. Active users are typically a small fraction of cumulative registered users: the average mobile … Read more Business Data Science

Exploratory Data Analysis Let’s do some basic data analysis. We are going to look at the distribution of variables using histograms first. import matplotlib.pyplot as plt plt.figure(figsize=(10,10)) df.hist() plt.tight_layout() What we can notice straight away is the fact that some variables are not continuous. Actually, only five features are continuous:’’age’, ‘chol’, ‘oldpeak’, ‘thalach’, ‘trestbps’ whereas … Read more Build your first Random Forest classifier

I find it somewhat difficult to watch tools like this one automate machine learning, and decrease the need for machine learning engineers in small and medium-sized companies. The reasons are many, but the biggest is that the purpose of machine learning was to automate other professions, but we’ve managed to automate machine learning with machine … Read more Introducing ObviouslyAI — No-Code Machine Learning Solution

With this our exploratory data analysis (EDA) for our gestures dataset is completed. We can proceed to build the gestures training model for appropriate gestures prediction. Let us look at the code block below to understand the libraries we are importing as well as set the number of classes along with their dimensions and their … Read more Human Emotion and Gesture Detector Using Deep Learning: Part-2

Summary of a research paper outlining how our vision systems can fall prey to adversarial attacks much like a Neural Net Image by Pixabay Neural Networks are exceptionally good at recognizing objects shown in an image and in many cases, they have shown superhuman levels of accuracy(E.g.-Traffic sign recognition). But they are also known to … Read more Adversarial attacks on the human visual system

We all had some sort of experience with linear regression. It’s one of the most used regression techniques used. Why? Because it is simple to explain and it is easy to implement. But what happens when you have more than one variable? How can you deal with this increased complexity and still use an easy … Read more Multiple linear regression with interactions unveiled by genetic programming

All Images are generated by the author And the Perils of Post-Selection Inference As there is a lot of buzz about AutoML, I decided to write about the original AutoML; step-wise regression and best subset selection. Then I decided to ignore step-wise regression because it is bad and should probably stop being taught. That leaves … Read more Origins of AutoML: Best Subset Selection

Initial Idea In 1972, J.D. Croston published “Forecasting and Stock Control for Intermittent Demands,” an article that introduced a new technique to forecast products with intermittent demand. His idea could be summarized in three simple steps:– Evaluate the average demand level when there is a demand occurrence.– Evaluate the average time between two demand occurrences.– … Read more Croston forecast model for intermittent demand

How you would benefit from Milvus, an open-source project for data scientists When talking about the open-source AI projects, people would think of the model framework projects like Google TensorFlow, PyTorch, etc. Since the model framework is the critical component while training the AI models, those projects usually receive the most attention. But Artificial Intelligence … Read more Passing the Turning Point of AI Transformation

Summing up, learning how to apply Interpolation Search will enrich your knowledge not only about algorithms but also as a programmer. Undoubtedly, it will have a place in your programming toolkit. However, it should always be remembered: Similar to Binary Search, Interpolation Search only works in sorted arrays. You need to have direct access to … Read more Demystifying Interpolation Search

Markov decision process is the fundamental problem which we try to solve in reinforcement learning. But, what is the definition of Markov Decision Process? Markov Decision Process or MDP is a formulation of sequential interaction between agent and environment. Here, the learner and decision maker is called agent, where the thing it interacts with is … Read more The Fundamentals of Reinforcement Learning

Single Shot Detectors Deep Learning at FAU. Image under CC BY 4.0 from the Deep Learning Lecture These are the lecture notes for FAU’s YouTube Lecture “Deep Learning”. This is a full transcript of the lecture video & matching slides. We hope, you enjoy this as much as the videos. Of course, this transcript was … Read more Segmentation and Object Detection — Part 4

Throughout my 10-year career, I have seen people often spend their time and energy in passionate debates about what data science can deliver, and what data scientists do or do not do. I submit that these are the wrong questions to focus on when you are looking to hire for your data department. In actuality, … Read more So you are ready to hire a data scientist?

Rust uses FFI (Foreign Function Interface) to interact with C/C++, but if the library is large, writing FFI code becomes monotonous, since it involves simply copying and pasting every function from one place to another. We use the famous and useful bindgen crate in Rust to help us create the FFI code. To know more … Read more Create a Rust Client for ROS2 from Scratch. Part 0: Integrate C API to Create ROS2 Node

Natural language processing is more approachable than ever Natural language processing (NLP) is everywhere lately, with OpenAI’s GPT-3 generating as much hype as we’ve ever seen from a single model. As I’ve written about before, the flood of projects being built on GPT-3 is not down to just its computational power, but its accessibility. The … Read more A list of things you can build with language models — not just GPT-3

As the previous code snippet shows, I created the user/movie profile based on the existing users’ rating records in history. It has not entirely solved the cold start problem yet nevertheless because the system still has no idea what to do for the new users or with the new movies. I will tell you how … Read more Movie Recommendation System based on MovieLens

Source: pixabay Understand the basic concepts of survival analysis and what tasks it can be used for! In our extremely competitive times, all businesses face the problem of customer churn/retention. To quickly give some context, churn happens when the customer stops using the services of a company (stops purchasing, cancels the subscription, etc.). Retention refers … Read more Introduction to Survival Analysis

Actually not. This is not a text about you. But, as Gladwell puts it in Outliers, if you find yourself being that type of outlier, you’re quite lucky. And rare. Photo by Daniel Reche from Pexels According to Meriam-Webster, an outlier is: “a statistical observation that is markedly different in value from the others of … Read more What is an outlier?

Dask has been reviewed by many and compared to various other tools, including Spark, Ray and Vaex. Developed in coordination with other community projects like Numpy, Pandas, and Scikit-Learn, it is definitely a great tool for scaling machine learning. Hence, the purpose of this article is not to compare the pros and cons of Dask … Read more Scalable Machine Learning with Dask on Google Cloud

From an introduction to modeling for medical diagnosis Photo by National Cancer Institute on Unsplash I am feeling sick. Fever. Cough. Stuffy nose. And it’s wintertime. Do I have the flu? Likely. Plus I have muscle pain. More likely. Bayesian networks are great for these types of inferences. We have variables, some whose values have … Read more Modeling with Bayesian Networks

Create an Object Detector from a pretrained Image Classifier in PyTorch The task of Object Detection is to detect and spatially identify (using bounding boxes etc.) various objects in an image, whereas Image Classification tells whether or not an image contains certain objects without any notion of where exactly they are located. In this post, … Read more Using FCNN Receptive Fields for Object Detection

Creativity involves breaking out of expected patterns in order to look at things in a different way. — Edward de Bono To come up with awesome and unique AI, ML, or data science ideas, one must be creative and willing to attempt things that have never been done. You should have the desire to birth … Read more How To Come Up With Amazing AI, ML or Data Science Project Ideas.

Practical use of topic modeling We discuss the ways in which topic modeling can be used to analyze various text posts on social media and the benefits of doing so. Image courtesy of Alina Grubnyak on Unsplash There is a substantial amount of data generated on the internet every second — posts, comments, photos, and … Read more Social media and topic modeling: how to analyze posts in practice

NASA provides an extensive library of data points that they’ve captured over the years from their satellites. These datasets include temperature, precipitation and more. NASA hosts this data on a website where you can search and grab information as needed, whether you want the data for the whole world or a specific area. The user … Read more Getting NASA data for your next geo-project

Error Handling When you misuse a function, it should throw you an error. For example, check out the function float that returns a floating-point from a number or string. When you pass the function float an integer value, the corresponding float is returned. Similarly, if you give it the string 8.7 it will return the … Read more Lambda Functions with Example and Error Handling

Obvious, but let’s spell it out: Bad space utilization (numbers as strings waste space) No type or structural checking (chars could wind up in numeric fields) CSV — no metadata/header info, JSON — repeated meta/formatting No native compression of repeating values No native indexing /search ability Note CSV and JSON have distinct advantages of being … Read more Columnar Stores — When/How/Why?

Once you have all your data organized the fun can begin. We will set the root directory of where your data is stored. This is the path of the folder where your test, train, and val folders reside. When we save our trained models later they will be saved in this directory. # Set path … Read more Deep Learning Image Classification with Fastai

A Guideline to Make the Best Use of FB Prophet for Time Series Forecasting Img from unsplash via link Time series prediction is one of the must-know techniques for any data scientist. Questions like predicting the weather, product sales, customer visit in the shopping center, or amount of inventory to maintain, etc – all about … Read more Crime Rate Prediction using Facebook Prophet