What Kagglers are using for Text Classification

Advanced NLP techniques for deep learning With the problem of Image Classification is more or less solved by Deep learning, Text Classification is the next new developing theme in deep learning. For those who don’t know, Text classification is a common task in natural language processing, which transforms a sequence of a text of indefinite length … Read more What Kagglers are using for Text Classification

Is a Picture Worth A Thousand Words?

Dec 16, 2018 Source: Dark Reading Background Our project was inspired by Jamie Ryan Kiros who created a model trained on 14 million romance passages to generate a short romantic story for a single image input. Similarly, the ultimate goal of our project was to output a short story for children. “neural-storyteller is a recurrent neural … Read more Is a Picture Worth A Thousand Words?

Getting Started with TensorFlow in Google Colaboratory

Opening up a Colab Notebook When using Colab for the first time, you can launch a new notebook here: Once you have a notebook created, it’ll be saved in your Google Drive (Colab Notebooks folder). You can access it by visiting your Google Drive page, then either double-click on the file name, or right-click, and then … Read more Getting Started with TensorFlow in Google Colaboratory

Develop a NLP Model in Python & Deploy It with Flask, Step by Step

Flask API, Document Classification, Spam Filter By far, we have developed many machine learning models, generated numeric predictions on the testing data, and tested the results. And we did everything offline. In reality, generating predictions is only part of a machine learning project, although it is the most important part in my opinion. Considering a system … Read more Develop a NLP Model in Python & Deploy It with Flask, Step by Step

Logic Theory —Basic Notation

The origin of logic theory starts at the concept of an argument. The majority of logic textbooks contain an opening, central definition for an argument — one that likely sounds much like the following: An argument contains one or more special statements, called premises , offered as a reason to believe that a further statement, called the conclusion, … Read more Logic Theory —Basic Notation

Advanced Queries With SQL That Will Save Your Time

Yes, SQL still exists During the years of working with telecom data my folder with code snippets collected a lot of reusable examples. And it is not about “SELECT * FROM Table1”, I am talking about finding and handling or removing duplicate values, selecting top N values from each group of data within same table, shuffling … Read more Advanced Queries With SQL That Will Save Your Time

Art of Generative Adversarial Networks (GAN)

Dec 16, 2018 Art of Generative Adversarial Networks Code link for all the work mention in the post:- We had this pleasure of working on Generative adversarial network project for our final project for Business Data Science in our curriculum. Though we could have chosen any other subject as our final project yet we went … Read more Art of Generative Adversarial Networks (GAN)

ProGAN: How NVIDIA Generated Images of Unprecedented Quality

Progressively growing GANs enables them to get bigger and more stable The people in the high resolution images above may look real, but they are actually not — they were synthesized by a ProGAN trained on millions of celebrity images. “ProGAN” is the colloquial term for a type of generative adversarial network that was pioneered at NVIDIA. It … Read more ProGAN: How NVIDIA Generated Images of Unprecedented Quality

Ways to Improve a Map Visualization

How to take a map visualization to the next level. First, I will cover two reasons why visualizing data using maps is often compelling to an audience. Then, I will cover three tips that will help you make the transition from good to exceptional when building map visualizations. Why Use a Map for a Data Visualization? … Read more Ways to Improve a Map Visualization

Robots that Reason

Inorganic knowledge traditions with model-based reinforcement learning This essay explores the concept of inorganic knowledge traditions capable of sequential improvement using model based reinforcement learning Many behavioral economists presently believe that there are two primary methods used by humans for strategic decision making. One is fast, intuitive and unconscious — what has been called System 1 thinking. … Read more Robots that Reason

Simple House Price Predictor using ML through TensorFlow in Python

The profession of reality is moving into the 21st century, and as you can imagine home listings are flooding the internet. If you have ever looked at buying a home, renting an apartment, or just wanted to see what the most expensive home in town is (we have all been there), then chances are you … Read more Simple House Price Predictor using ML through TensorFlow in Python

Regression Analysis: Linear Regression

3. Model Building in R I have used the dataset which contains the details of 2,201 flights. The descriptions of each variable are as below. 3.1) Datasets schedtime : the scheduled time of departure (using the 24-hour clock) carrier : the two-letter code indicating which airline operated the flight deptime : the actual departure time dest : the three-letter code … Read more Regression Analysis: Linear Regression

What’s the fuss about Regularization?

As a newbie to machine learning most people get excited when their training error starts reducing. They try hard further and it starts reducing even further, their excitement knows no bounds. They show their results to master Oogway ( elderly wise tortoise in Kungfu Panda) and he calmly says well not a good model you … Read more What’s the fuss about Regularization?

Finding Local Events Using Twitter Data

Project by David Chen, Ashwin Gupta, Shruthi Krish, Raghav Prakash, Wei Wang Twitter is a social media platform that millions of users use to share updates about their lives. Often, these tweets are about local events happening around the user. Though news agencies report on local events, the time it takes an agency to learn … Read more Finding Local Events Using Twitter Data

Predicting hospital length-of-stay at time of admission

Exploring an important healthcare performance metric Photo by Hush Naidoo on Unsplash Project Overview Predictive analytics is an increasingly important tool in the healthcare field since modern machine learning (ML) methods can use large amounts of available data to predict individual outcomes for patients. For example, ML predictions can help healthcare providers determine likelihoods of disease, … Read more Predicting hospital length-of-stay at time of admission

Using Markov Chain Monte Carlo method for project estimation

Using TensorFlow probability for Hamiltonian Sampling Free photo from https://pixabay.com One type of criticism I received for the previous work on project estimation is that the log-Normal distribution has short tails. And this is true, despite all the benefits of log-Normal distribution. The reason is very simple: when fitting the data to the distribution shape … Read more Using Markov Chain Monte Carlo method for project estimation

Processing Time Series Data in Real-Time with InfluxDB and Structured Streaming

This article focuses on how to utilize a popular open source database “Influxdb” along with spark-structured streaming to process, store and visualize data in real time. Here, we will go in detail over how to set up a single node instance of Influxdb, how to extend the Foreach writer of SPARK to use it to … Read more Processing Time Series Data in Real-Time with InfluxDB and Structured Streaming

Modeling tree height and basal area in the Finger Lakes National Forest, NY

I tried my hand at using the R package, randomForest to create two regression models for tree height and basal area based off some lidar and field-collected data in the Finger Lakes National Forest, NY. Disclaimer: this project was my first real taste of R. Earlier in the semester I had done some simple learning into … Read more Modeling tree height and basal area in the Finger Lakes National Forest, NY

Text Generation Using RNNs

Generate characters from Alice in Wonderland Introduction Text generation is a popular problem in Data Science and Machine Learning, and it is a suitable task for Recurrent Neural Nets. This report uses TensorFlow to build an RNN text generator and builds a high-level API in Python3. The report is inspired by @karpathy ( min-char-rnn) and … Read more Text Generation Using RNNs

Introduction to Interactive Time Series Visualizations with Plotly in Python

Introduction to Plotly Plotly is a company that makes visualization tools including a Python API library. (Plotly also makes Dash, a framework for building interactive web-based applications with Python code). For this article, we’ll stick to working with the plotly Python library in a Jupyter Notebook and touching up images in the online plotly editor. When … Read more Introduction to Interactive Time Series Visualizations with Plotly in Python

The Ultimate NanoBook to understand Deep Learning based Image Classifier

The first and most important step of our journey: As I have said before, we are going to simply ask questions that will guide us to build an image classifier. For the sake of brevity, we will call Image Classifier an ICNow, we are ready to start our journey. So let us ask the first question: … Read more The Ultimate NanoBook to understand Deep Learning based Image Classifier

Applying GANs to Super Resolution

SRGAN Results from Ledig et al. [3] Generative adversarial networks (GANs) have found many applications in Deep Learning. One interesting problem that can be better solved using GANs is super-resolution. Super-resolution is a task concerned with upscaling images from low-resolution sizes such as 90 x 90, into high-resolution sizes such as 360 x 360. In this … Read more Applying GANs to Super Resolution

A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way

Artificial Intelligence has been witnessing a monumental growth in bridging the gap between the capabilities of humans and machines. Researchers and enthusiasts alike, work on numerous aspects of the field to make amazing things happen. One of many such areas is the domain of Computer Vision. The agenda for this field is to enable machines … Read more A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way

Dealing With Class Imbalanced Datasets For Classification.

Skewed datasets are not uncommon. And they are tough to handle. Usual classification models and techniques often fail miserably when presented with such a problem. Although your model could get you to even a 99% accuracy on such cases, yet, if you are measuring yourself against a sensible metric such as the ROC Auc score, … Read more Dealing With Class Imbalanced Datasets For Classification.

Google Landmark Recognition using Transfer Learning

Image classification with 15k classes! Project by Catherine McNabb, Anuraag Mohile, Avani Sharma, Evan David, Anisha Garg Dealing with a large number of classes with very few images in many classes is what makes this task really challenging! The problem comes from a famous Kaggle competition, the Google Landmark Recognition Challenge. Training set contains over 1.2 … Read more Google Landmark Recognition using Transfer Learning

Because it’s Friday: CGI you never knew was CGI

Computer-generated imagery in movies has gotten so good these days, much of the time you don’t even realize it’s there. You probably never noticed how Michael Cera’s physique had been altered, or how Lost in Translation used motion capture technology from the future. [embedded content] That’s all from the blog team for this week. Have … Read more Because it’s Friday: CGI you never knew was CGI

Anime Recommendation engine: From Matrix Factorization to Learning-to-rank

Anime Obsession gone too far!! OtakusHenry Chang, Joey Chen, Guanhua Zhang, Preetika Srivastava and Cherry Agarwal The vast amount of data that is hosted on the internet today has led to the information overflow and thus there is a constant need to improve the user experience. A recommendation engine is a system that helps support … Read more Anime Recommendation engine: From Matrix Factorization to Learning-to-rank

Introduction to Web Scraping with BeautifulSoup

Find specific elements in the page The created BeautifulSoup object can now be used to find elements in the HTML. When we inspected the website we saw that every list item in the content section has a class that starts with tocsection- and we can us BeautifulSoup’s find_all method to find all list items with that … Read more Introduction to Web Scraping with BeautifulSoup

Home Remodeling Analysis Turned Excel Data Handling in Python

Why cleaning data is the most important step Original Project Mission: Find interesting insights to see where the remodeling market is headed Project Mission (Twist): How to handle well manicured excel data in Python (spoiler: neat is a deceptive word) Timeline: One week (I tell you, it’s not enough!) Project Findings for the Original Goal : … Read more Home Remodeling Analysis Turned Excel Data Handling in Python

The complete guide for topics extraction with LDA (Latent Dirichlet Allocation) in Python

A recurring subject in NLP is to understand large corpus of texts through topic extraction. Whether you analyze users’ online reviews, product descriptions, or text entered in search bars, understanding key topics will always come in handy. Popular picture explaining LDA Before going into the LDA method, let me remind you that not reinventing the … Read more The complete guide for topics extraction with LDA (Latent Dirichlet Allocation) in Python

Exploratory Data Analysis, Feature Engineering and Modelling using Supermarket Sales Data. Part 1.

The first thing I like to do when doing EDA on a dataset with a reasonable amount of numeric columns, is to check the relationship between my target variable and these numeric features. One quick way to do this is to use the seaborn heatmap plot. This seaborn heatmap takes the correlation matrix calculated on … Read more Exploratory Data Analysis, Feature Engineering and Modelling using Supermarket Sales Data. Part 1.

Want to Cluster Text? Try Custom Word-Embeddings!

Tf-idf vectors with word-embeddings are analyzed for clustering effectiveness. The text corpus examples considered here indicate that custom word-embeddings can help improve clusterability of the corpus That is welcome news after our ho-hum results for text classification when using word-embeddings. In the context of classification we concluded that keeping it simple with naive bayes and tf-idf … Read more Want to Cluster Text? Try Custom Word-Embeddings!

Weekly Selection — Dec 14, 2018

3rd Wave Data Visualization By Elijah Meeks — 12 min read Imagine what it was like to do data visualization 30 years ago. It’s 1988 and you’re using Excel 2.0 for simple charts like pie charts and line charts, or maybe something like SPSS for more complicated exploration and Arc/Info for geospatial data visualization. Favorite

CSV Analysis with Amazon Athena

Executing standard SQL queries on your Amazon S3 bucket files Dec 14, 2018 “What’s Amazon Athena?”, I hear you ask. Good question. It’s one of Amazon Web Services’ amenities for architecture in the cloud. More specifically, Athena allows us to query data we hold in another service called Amazon Simple Storage Service (S3) using standard SQL … Read more CSV Analysis with Amazon Athena

Using Analysis of Variance with Experimentation Data

“The one-way analysis of variance (ANOVA) is used to determine whether there are any statistically significant differences between the means of two or more independent (unrelated) groups (although you tend to only see it used when there are a minimum of three, rather than two groups)”. Having entered the world of digital analytics from a … Read more Using Analysis of Variance with Experimentation Data

Get a glimpse of future using time series forecasting using Auto-ARIMA and Artificial Intelligence

Time Series Forecasting using Auto-ARIMA in python. AI and future Currently, there is a lot of development going on in Artificial intelligence research to get an accurate glimpse of the future. If any mathematical model predicts future data taking input as only time then that terminology called as time series forecasting. There are many machine learning and … Read more Get a glimpse of future using time series forecasting using Auto-ARIMA and Artificial Intelligence

The Naive Bayes Classifier

Joseph Catanzarite The Naïve Bayes Classifier is perhaps the simplest machine learning classifier to build, train, and predict with. This post will show how and why it works. Part 1 reveals that the much-celebrated Bayes Rule is just a simple statement about joint and conditional probabilities. But its blandness belies astonishing power, as we’ll see … Read more The Naive Bayes Classifier

Kuzushiji-MNIST – Japanese Literature Alternative Dataset for Deep Learning Tasks

Plus our VGG-ResNet ensemble model with state-of-the-art results MNIST, a dataset with 70,000 labeled images of handwritten digits, has been one of the most popular datasets for image processing and classification for over twenty years. Despite its popularity, contemporary deep learning algorithms handle it easily, often surpassing an accuracy result of 99.5%. A new paper … Read more Kuzushiji-MNIST – Japanese Literature Alternative Dataset for Deep Learning Tasks

Learning Data Science on Generic Datasets is Useless

Alright it’s most definitely not useless. But it is far more useless than it needs to be. This article will outline some of the potential roles that data plays in learning data science, with an argument against using generic (and static for that matter) datasets. All too often we see machine learning topics taught on … Read more Learning Data Science on Generic Datasets is Useless

Closing the Sale: Predicting Home Prices via Linear Regression

Imports, Data Cleansing, and EDA Cleaning and EDA are important for this challenge as this data set contains many ordinal / categorical features that may be important in categorization and will need to be converted to numerical values. As a baseline, I imported the following libraries to clean, explore and model the training data. One of … Read more Closing the Sale: Predicting Home Prices via Linear Regression

Nobody UNDERSTANDS Me … But Soon, Artificial Intelligence Just Might

Our faces and voices can be analyzed for emotion. As I mentioned, biomimicry, or imitating natural design in the things we create, is critical in recreating this human tendency in AI. Our end goal is artificial empathy, which (for now, at least) describes a machine’s ability to recognize and respond to human emotion. Going in line … Read more Nobody UNDERSTANDS Me … But Soon, Artificial Intelligence Just Might

Pandas for data.table Users

R and Python are both great languages for data analysis. While they are remarkably similar in some aspects, they are drastically different in others. In this post, I will focus on the similarities and differences between Pandas and data.table, two of the most prominent data manipulation packages in Python/R. There is alreay an excellent post … Read more Pandas for data.table Users

An introduction to high-dimensional hyper-parameter tuning

If you ever struggled with tuning Machine Learning (ML) models, you are reading the right piece. Hyper-parameter tuning refers to the problem of finding an optimal set of parameter values for a learning algorithm. Usually, the process of choosing these values is a time-consuming task. Even for simple algorithms like Linear Regression, finding the best … Read more An introduction to high-dimensional hyper-parameter tuning