Modeling tree height and basal area in the Finger Lakes National Forest, NY

I tried my hand at using the R package, randomForest to create two regression models for tree height and basal area based off some lidar and field-collected data in the Finger Lakes National Forest, NY. Disclaimer: this project was my first real taste of R. Earlier in the semester I had done some simple learning into … Read more Modeling tree height and basal area in the Finger Lakes National Forest, NY

Request for comments on planned features for futile.logger 1.5

I will be pushing a new version of futile.logger (version 1.5) to CRAN in January. This version introduces a number of enhancements and fixes some bugs. It will also contain at least one breaking change. I am making the release process public, since the package is now used in a number of other packages. If … Read more Request for comments on planned features for futile.logger 1.5

Text Generation Using RNNs

Generate characters from Alice in Wonderland Introduction Text generation is a popular problem in Data Science and Machine Learning, and it is a suitable task for Recurrent Neural Nets. This report uses TensorFlow to build an RNN text generator and builds a high-level API in Python3. The report is inspired by @karpathy ( min-char-rnn) and … Read more Text Generation Using RNNs

Introduction to Interactive Time Series Visualizations with Plotly in Python

Introduction to Plotly Plotly is a company that makes visualization tools including a Python API library. (Plotly also makes Dash, a framework for building interactive web-based applications with Python code). For this article, we’ll stick to working with the plotly Python library in a Jupyter Notebook and touching up images in the online plotly editor. When … Read more Introduction to Interactive Time Series Visualizations with Plotly in Python

The Ultimate NanoBook to understand Deep Learning based Image Classifier

The first and most important step of our journey: As I have said before, we are going to simply ask questions that will guide us to build an image classifier. For the sake of brevity, we will call Image Classifier an ICNow, we are ready to start our journey. So let us ask the first question: … Read more The Ultimate NanoBook to understand Deep Learning based Image Classifier

Applying GANs to Super Resolution

SRGAN Results from Ledig et al. [3] Generative adversarial networks (GANs) have found many applications in Deep Learning. One interesting problem that can be better solved using GANs is super-resolution. Super-resolution is a task concerned with upscaling images from low-resolution sizes such as 90 x 90, into high-resolution sizes such as 360 x 360. In this … Read more Applying GANs to Super Resolution

A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way

Artificial Intelligence has been witnessing a monumental growth in bridging the gap between the capabilities of humans and machines. Researchers and enthusiasts alike, work on numerous aspects of the field to make amazing things happen. One of many such areas is the domain of Computer Vision. The agenda for this field is to enable machines … Read more A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way

Dealing With Class Imbalanced Datasets For Classification.

Skewed datasets are not uncommon. And they are tough to handle. Usual classification models and techniques often fail miserably when presented with such a problem. Although your model could get you to even a 99% accuracy on such cases, yet, if you are measuring yourself against a sensible metric such as the ROC Auc score, … Read more Dealing With Class Imbalanced Datasets For Classification.

Google Landmark Recognition using Transfer Learning

Image classification with 15k classes! Project by Catherine McNabb, Anuraag Mohile, Avani Sharma, Evan David, Anisha Garg Dealing with a large number of classes with very few images in many classes is what makes this task really challenging! The problem comes from a famous Kaggle competition, the Google Landmark Recognition Challenge. Training set contains over 1.2 … Read more Google Landmark Recognition using Transfer Learning

Advent of Code: Most Popular Languages

You might have heard of the Advent of Code,a 25-day challenge involving a programming puzzle a day, to be solvedwith the language of your choice. I’ve noted the popularity of thisactivity in my Twitter timeline but also in my GitHub timeline whereI’ve seen the creation of a few advent-of-code or so repositories. AoC is largely … Read more Advent of Code: Most Popular Languages

Learning R: A gentle introduction to higher-order functions

Have you ever thought about why the definition of a function in R is different from many other programming languages? The part that causes the biggest difficulties (especially for beginners of R) is that you state the name of the function at the beginning and use the assignment operator – as if functions were like … Read more Learning R: A gentle introduction to higher-order functions

Because it’s Friday: CGI you never knew was CGI

Computer-generated imagery in movies has gotten so good these days, much of the time you don’t even realize it’s there. You probably never noticed how Michael Cera’s physique had been altered, or how Lost in Translation used motion capture technology from the future. [embedded content] That’s all from the blog team for this week. Have … Read more Because it’s Friday: CGI you never knew was CGI

Anime Recommendation engine: From Matrix Factorization to Learning-to-rank

Anime Obsession gone too far!! OtakusHenry Chang, Joey Chen, Guanhua Zhang, Preetika Srivastava and Cherry Agarwal The vast amount of data that is hosted on the internet today has led to the information overflow and thus there is a constant need to improve the user experience. A recommendation engine is a system that helps support … Read more Anime Recommendation engine: From Matrix Factorization to Learning-to-rank

Introduction to Web Scraping with BeautifulSoup

Find specific elements in the page The created BeautifulSoup object can now be used to find elements in the HTML. When we inspected the website we saw that every list item in the content section has a class that starts with tocsection- and we can us BeautifulSoup’s find_all method to find all list items with that … Read more Introduction to Web Scraping with BeautifulSoup

Home Remodeling Analysis Turned Excel Data Handling in Python

Why cleaning data is the most important step Original Project Mission: Find interesting insights to see where the remodeling market is headed Project Mission (Twist): How to handle well manicured excel data in Python (spoiler: neat is a deceptive word) Timeline: One week (I tell you, it’s not enough!) Project Findings for the Original Goal : … Read more Home Remodeling Analysis Turned Excel Data Handling in Python

The complete guide for topics extraction with LDA (Latent Dirichlet Allocation) in Python

A recurring subject in NLP is to understand large corpus of texts through topic extraction. Whether you analyze users’ online reviews, product descriptions, or text entered in search bars, understanding key topics will always come in handy. Popular picture explaining LDA Before going into the LDA method, let me remind you that not reinventing the … Read more The complete guide for topics extraction with LDA (Latent Dirichlet Allocation) in Python

Exploratory Data Analysis, Feature Engineering and Modelling using Supermarket Sales Data. Part 1.

The first thing I like to do when doing EDA on a dataset with a reasonable amount of numeric columns, is to check the relationship between my target variable and these numeric features. One quick way to do this is to use the seaborn heatmap plot. This seaborn heatmap takes the correlation matrix calculated on … Read more Exploratory Data Analysis, Feature Engineering and Modelling using Supermarket Sales Data. Part 1.

Want to Cluster Text? Try Custom Word-Embeddings!

Tf-idf vectors with word-embeddings are analyzed for clustering effectiveness. The text corpus examples considered here indicate that custom word-embeddings can help improve clusterability of the corpus That is welcome news after our ho-hum results for text classification when using word-embeddings. In the context of classification we concluded that keeping it simple with naive bayes and tf-idf … Read more Want to Cluster Text? Try Custom Word-Embeddings!

In case you missed it: November 2018 roundup

Related To leave a comment for the author, please follow the link and comment on their blog: Revolutions. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, … Read more In case you missed it: November 2018 roundup

Weekly Selection — Dec 14, 2018

3rd Wave Data Visualization By Elijah Meeks — 12 min read Imagine what it was like to do data visualization 30 years ago. It’s 1988 and you’re using Excel 2.0 for simple charts like pie charts and line charts, or maybe something like SPSS for more complicated exploration and Arc/Info for geospatial data visualization. Favorite

CSV Analysis with Amazon Athena

Executing standard SQL queries on your Amazon S3 bucket files Dec 14, 2018 “What’s Amazon Athena?”, I hear you ask. Good question. It’s one of Amazon Web Services’ amenities for architecture in the cloud. More specifically, Athena allows us to query data we hold in another service called Amazon Simple Storage Service (S3) using standard SQL … Read more CSV Analysis with Amazon Athena

Using Analysis of Variance with Experimentation Data

“The one-way analysis of variance (ANOVA) is used to determine whether there are any statistically significant differences between the means of two or more independent (unrelated) groups (although you tend to only see it used when there are a minimum of three, rather than two groups)”. Having entered the world of digital analytics from a … Read more Using Analysis of Variance with Experimentation Data

Get a glimpse of future using time series forecasting using Auto-ARIMA and Artificial Intelligence

Time Series Forecasting using Auto-ARIMA in python. AI and future Currently, there is a lot of development going on in Artificial intelligence research to get an accurate glimpse of the future. If any mathematical model predicts future data taking input as only time then that terminology called as time series forecasting. There are many machine learning and … Read more Get a glimpse of future using time series forecasting using Auto-ARIMA and Artificial Intelligence

The Naive Bayes Classifier

Joseph Catanzarite The Naïve Bayes Classifier is perhaps the simplest machine learning classifier to build, train, and predict with. This post will show how and why it works. Part 1 reveals that the much-celebrated Bayes Rule is just a simple statement about joint and conditional probabilities. But its blandness belies astonishing power, as we’ll see … Read more The Naive Bayes Classifier

Kuzushiji-MNIST – Japanese Literature Alternative Dataset for Deep Learning Tasks

Plus our VGG-ResNet ensemble model with state-of-the-art results MNIST, a dataset with 70,000 labeled images of handwritten digits, has been one of the most popular datasets for image processing and classification for over twenty years. Despite its popularity, contemporary deep learning algorithms handle it easily, often surpassing an accuracy result of 99.5%. A new paper … Read more Kuzushiji-MNIST – Japanese Literature Alternative Dataset for Deep Learning Tasks

My book ‘Deep Learning from first principles:Second Edition’ now on Amazon

The second edition of my book ‘Deep Learning from first principles:Second Edition- In vectorized Python, R and Octave’, is now available on Amazon, in both paperback ($14.99)  and kindle ($9.99/Rs449/-)  versions. Since this book is almost 70% code, all functions, and code snippets have been formatted to use the fixed-width font ‘Lucida Console’. In addition … Read more My book ‘Deep Learning from first principles:Second Edition’ now on Amazon

Learning Data Science on Generic Datasets is Useless

Alright it’s most definitely not useless. But it is far more useless than it needs to be. This article will outline some of the potential roles that data plays in learning data science, with an argument against using generic (and static for that matter) datasets. All too often we see machine learning topics taught on … Read more Learning Data Science on Generic Datasets is Useless

Closing the Sale: Predicting Home Prices via Linear Regression

Imports, Data Cleansing, and EDA Cleaning and EDA are important for this challenge as this data set contains many ordinal / categorical features that may be important in categorization and will need to be converted to numerical values. As a baseline, I imported the following libraries to clean, explore and model the training data. One of … Read more Closing the Sale: Predicting Home Prices via Linear Regression

Nobody UNDERSTANDS Me … But Soon, Artificial Intelligence Just Might

Our faces and voices can be analyzed for emotion. As I mentioned, biomimicry, or imitating natural design in the things we create, is critical in recreating this human tendency in AI. Our end goal is artificial empathy, which (for now, at least) describes a machine’s ability to recognize and respond to human emotion. Going in line … Read more Nobody UNDERSTANDS Me … But Soon, Artificial Intelligence Just Might

Pdftools 2.0: powerful pdf text extraction tools

A new version of pdftools has been released to CRAN. Go get it while it’s hot: install.packages(“pdftools”) This version has two major improvements: low level text extraction and encoding improvements. About PDF textboxes A pdf document may seem to contain paragraphs or tables in a viewer, but this is not actually true. PDF is a … Read more Pdftools 2.0: powerful pdf text extraction tools

Easy CI/CD of GPU applications on Google Cloud including bare-metal using Gitlab and Kubernetes

Summary Are you a data scientist who only wants to focus on modelling and coding and not on setting up a GPU cluster? Then, this blog might be interesting for you. We developed an automated pipeline using gitlab and Kubernetes that is able to run code in two GPU environments, GCP and bare-metal; no need … Read more Easy CI/CD of GPU applications on Google Cloud including bare-metal using Gitlab and Kubernetes

Yet another visualization of the Bayesian Beta-Binomial model

The Beta-Binomial model is the “hello world” of Bayesian statistics. That is, it’s the first model you get to run, often before you even know what you are doing. There are many reasons for this: It only has one parameter, the underlying proportion of success, so it’s easy to visualize and reason about. It’s easy … Read more Yet another visualization of the Bayesian Beta-Binomial model

Reusable Pipelines in R

Pipelines in R are popular, the most popular one being magrittr as used by dplyr. This note will discuss the advanced re-usable piping systems: rquery/rqdatatable operator trees and wrapr function object pipelines. In each case we have a set of objects designed to extract extra power from the wrapr dot-arrow pipe %.>%. Piping Piping is … Read more Reusable Pipelines in R

Pandas for data.table Users

R and Python are both great languages for data analysis. While they are remarkably similar in some aspects, they are drastically different in others. In this post, I will focus on the similarities and differences between Pandas and data.table, two of the most prominent data manipulation packages in Python/R. There is alreay an excellent post … Read more Pandas for data.table Users

An introduction to high-dimensional hyper-parameter tuning

If you ever struggled with tuning Machine Learning (ML) models, you are reading the right piece. Hyper-parameter tuning refers to the problem of finding an optimal set of parameter values for a learning algorithm. Usually, the process of choosing these values is a time-consuming task. Even for simple algorithms like Linear Regression, finding the best … Read more An introduction to high-dimensional hyper-parameter tuning

R community update: announcing sessions for useR Delhi December meetup

As referenced in my last blog post, useR Delhi NCR is all set to host our second meetup on 15th December, i.e. upcoming Saturday. We’ve finalized two exciting speaker sessions for the same. They’re as follows: Basics of Shiny and geospatial visualizations by Sean Angiolillo Up in the air: cloud storage based workflows in R … Read more R community update: announcing sessions for useR Delhi December meetup

Preprocessing with sklearn: a complete and comprehensive guide

For aspiring data scientist it might sometimes be difficult to find their way through the forest of preprocessing techniques. Sklearn its preprocessing library forms a solid foundation to guide you through this important task in the data science pipeline. Although Sklearn a has pretty solid documentation, it often misses streamline and intuition between different concepts. … Read more Preprocessing with sklearn: a complete and comprehensive guide