## running plot [and simulated annealing]

Last weekend, I found out a way to run updated plots within a loop in R, when calling plot() within the loop was never updated in real time. The above suggestion of including a Sys.sleep(0.25) worked perfectly on a simulated annealing example for determining the most dispersed points in a unit disc. Related To leave … Read more running plot [and simulated annealing]

## The Naive Bayes Classifier

Joseph Catanzarite The Naïve Bayes Classifier is perhaps the simplest machine learning classifier to build, train, and predict with. This post will show how and why it works. Part 1 reveals that the much-celebrated Bayes Rule is just a simple statement about joint and conditional probabilities. But its blandness belies astonishing power, as we’ll see … Read more The Naive Bayes Classifier

## Unpacking (**PCA)

Alright, better to implement PCA to get the image. Let’s start by making 5 *10 matrix, and take steps of the process. Matrix X import numpy as npX = np.random.rand(5,10) The column are variables (characteristics) and the row are samples(say, ‘cat’ or ‘dog’). What we want to do with this matrix is to get eigenvalues … Read more Unpacking (**PCA)

## Kuzushiji-MNIST – Japanese Literature Alternative Dataset for Deep Learning Tasks

Plus our VGG-ResNet ensemble model with state-of-the-art results MNIST, a dataset with 70,000 labeled images of handwritten digits, has been one of the most popular datasets for image processing and classification for over twenty years. Despite its popularity, contemporary deep learning algorithms handle it easily, often surpassing an accuracy result of 99.5%. A new paper … Read more Kuzushiji-MNIST – Japanese Literature Alternative Dataset for Deep Learning Tasks

## My book ‘Deep Learning from first principles:Second Edition’ now on Amazon

The second edition of my book ‘Deep Learning from first principles:Second Edition- In vectorized Python, R and Octave’, is now available on Amazon, in both paperback (\$14.99)  and kindle (\$9.99/Rs449/-)  versions. Since this book is almost 70% code, all functions, and code snippets have been formatted to use the fixed-width font ‘Lucida Console’. In addition … Read more My book ‘Deep Learning from first principles:Second Edition’ now on Amazon

## Learning Data Science on Generic Datasets is Useless

Alright it’s most definitely not useless. But it is far more useless than it needs to be. This article will outline some of the potential roles that data plays in learning data science, with an argument against using generic (and static for that matter) datasets. All too often we see machine learning topics taught on … Read more Learning Data Science on Generic Datasets is Useless

## Closing the Sale: Predicting Home Prices via Linear Regression

Imports, Data Cleansing, and EDA Cleaning and EDA are important for this challenge as this data set contains many ordinal / categorical features that may be important in categorization and will need to be converted to numerical values. As a baseline, I imported the following libraries to clean, explore and model the training data. One of … Read more Closing the Sale: Predicting Home Prices via Linear Regression

## Nobody UNDERSTANDS Me … But Soon, Artificial Intelligence Just Might

Our faces and voices can be analyzed for emotion. As I mentioned, biomimicry, or imitating natural design in the things we create, is critical in recreating this human tendency in AI. Our end goal is artificial empathy, which (for now, at least) describes a machine’s ability to recognize and respond to human emotion. Going in line … Read more Nobody UNDERSTANDS Me … But Soon, Artificial Intelligence Just Might

## Pdftools 2.0: powerful pdf text extraction tools

A new version of pdftools has been released to CRAN. Go get it while it’s hot: install.packages(“pdftools”) This version has two major improvements: low level text extraction and encoding improvements. About PDF textboxes A pdf document may seem to contain paragraphs or tables in a viewer, but this is not actually true. PDF is a … Read more Pdftools 2.0: powerful pdf text extraction tools

## Easy CI/CD of GPU applications on Google Cloud including bare-metal using Gitlab and Kubernetes

Summary Are you a data scientist who only wants to focus on modelling and coding and not on setting up a GPU cluster? Then, this blog might be interesting for you. We developed an automated pipeline using gitlab and Kubernetes that is able to run code in two GPU environments, GCP and bare-metal; no need … Read more Easy CI/CD of GPU applications on Google Cloud including bare-metal using Gitlab and Kubernetes

## Yet another visualization of the Bayesian Beta-Binomial model

The Beta-Binomial model is the “hello world” of Bayesian statistics. That is, it’s the first model you get to run, often before you even know what you are doing. There are many reasons for this: It only has one parameter, the underlying proportion of success, so it’s easy to visualize and reason about. It’s easy … Read more Yet another visualization of the Bayesian Beta-Binomial model

## Reusable Pipelines in R

Pipelines in R are popular, the most popular one being magrittr as used by dplyr. This note will discuss the advanced re-usable piping systems: rquery/rqdatatable operator trees and wrapr function object pipelines. In each case we have a set of objects designed to extract extra power from the wrapr dot-arrow pipe %.>%. Piping Piping is … Read more Reusable Pipelines in R

## Pandas for data.table Users

R and Python are both great languages for data analysis. While they are remarkably similar in some aspects, they are drastically different in others. In this post, I will focus on the similarities and differences between Pandas and data.table, two of the most prominent data manipulation packages in Python/R. There is alreay an excellent post … Read more Pandas for data.table Users

## An introduction to high-dimensional hyper-parameter tuning

If you ever struggled with tuning Machine Learning (ML) models, you are reading the right piece. Hyper-parameter tuning refers to the problem of finding an optimal set of parameter values for a learning algorithm. Usually, the process of choosing these values is a time-consuming task. Even for simple algorithms like Linear Regression, finding the best … Read more An introduction to high-dimensional hyper-parameter tuning

## R community update: announcing sessions for useR Delhi December meetup

As referenced in my last blog post, useR Delhi NCR is all set to host our second meetup on 15th December, i.e. upcoming Saturday. We’ve finalized two exciting speaker sessions for the same. They’re as follows: Basics of Shiny and geospatial visualizations by Sean Angiolillo Up in the air: cloud storage based workflows in R … Read more R community update: announcing sessions for useR Delhi December meetup

## Preprocessing with sklearn: a complete and comprehensive guide

For aspiring data scientist it might sometimes be difficult to find their way through the forest of preprocessing techniques. Sklearn its preprocessing library forms a solid foundation to guide you through this important task in the data science pipeline. Although Sklearn a has pretty solid documentation, it often misses streamline and intuition between different concepts. … Read more Preprocessing with sklearn: a complete and comprehensive guide

## IcoOmen: Using Machine Learning to Predict ICO Prices

Methodology Choose inputs and outputs. Collect and aggregate the data. Prepare the data. Explore and attempt to understand the data. Choose a Machine Learning Model. Measure the performance of the Model. Save the Model. Use the Model to make predictions. 1. Choosing Inputs and Outputs Inputs Choosing the right inputs and outputs (in the case of … Read more IcoOmen: Using Machine Learning to Predict ICO Prices

## How to Predict Severe Traffic Jams with Python and Recurrent Neural Networks?

An Application of Sequence Model to Mine Waze Open Data of Traffic Incidents, using Python and Keras. In this tutorial, I will show you how to use RNN deep learning model to find patterns from Waze Traffic Open Data of Incidents Report, and predict if severe traffic jams will happen shortly. Interventions can be taken out … Read more How to Predict Severe Traffic Jams with Python and Recurrent Neural Networks?

## How to get the most out of Towards Data Science?

Our Readers’ Guide We have received feedback that some of you find it difficult to efficiently navigate our Medium publication. So we have put together a few bullet points that will hopefully aid your experience on our blog. Subscribe to our publication to receive our Monthly Edition and Weekly Selection directly in your mailbox. Follow us … Read more How to get the most out of Towards Data Science?

## Gold-Mining Week 15 (2018)

The post Gold-Mining Week 15 (2018) appeared first on Fantasy Football Analytics. Related R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and … Read more Gold-Mining Week 15 (2018)

## Vaex: Out of Core Dataframes for Python and Fast Visualization

So… no pandas ?? There are some issues with pandas that the original author Wes McKinney outlines in his insightful blogpost: “Apache Arrow and the “10 Things I Hate About pandas”. Many of these issues will be tackled in the next version of pandas (pandas2?), building on top of Apache Arrow and other libraries. Vaex starts … Read more Vaex: Out of Core Dataframes for Python and Fast Visualization

## RTutor: Better Incentive Contracts For Road Construction

Since about two weeks, I face a large additional traffic jam every morning due to a construction site on the road. When passing the construction site, often only few people or sometimes nobody seems to be working there. Being an economist, I really wonder how much of such traffic jams could be avoided with better … Read more RTutor: Better Incentive Contracts For Road Construction

## Day 13 – little helper read_files

We at STATWORX work a lot with R and we often use the same little helper functions within our projects. These functions ease our daily work life by reducing repetitive code parts or by creating overviews of our projects. At first, there was no plan to make a package, but soon I realised, that it … Read more Day 13 – little helper read_files

## Recreating the NBA lead tracker graphic

For each NBA game, nba.com has a really nice graphic which tracks the point differential between the two teams throughout the game. Here is the lead tracker graphic for the game between the LA Clippers and the Phoenix Suns on 10 Dec 2018: Taken from https://www.nba.com/games/20181210/LACPHX#/matchup I thought it would be cool to try recreating … Read more Recreating the NBA lead tracker graphic

## Music Genre Classification with Python

Objective Companies nowadays use music classification, either to be able to place recommendations to their customers (such as Spotify, Soundcloud) or simply as a product (for example Shazam). Determining music genres is the first step in that direction. Machine Learning techniques have proved to be quite successful in extracting trends and patterns from the large … Read more Music Genre Classification with Python

## Rsampling Fama French

Today we will continue our work on Fama French factor models, but more as a vehicle to explore some of the awesome stuff happening in the world of tidy models. For new readers who want get familiar with Fama French before diving into this post, see here where we covered importing and wrangling the data, … Read more Rsampling Fama French

## Twins on the up

Are multiple births on the increase? My twin boys turned 5 years old today. Wow, time flies. Life is never dull, because twins are still seen as something of a novelty, so wherever we go, we find ourselves in conversation with strangers, who are intrigued by the whole thing. In order to save time if … Read more Twins on the up

## Named Entity Recognition (NER), Meeting Industry’s Requirement by Applying state-of-the-art Deep…

we are going to have a quick look at the architecture of four different state-of-the-art approaches by referring to the actual research paper and then we will move on to implement the one with the highest accuracy. Bidirectional LSTM-CRF: More details and implementation in keras. from the paper(Bidirectional LSTM-CRF Models for Sequence Tagging) 2. Bidirectional LSTM-CNNs: … Read more Named Entity Recognition (NER), Meeting Industry’s Requirement by Applying state-of-the-art Deep…

## The Importance of Being Recurrent for Modeling Hierarchical Structure

RNNs have inherent performance limitations For a while, it seemed that RNN’s were taking the Natural Language Processing (NLP) world by storm (from about 2014–17). However, we’ve recently started realizing the limitations of RNN’s, primarily that they are “inefficient and not scalable”. While there is great promise in overcoming these limitations by using more specialized … Read more The Importance of Being Recurrent for Modeling Hierarchical Structure

## Text Summarization on the Books of Harry Potter

Hermione interrupted them. “Aren’t you two ever going to read Hogwarts, A History?” How many times throughout the Harry Potter series does Hermione bug Harry and Ron to read the enormous tome Hogwarts, A History? Hint: it’s a lot. How many nights do the three of them spend in the library, reading through every book … Read more Text Summarization on the Books of Harry Potter

## My introductory course on Bayesian statistics

So, after having held workshops introducing Bayes for a couple of years now, I finally pulled myself together and completed my DataCamp course: Fundamentals of Bayesian Data Analysis in R! ? While it’s called a course, it’s more like a 4 hour workshop and — without requiring anything but basic R skills and a vague … Read more My introductory course on Bayesian statistics

## Supervised Machine Learning: Classification

Machine Learning is the science (and art) of programming computers so they can learn from data. [Machine Learning is the] field of study that gives computers the ability to learn without being explicitly programmed. Arthur Samuel, 1959 A better definition: A computer program is said to learn from experience E with respect to some task … Read more Supervised Machine Learning: Classification

## Teaching and Learning Materials for Data Visualization

Data Visualization: A Practical Introduction will begin shipping next week. I’ve written an R package that contains datasets, functions, and a course packet to go along with the book. The socviz package contains about twenty five datasets and a number of utility and convenience functions. The datasets range in size from things with just a … Read more Teaching and Learning Materials for Data Visualization

## 7 Tips to Getting a Data Science Job Faster

Data science is a booming field, but with great publicity comes great difficulty. Breaking into the data science field gets twice as hard each year. The growth of training programs like graduate degrees/certificates and bootcamps far exceeds the growth of new entry-level positions. Prior to 2015, it was a cake-walk to get multiple interviews. Now … Read more 7 Tips to Getting a Data Science Job Faster

## First Mile

The Electric Pulse Thomas Parker Electric Car (1895) | Fisker Karma (2012) The credit to who invented the first electric vehicle is also debated due to the fact that many scientists and tinkerers were working with various forms of electric sources (batteries and electric motors) around the same time. However, there is a prominent name in electric … Read more First Mile

## The Kernel Trick

The Kernel Trick We have seen how higher dimensional transformations can allow us to separate data in order to make classification predictions. It seems that in order to train a support vector classifier and optimize our objective function, we would have to perform operations with the higher dimensional vectors in the transformed feature space. In real … Read more The Kernel Trick

## Amazon Customer Analysis

User review networks for customer segmentation Over the past decade or two, Americans have continued to prefer payment methods that are traceable, providing retailers and vendors with a rich source of data on their customers. This data is used by data scientists to help businesses make more informed decisions with respect to inventory, marketing, and … Read more Amazon Customer Analysis

## A Guide for Building Convolutional Neural Networks

Computer Vision it at the forefront of advancements in Artificial Intelligence (AI). It’s moving fast with new research coming out each and every day allowing us to do truly amazing things that we could’t do before with computers and AI. Convolutional Neural Networks (CNNs) are the driving force behind every advancement in Computer Vision research … Read more A Guide for Building Convolutional Neural Networks

## The invisible workers of the AI era

50 ways to label data There are different ways to get your data labeled. Some firms label their data themselves — although this can be costly, as hiring people simply for these tasks costs firms both money and flexibility. Other companies even find ways to get people to label their data for free. Ever wonder why Google’s reCAPTCHA … Read more The invisible workers of the AI era

## Day 12 – little helper dive

We at STATWORX work a lot with R and we often use the same little helper functions within our projects. These functions ease our daily work life by reducing repetitive code parts or by creating overviews of our projects. At first, there was no plan to make a package, but soon I realised, that it … Read more Day 12 – little helper dive

## AI and Machine Learning: Moving from Training to Education

The debate of whether AI will ever achieve capabilities at par or beyond human intelligence is ever ongoing. It certainly has intensified with the recent advancements in AI, Machine Learning (ML), and Deep Learning (DL) with some believing that the current technologies are already capable of paving the way for Artificial General Intelligence (AGI). You … Read more AI and Machine Learning: Moving from Training to Education

## Visualizing Hurricane Data with Shiny

Motivation for Project Around the time that I was selecting a topic for this project, my parents and my hometown found themselves in the path of a Category 1 hurricane. Thankfully, everyone was ok, and there was only minor damage to their property. But this event made me think about how long it had been … Read more Visualizing Hurricane Data with Shiny

## Scraping the Turkey Accordion

Related To leave a comment for the author, please follow the link and comment on their blog: R on datawookie. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) … Read more Scraping the Turkey Accordion

## Towards Ethical Machine Learning

https://initiatives.provost.uci.edu/event/philosophy-machine-learning-knowledge-causality/ I quit my job to enter an intensive data science bootcamp. I understand the value behind the vast amount of data available that enables us to create predictive machine learning algorithms. In addition to recognizing its value on a professional level, I benefit from these technologies as a consumer. Whenever I find myself in … Read more Towards Ethical Machine Learning

## Reading List Faster With parallel, doParallel, and pbapply

I have several tables that I would like to load as a sole data frame. Derived functions from read. table () have a lot of convenient features, but it seems like there is a lot of steps in the implementation that would slow things down. The gain in performance of reading 29 CSV files (about … Read more Reading List Faster With parallel, doParallel, and pbapply

## Using ggplot2 for functional time series

I spoke yesterday about using ggplot2 for functional data graphics, rather than the custom-built plotting functionality available in the many functional data packages, including my own rainbow package written with Hanlin Shang. It is a much more powerful and flexible way to work, so I thought it would be useful to share some examples. French … Read more Using ggplot2 for functional time series

## Network Centrality in R: New ways of measuring Centrality

This is the third post of a series on the concept of “network centrality” withapplications in R and the package netrankr. The last part introduced the concept ofneighborhood-inclusion and its implications for centrality. In this post, weextend the concept to a broader class of dominance relations by deconstructing indicesinto a series of building blocks and … Read more Network Centrality in R: New ways of measuring Centrality

## Code for case study – Customer Churn with Keras/TensorFlow and H2O

The code you find below can be used to recreate all figures and analyses from this book chapter. Because the content is exclusively for the book, my descriptions around the code had to be minimal. But I’m sure, you can get the gist, even without the book. ? Thank you to the following people for … Read more Code for case study – Customer Churn with Keras/TensorFlow and H2O

## Geocomputation with R – the afterword

I am extremely proud to announce that Geocomputation with R is complete.It took Robin, Jannes, and me almost 2 years of collaborative planning, writing, refinement, and deployment to make the book available for anyone interested in open source, command-line approaches for handling geographic data.We’re very happy that it’s now ready to present to the world … Read more Geocomputation with R – the afterword

## Sharing Modeling Pipelines in R

Reusable modeling pipelines are a practical idea that gets re-developed many times in many contexts. wrapr supplies a particularly powerful pipeline notation, and a pipe-stage re-use system (notes here). We will demonstrate this with the vtreat data preparation system. Our example task is to fit a model on some arbitrary data. Our model will try … Read more Sharing Modeling Pipelines in R