An R Shiny app to recognize flower species

Introduction Playing around with PyTorch and R Shiny resulted in a simple Shiny app where the user can upload a flower image, the system will then predict the flower species. Steps that I took Download labeled flower data from the Visual Geometry Group, Install Pytorch and download their transfer learning tutorial script, You need to … Read moreAn R Shiny app to recognize flower species

Histopathologic Cancer Detector – Finding Cancer Cells with Machine Learning

Let’s take a look at the following diagram that illustrates the purposes of the specific layers in the CNN. As we can see above, starting from the left we are learning low-level features and the more we go to the right, the more specific things are being learned. The idea behind Transfer Learning is to … Read moreHistopathologic Cancer Detector – Finding Cancer Cells with Machine Learning

Detecting Firms with Intentional Misstatements using Machine Learning

Identifying firms with intentional distortion of financial statements is a challenging and exciting problem among auditors, banks and investors who rely on financial information to make decisions. Yet it is difficult to flag out these firms as intentional accounting misstatement (cooking the books) can take several forms: hiding company losses through other entities, recognizing revenue … Read moreDetecting Firms with Intentional Misstatements using Machine Learning

49 Years of Lyrics: Why so Angry?

Data Collection There are three datasets we’re using to run this experiment: A dataset we’ll collect ourselves that includes over 3400 song lyrics between 1970 and 2018. A list of prohibited/restricted words from www.freewebheaders.com that we’ll use to assess the perceived levels of profanity in lyrics. A training dataset from Kaggle (originally used for the … Read more49 Years of Lyrics: Why so Angry?

On the Perils of Automated Face Recognition

For anyone who has been paying attention, it will not have gone unnoticed that the past year has seen a dramatic expansion in the use of face recognition technology, including at schools, border crossing, and interactions with the police. Most recently, Delta announced that some passengers in Atlanta will be able to check in and … Read moreOn the Perils of Automated Face Recognition

Phillips-Ouliaris Test For Cointegration

In a project of developing PPNR balance projection models, I tried to use the Phillips-Ouliaris (PO) test to investigate the cointegration between the historical balance and a set of macro-economic variables and noticed that implementation routines of PO test in various R packages, e.g. urca and tseries, would give different results. After reading through the … Read morePhillips-Ouliaris Test For Cointegration

Our Collections

Explore further the world of data science, machine learning and artificial intelligence We are on a mission to get the best content relevant to data science, machine learning, and artificial intelligence out there for everyone. One of the challenges with any content platform on the internet is having a dedicated and curated list of resources … Read moreOur Collections

Improve your scientific models with meta-learning and likelihood-free inference

Article jointly written by Arthur Pesah and Antoine Wehenkel Motivation There are usually two ways of coming up with a new scientific theory: Starting from first principles, deducing the consequent laws, and coming up with experimental predictions in order to verify the theory Starting from experiments and inferring the simplest laws that explain your data. … Read moreImprove your scientific models with meta-learning and likelihood-free inference

6 uncommon principles for effective data sciences

How to conceptualize and implement effective data science projects Results, not hype Motivation The more I delve in data science, the more convinced I am that companies and data science practitioners must have a clear view on how to cut through the machine learning and AI hype, to implement an effective data science strategy that drives business … Read more6 uncommon principles for effective data sciences

Building a Skin Lesion Classification Web App

Using Keras and TensorFlow.js to classify seven types of skin lesions Alex YuBlockedUnblockFollowFollowing Dec 16 After doing research on Convolutional Neural Networks, I became interested in developing an end-to-end machine learning solution. I decided to use the HAM10000 dataset to build a web app to classify skin lesions. In this article, I’ll provide some background information … Read moreBuilding a Skin Lesion Classification Web App

How to Learn Data Science: Staying Motivated.

Learn how to Deal with Anxiety. When you start researching how to become a data scientist, you will discover an unfortunate fact about the profession. Namely, that becoming a data scientist requires knowledge of a broad and deep set of tools, technologies, and skills. All of which makes the prospect of becoming a data scientist VERY … Read moreHow to Learn Data Science: Staying Motivated.

What Kagglers are using for Text Classification

Advanced NLP techniques for deep learning With the problem of Image Classification is more or less solved by Deep learning, Text Classification is the next new developing theme in deep learning. For those who don’t know, Text classification is a common task in natural language processing, which transforms a sequence of a text of indefinite length … Read moreWhat Kagglers are using for Text Classification

2018-13 Rendering HTML Content in R Graphics

This report describes several R packages that allow HTML content to be rendered as part of an R plot. The core package is called ‘layoutEngine’, but that package requires a “backend” package to perform HTML layout calculations. Three example backends are demonstrated: ‘layoutEngineCSSBox’, ‘layoutEnginePhantomJS’, and ‘layoutEngineDOM’. We also introduce two new font packages, ‘gyre’ and … Read more2018-13 Rendering HTML Content in R Graphics

Minimum CRPS vs. maximum likelihood

In a new paper in Monthly Weather Review, minimum CRPS and maximum likelihood estimation are compared for fitting heteroscedastic (or nonhomogenous) regression models under different response distributions. Minimum CRPS is more robust to distributional misspecification while maximum likelihood is slightly more efficient under correct specification. An R implementation is available in the crch package. Citation … Read moreMinimum CRPS vs. maximum likelihood

Is a Picture Worth A Thousand Words?

Dec 16, 2018 Source: Dark Reading Background Our project was inspired by Jamie Ryan Kiros who created a model trained on 14 million romance passages to generate a short romantic story for a single image input. Similarly, the ultimate goal of our project was to output a short story for children. “neural-storyteller is a recurrent neural … Read moreIs a Picture Worth A Thousand Words?

Getting Started with TensorFlow in Google Colaboratory

Opening up a Colab Notebook When using Colab for the first time, you can launch a new notebook here: Once you have a notebook created, it’ll be saved in your Google Drive (Colab Notebooks folder). You can access it by visiting your Google Drive page, then either double-click on the file name, or right-click, and then … Read moreGetting Started with TensorFlow in Google Colaboratory

Develop a NLP Model in Python & Deploy It with Flask, Step by Step

Flask API, Document Classification, Spam Filter By far, we have developed many machine learning models, generated numeric predictions on the testing data, and tested the results. And we did everything offline. In reality, generating predictions is only part of a machine learning project, although it is the most important part in my opinion. Considering a system … Read moreDevelop a NLP Model in Python & Deploy It with Flask, Step by Step

Logic Theory —Basic Notation

The origin of logic theory starts at the concept of an argument. The majority of logic textbooks contain an opening, central definition for an argument — one that likely sounds much like the following: An argument contains one or more special statements, called premises , offered as a reason to believe that a further statement, called the conclusion, … Read moreLogic Theory —Basic Notation

Advanced Queries With SQL That Will Save Your Time

Yes, SQL still exists During the years of working with telecom data my folder with code snippets collected a lot of reusable examples. And it is not about “SELECT * FROM Table1”, I am talking about finding and handling or removing duplicate values, selecting top N values from each group of data within same table, shuffling … Read moreAdvanced Queries With SQL That Will Save Your Time

Quoting Concatenate

In our last note we used wrapr::qe() to help quote expressions. In this note we will discuss quoting and code-capturing interfaces (interfaces that capture user source code) a bit more. My position on code-capturing interfaces (or non-standard-evaluation/NSE) is: if poorly handled, they can be a large interface price/risk to pay for the minor convenience of … Read moreQuoting Concatenate

Art of Generative Adversarial Networks (GAN)

Dec 16, 2018 Art of Generative Adversarial Networks Code link for all the work mention in the post:- We had this pleasure of working on Generative adversarial network project for our final project for Business Data Science in our curriculum. Though we could have chosen any other subject as our final project yet we went … Read moreArt of Generative Adversarial Networks (GAN)

ProGAN: How NVIDIA Generated Images of Unprecedented Quality

Progressively growing GANs enables them to get bigger and more stable The people in the high resolution images above may look real, but they are actually not — they were synthesized by a ProGAN trained on millions of celebrity images. “ProGAN” is the colloquial term for a type of generative adversarial network that was pioneered at NVIDIA. It … Read moreProGAN: How NVIDIA Generated Images of Unprecedented Quality

Robots that Reason

Inorganic knowledge traditions with model-based reinforcement learning This essay explores the concept of inorganic knowledge traditions capable of sequential improvement using model based reinforcement learning Many behavioral economists presently believe that there are two primary methods used by humans for strategic decision making. One is fast, intuitive and unconscious — what has been called System 1 thinking. … Read moreRobots that Reason

Simple House Price Predictor using ML through TensorFlow in Python

The profession of reality is moving into the 21st century, and as you can imagine home listings are flooding the internet. If you have ever looked at buying a home, renting an apartment, or just wanted to see what the most expensive home in town is (we have all been there), then chances are you … Read moreSimple House Price Predictor using ML through TensorFlow in Python

Regression Analysis: Linear Regression

3. Model Building in R I have used the dataset which contains the details of 2,201 flights. The descriptions of each variable are as below. 3.1) Datasets schedtime : the scheduled time of departure (using the 24-hour clock) carrier : the two-letter code indicating which airline operated the flight deptime : the actual departure time dest : the three-letter code … Read moreRegression Analysis: Linear Regression

What’s the fuss about Regularization?

As a newbie to machine learning most people get excited when their training error starts reducing. They try hard further and it starts reducing even further, their excitement knows no bounds. They show their results to master Oogway ( elderly wise tortoise in Kungfu Panda) and he calmly says well not a good model you … Read moreWhat’s the fuss about Regularization?

Finding Local Events Using Twitter Data

Project by David Chen, Ashwin Gupta, Shruthi Krish, Raghav Prakash, Wei Wang Twitter is a social media platform that millions of users use to share updates about their lives. Often, these tweets are about local events happening around the user. Though news agencies report on local events, the time it takes an agency to learn … Read moreFinding Local Events Using Twitter Data

Predicting hospital length-of-stay at time of admission

Exploring an important healthcare performance metric Photo by Hush Naidoo on Unsplash Project Overview Predictive analytics is an increasingly important tool in the healthcare field since modern machine learning (ML) methods can use large amounts of available data to predict individual outcomes for patients. For example, ML predictions can help healthcare providers determine likelihoods of disease, … Read morePredicting hospital length-of-stay at time of admission

Using Markov Chain Monte Carlo method for project estimation

Using TensorFlow probability for Hamiltonian Sampling Free photo from https://pixabay.com One type of criticism I received for the previous work on project estimation is that the log-Normal distribution has short tails. And this is true, despite all the benefits of log-Normal distribution. The reason is very simple: when fitting the data to the distribution shape … Read moreUsing Markov Chain Monte Carlo method for project estimation

Processing Time Series Data in Real-Time with InfluxDB and Structured Streaming

This article focuses on how to utilize a popular open source database “Influxdb” along with spark-structured streaming to process, store and visualize data in real time. Here, we will go in detail over how to set up a single node instance of Influxdb, how to extend the Foreach writer of SPARK to use it to … Read moreProcessing Time Series Data in Real-Time with InfluxDB and Structured Streaming

Word associations from the Small World of Words

Do you subscribe to the Data is Plural newsletter from Jeremy Singer-Vine? You probably should, because it is a treasure trove of interesting datasets arriving in your email inbox. In the November 28 edition, Jeremy linked to the Small World of Words project, and I was entranced. I love stuff like that, all about words … Read moreWord associations from the Small World of Words

Modeling tree height and basal area in the Finger Lakes National Forest, NY

I tried my hand at using the R package, randomForest to create two regression models for tree height and basal area based off some lidar and field-collected data in the Finger Lakes National Forest, NY. Disclaimer: this project was my first real taste of R. Earlier in the semester I had done some simple learning into … Read moreModeling tree height and basal area in the Finger Lakes National Forest, NY

Request for comments on planned features for futile.logger 1.5

I will be pushing a new version of futile.logger (version 1.5) to CRAN in January. This version introduces a number of enhancements and fixes some bugs. It will also contain at least one breaking change. I am making the release process public, since the package is now used in a number of other packages. If … Read moreRequest for comments on planned features for futile.logger 1.5

Text Generation Using RNNs

Generate characters from Alice in Wonderland Introduction Text generation is a popular problem in Data Science and Machine Learning, and it is a suitable task for Recurrent Neural Nets. This report uses TensorFlow to build an RNN text generator and builds a high-level API in Python3. The report is inspired by @karpathy ( min-char-rnn) and … Read moreText Generation Using RNNs

Introduction to Interactive Time Series Visualizations with Plotly in Python

Introduction to Plotly Plotly is a company that makes visualization tools including a Python API library. (Plotly also makes Dash, a framework for building interactive web-based applications with Python code). For this article, we’ll stick to working with the plotly Python library in a Jupyter Notebook and touching up images in the online plotly editor. When … Read moreIntroduction to Interactive Time Series Visualizations with Plotly in Python

The Ultimate NanoBook to understand Deep Learning based Image Classifier

The first and most important step of our journey: As I have said before, we are going to simply ask questions that will guide us to build an image classifier. For the sake of brevity, we will call Image Classifier an ICNow, we are ready to start our journey. So let us ask the first question: … Read moreThe Ultimate NanoBook to understand Deep Learning based Image Classifier

Applying GANs to Super Resolution

SRGAN Results from Ledig et al. [3] Generative adversarial networks (GANs) have found many applications in Deep Learning. One interesting problem that can be better solved using GANs is super-resolution. Super-resolution is a task concerned with upscaling images from low-resolution sizes such as 90 x 90, into high-resolution sizes such as 360 x 360. In this … Read moreApplying GANs to Super Resolution

A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way

Artificial Intelligence has been witnessing a monumental growth in bridging the gap between the capabilities of humans and machines. Researchers and enthusiasts alike, work on numerous aspects of the field to make amazing things happen. One of many such areas is the domain of Computer Vision. The agenda for this field is to enable machines … Read moreA Comprehensive Guide to Convolutional Neural Networks — the ELI5 way

Dealing With Class Imbalanced Datasets For Classification.

Skewed datasets are not uncommon. And they are tough to handle. Usual classification models and techniques often fail miserably when presented with such a problem. Although your model could get you to even a 99% accuracy on such cases, yet, if you are measuring yourself against a sensible metric such as the ROC Auc score, … Read moreDealing With Class Imbalanced Datasets For Classification.

Google Landmark Recognition using Transfer Learning

Image classification with 15k classes! Project by Catherine McNabb, Anuraag Mohile, Avani Sharma, Evan David, Anisha Garg Dealing with a large number of classes with very few images in many classes is what makes this task really challenging! The problem comes from a famous Kaggle competition, the Google Landmark Recognition Challenge. Training set contains over 1.2 … Read moreGoogle Landmark Recognition using Transfer Learning

Advent of Code: Most Popular Languages

You might have heard of the Advent of Code,a 25-day challenge involving a programming puzzle a day, to be solvedwith the language of your choice. I’ve noted the popularity of thisactivity in my Twitter timeline but also in my GitHub timeline whereI’ve seen the creation of a few advent-of-code or so repositories. AoC is largely … Read moreAdvent of Code: Most Popular Languages