New features in Azure Monitor Metrics Explorer based on your feedback

A few months ago, we posted a survey to gather feedback on your experience with metrics in Azure Portal. Thank you for participation and for providing valuable suggestions! We want to share some of the insights we gained from the survey and highlight some of the features that we delivered based on your feedback. These … Read more New features in Azure Monitor Metrics Explorer based on your feedback

Google Drive + Google Colab + GitHub; Don’t Just Read, Do It!

How GitHub, Google Colab and Google Drive work together; How to deal with custom file, and push Juypter notebook changes to GitHub Repo. Interaction Between The Three Components I have recently been accepted into Phase I of Bertelsmann Tech Scholarship Challenge Course — AI Track Nanodegree Program. In the lessons, Jupyter notebook exercises are provided … Read more Google Drive + Google Colab + GitHub; Don’t Just Read, Do It!

A list of beginner-friendly NLP projects—using pre-trained models

If you’re interested in studying machine learning from the ground up, there are plenty of great resources. Organizations like fast.ai have made it so that anyone with a vaguely technical background can learn the foundations of machine learning and train their own models. If you’re interested in building production software with machine learning, however, there … Read more A list of beginner-friendly NLP projects—using pre-trained models

American Sign Language Hand Gesture Recognition

Based on these images, it is easy to understand why our neural network has trouble distinguishing between these two signs. In future work, we will use images with higher resolution that allow for more intricate details to be extracted from the images. Hopefully, this will further improve the accuracy of our model. An additional limitation … Read more American Sign Language Hand Gesture Recognition

What are algorithms really and does it matter?

Algorithms capture it’s builders’ subjective opinions. Here is why we should always consider this. What are algorithms really? Welcome to the age of algorithms — mathematically encoded models that influence our book and movie wish lists, the curation of our digital worlds, our decision making in social networks and every other corner of our digitally-infused … Read more What are algorithms really and does it matter?

Gender Bias in Admission Statistics? The Simpson-Paradox.

First, let me tell you two facts. Fact 1: In 1973, at the University of California, Berkeley, the overall acceptance rate in four departments for female applicants was roughly 30%. At the same time, the overall acceptance rate for male applicants was roughly 47%. and Fact 2: In each of the departments, the acceptance rate … Read more Gender Bias in Admission Statistics? The Simpson-Paradox.

How Machine Learning Is Helping Doctors, Nurses, Bankers, Transport Workers, and Manufacturers.

Tangible examples of how data science is helping to advance crucial industries at a compelling rate. Image Source: Pixabay.com With the advent of wearable technologies such as the Fitbit and the Apple Watch, devices are collecting billions of data points on consumers — everything from their sleep cycles to their step counts to their heart … Read more How Machine Learning Is Helping Doctors, Nurses, Bankers, Transport Workers, and Manufacturers.

Pandas Tips & Tricks: Need For Speed

DATA ANALYTICS LIKE A PYTHON PRO A Personal Favorite 1-Liner Kungfu Panda My last post demonstrated a simple process for evaluating a set of face pairs to determine whether or not the two are blood relatives. Several snippets were breezed over like black-boxes. Let us look at one of my those snippets, a simple 1-liner: … Read more Pandas Tips & Tricks: Need For Speed

A Data Scientist’s Toolkit to Encode Categorical Variables to Numeric

Encoding categorical variables into numeric variables is part of a data scientist’s daily work. I have been wanting to write down some tips for readers who need to encode categorical variables. The techniques in this article are the frequently used techniques in my professional work. I hope this article will assist you for any additional … Read more A Data Scientist’s Toolkit to Encode Categorical Variables to Numeric

PSA: Code Legibly or Be Hated

Source: Imgur This is a public service announcement for all data scientists. Please code legibly. If not for the sanity of your collaborators then for your own sanity x months/years from now. As a Masters student at the best Bayesian statistics program in the country, I was surrounded by hard-working, well-honed minds and challenging problems. … Read more PSA: Code Legibly or Be Hated

How AI and Deep Learning is Decoding Ancient Geoglyphic Etchings in Peru

Scientists recently discovered 142 more ‘Nazca Lines,’ and AI is on the hunt for more hidden relics from a lost civilization. Image Source: MachuTravelPeru.com Triangles, trapezoids, and a variety of other geometric shapes and patterns — that’s what can be seen when one flies a drone over Peru’s high desert in South America. These colossal … Read more How AI and Deep Learning is Decoding Ancient Geoglyphic Etchings in Peru

Getting Started with Apache Zeppelin on Amazon EMR, using AWS Glue, RDS, and S3

Explore the use of Apache Zeppelin on Amazon EMR for data analytics and data science, using a series of pre-made Zeppelin notebooks. There is little question big data analytics, data science, artificial intelligence (AI), and machine learning (ML), a subcategory of AI, have all experienced a tremendous surge in popularity over the last 3–5 years. … Read more Getting Started with Apache Zeppelin on Amazon EMR, using AWS Glue, RDS, and S3

Your Company is Probably not Ready for AI

I was recently asked why a Data Scientist in our team wasn’t data-sciencing but was instead working on data wrangling: data preparation, cleaning and data quality assurance. Where were the fancy statistics? Where was modelling? A graph perhaps? Or at least a slide that said Artificial Intelligence, Machine Learning, or Deep Learning. I realized that … Read more Your Company is Probably not Ready for AI

Using Python and Robinhood to Create a Simple Buy Low — Sell High Trading Bot

Photo by Ishant Mishra on Unsplash So I have been messing with Robinhood lately and been trying to understand stocks. I am not a financial advisor or anything, but I wanted to create a simple trading bot so I could understand robin_stocks a little bit more before I create more complex code. For those who … Read more Using Python and Robinhood to Create a Simple Buy Low — Sell High Trading Bot

Use third-party keys in the cloud with Cloud External Key Manager, now betaUse third-party keys in the cloud with Cloud External Key Manager, now betaSenior Product Manager, Google Cloud

This approach offers several unique security benefits:  Maintain key provenance over your third-party keys. You have strict control over the creation, location, and distribution of your keys.   Full control over who accesses your keys. Because keys are always stored outside Google Cloud, you can enforce that access to data at rest for BigQuery and Compute … Read more Use third-party keys in the cloud with Cloud External Key Manager, now betaUse third-party keys in the cloud with Cloud External Key Manager, now betaSenior Product Manager, Google Cloud

Amazon EC2 Fleet Now Lets You Preferentially use Available Capacity Reservations

Amazon EC2 Fleet simplifies the provisioning of EC2 capacity across different instance types, Availability Zones (AZs), and purchase options to optimize for scale, performance, and cost. Allocation strategies let you determine how EC2 Fleet should select from the instance types and AZs you have specified to fulfill the desired capacity. The new Capacity Reservation usage … Read more Amazon EC2 Fleet Now Lets You Preferentially use Available Capacity Reservations

Predicting MLB Pitch Probability Based on the Game Situation

Can we determine what this pitch will be before it is thrown? Baseball is a wonderful sport for a number person. It was one of the first sports that come to mind regarding the use of data to make decisions at just about every level. Managers use data to make in-game decisions and General Managers … Read more Predicting MLB Pitch Probability Based on the Game Situation

File storage made easier with NetApp Cloud Volumes, now GAFile storage made easier with NetApp Cloud Volumes, now GATechnical Director, Google Cloud Office of the CTOProduct Manager, NetApp

At both NetApp and Google Cloud, we share a mission to offer our users a top-notch file service in Google Cloud. Whether you’re moving workloads to cloud or deploying net-new applications in cloud that need file interfaces, our aim is to offer a highly available, feature-rich, and high-performing file service.  We recently announced two major … Read more File storage made easier with NetApp Cloud Volumes, now GAFile storage made easier with NetApp Cloud Volumes, now GATechnical Director, Google Cloud Office of the CTOProduct Manager, NetApp

BeyondProd: How Google moved from perimeter-based to cloud-native securityBeyondProd: How Google moved from perimeter-based to cloud-native securityProduct Manager, Container securityHorizontal Lead, Cloud Security

At Google, our infrastructure runs on containers, using a container orchestration system Borg, the precursor to Kubernetes. Google’s architecture is the inspiration and template for what’s widely known as “cloud-native” today—using microservices and containers to enable workloads to be split into smaller, more manageable units for maintenance and discovery. Google’s cloud-native architecture was developed prioritizing … Read more BeyondProd: How Google moved from perimeter-based to cloud-native securityBeyondProd: How Google moved from perimeter-based to cloud-native securityProduct Manager, Container securityHorizontal Lead, Cloud Security

The Last Matplotlib Tweaking Guide You’ll Ever Need

As with any of mine coding-related article, we’ll start with the imports. You’ll need the Pandas library for loading in and dealing with tabular data, and you will also need Matplotlib (with some stuff that will enable us to do the tweaking): Great. Now onto the dataset. I’m using the International Airline Passengers dataset, mainly … Read more The Last Matplotlib Tweaking Guide You’ll Ever Need

Federated Learning for Medical AI

Unlike conventional machine learning which works by bringing data to the code, Federated Learning brings code to the data. In healthcare where the available data is never enough, Federated Learning can do wonders by enabling AI models to learn on private data without compromising privacy. Conventional machine learning works on the paradigm of bringing the … Read more Federated Learning for Medical AI

Application of Computational Intelligence to Manage Data Science Activities

A few years ago, Google announced a remarkable finding in the journal Nature. According to the claim, Google’s research team was quick enough to be first to track the spread of influenza across the US. They did it without consulting a physician and without performing a single medical check-up. Such speedy predication is done by … Read more Application of Computational Intelligence to Manage Data Science Activities

MachineX: Image Data Augmentation Using Keras

In this blog, we will focus on Image Data Augmentation using Keras and how we can implement the same. When we work with image classification projects, the input which a user will give can vary in many aspects, like angles, zoom and stability while clicking the picture. So we should train our model to accept … Read more MachineX: Image Data Augmentation Using Keras

BH 1.72.0-2 on CRAN

[This article was first published on Thinking inside the box , and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t. Yesterday’s release of BH 1.72.0-1 was so much fun … Read more BH 1.72.0-2 on CRAN

Software Developers: You’re Learning Machine Learning Upside Down

Don’t be this guy. Most of us learn Machine Learning from the ground up. But the skills developers need to be effective users of ML is something totally different. So what should the forward-looking software engineer learn about AI today? I remember the first time I ever tried to learn to code. I was in … Read more Software Developers: You’re Learning Machine Learning Upside Down

Advancing Azure Active Directory availability

“Continuing our Azure reliability series to be as transparent as possible about key initiatives underway to keep improving availability, today we turn our attention to Azure Active Directory. Microsoft Azure Active Directory (Azure AD) is a cloud identity service that provides secure access to over 250 million monthly active users, connecting over 1.4 million unique … Read more Advancing Azure Active Directory availability

Microsoft is a leader in The Forrester Wave™: Streaming Analytics, Q3 2019

Processing Big data in real-time is an operational necessity for many businesses. Azure Stream Analytics is Microsoft’s serverless real-time analytics offering for complex event processing. We are excited and humbled to announce that Microsoft has been named a leader in The Forrester Wave™: Streaming Analytics, Q3 2019. Microsoft believes this report truly reflects the market … Read more Microsoft is a leader in The Forrester Wave™: Streaming Analytics, Q3 2019

Exploring EU wide data on new car registrations and CO2 efficiency

I just found out that here the EU made large data sets available about every newly registered car in the EU (currently from 2012 to 2018). The data set from 2018 has over 15 million rows and many rows correspond to a single registered car. The following analysis uses a smaller data set that aggregated … Read more Exploring EU wide data on new car registrations and CO2 efficiency

New enhancements for Azure IoT Edge automatic deployments

Since releasing Microsoft Azure IoT Edge, we have seen many customers using IoT Edge automatic deployments to deploy workloads to the edge at scale. IoT Edge automatic deployments handle the heavy lifting of deploying modules to the relevant Azure IoT Edge devices and allow operators to keep a close eye on status to quickly address … Read more New enhancements for Azure IoT Edge automatic deployments

AI Engineers: What They Do and How Much They Cost?

“If you want to command a multiyear, seven-figure salary, you used to have only four career options: chief executive officer, banker, celebrity entertainer, or pro athlete. Now there’s a fifth — artificial intelligence expert.” — Unknown Imagine a glass with balls. This glass is a field of Computer science knowledge, and balls are various fields: … Read more AI Engineers: What They Do and How Much They Cost?

Can Machine Learning Predict The Stock Market?

We test to see if technology is powerful enough to solve the mystery of financial markets If invested correctly and wisely, the U.S. stock market can generate astonishing returns. Following the steps of quantitative investment firms like Two Sigma and Citadel, we are interested in knowing if we can use machine learning to predict stock … Read more Can Machine Learning Predict The Stock Market?

Preventing imbalanced data from leading to abominable classification

Let’s get an insight into the scenario with our dataset. I used GDAL python package to fetch the raster bands of the composite as arrays and converted the data into CSV file format. However, this is not mandatory and was just done for convenience. As is evident from the bar graph, our dataset is highly … Read more Preventing imbalanced data from leading to abominable classification

Seven Important Predictions for Machine Learning in 2020

Will driverless cars have to break the law? Will AI solve 15 of the world’s social challenges? What about small businesses? Read on to find out… Image Source: Pixabay Machine learning and AI are continually progressing. This year, we’ve seen this technology begin to exceed human performance across various narrow and specific domains, including certain … Read more Seven Important Predictions for Machine Learning in 2020

How To Visualize A Decision Tree In 5 Steps

Step 1: Download and install Anaconda for Windows Depending on your Python and computer versions, choose the right Anaconda package to download. Anaconda is a common Python distribution that is usually allowed to download and install in large corporations. Anaconda Python/R Distribution – Free Download The open-source Anaconda Distribution is the easiest way to perform … Read more How To Visualize A Decision Tree In 5 Steps

Quant’s Advice on Generating Trading Ideas

Craft your own world view, stick to your own process, come up with ideas that are uniquely yours… Let’s go on a journey… You’re an aspiring quant trader. You’ve got your starter pack: laptop, Python 3.7, fundamental data, indicators, “Security Analysis” in one hand and “Options, Futures & Other Derivatives” in the other. You’ve got … Read more Quant’s Advice on Generating Trading Ideas

GPU Datascience: Converting CSV to Parquet / ORC faster & Cheaper than using a cluster!

Quite simply, because it is a relatively difficult thing to do computationally. In order to transfer from Row-based to Column-based, the whole dataset needs to be pivoted On top of that, the data is split into stripes and there are various statistics that get calculated and file headers and footers to be written that contain … Read more GPU Datascience: Converting CSV to Parquet / ORC faster & Cheaper than using a cluster!

Can You Predict If a Customer Will Make a Purchase on a Website?

This article discusses our analysis of customer behavior data provided for an E-commerce site. We begin by discussing our motivations for this project. We will then discuss the data set and its features. Then our initial data exploration and feature engineering. Finally we will describe the models we used to predict if a site visitor … Read more Can You Predict If a Customer Will Make a Purchase on a Website?

Comparison of indices of significance in the Bayesian framework

[This article was first published on R on easystats, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t. The bayestestR package has several functions to compute indices of effect … Read more Comparison of indices of significance in the Bayesian framework

Herds of statistical models

by Carlos J. Gil Bellosta Big datasets found in statistical practice often have a rich structure. Most traditional methods, including their modern counterparts, fail to efficiently use the information contained in them. Here we propose and discuss an alternative modelling strategy based on herds of simple models. Big Data: How big datasets came to be … Read more Herds of statistical models

R vs. Python: What’s the best language for Data Science?

[This article was first published on RStudio Blog, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want to share your content on R-bloggers? click here if you have a blog, or here if you don’t. This is a question that we at RStudio hear a lot. With … Read more R vs. Python: What’s the best language for Data Science?

Speeding up R Plotly web apps – R x Javascript part I

Back to blogging! Sorry for the long hiatus, had some personal projects which kept me really occupied over the past few months. Hope to share about them one of these days and potentially even explore open sourcing parts of them but the idea of this post is to transfer some of my learnings over the … Read more Speeding up R Plotly web apps – R x Javascript part I

Automating update of an international database for the Euro Area

Our purpose is to create an international quarterly database for the Euro area that could be updated automatically. We want to build the following series: Foreign demand (without trade between Euro area countries) Foreign interest rate Oil prices Real effective exchange rate Import and export To construct these series we use data from DBnomics. The DBnomics API is called using the … Read more Automating update of an international database for the Euro Area

Could data analysis prevent unnecessary suffering? Lessons to learn from the vaginal mesh scandal.

Lessons to learn from the vaginal mesh scandal. The most frequent words used in the titles of scientific articles concerning meshes for pelvic organ prolapse from 1999–2019. Complications high up. In the aftermath of the vaginal mesh scandal, the NY times and the Guardian, amongst many, wrote extensively about how J&J subsidiary Ethicon failed to … Read more Could data analysis prevent unnecessary suffering? Lessons to learn from the vaginal mesh scandal.

Beautiful paper on HMMs and derivatives

I’ve been talking to Michael Betancourt and Charles Margossian about implementing analytic derivatives for HMMs in Stan to reduce memory overhead and increase speed. For now, one has to implement the forward algorithm in the Stan program and let Stan autodiff through it. I worked out the adjoint method (aka reverse-mode autodiff) derivatives of the … Read more Beautiful paper on HMMs and derivatives

TensorFlow Image Recognition Tutorial using Serverless Architecture — Node JS

A step-by-step tutorial using TensorFlow JS + Serverless Convolutional Neural Network (CNN) based architectures such as VGG-16, GoogLeNet, ResNet, MobileNet, etc., are widely used models for image recognition, object detections from images and videos. For more details about convolutional neural network details, please refer to my older blog. In this blog, I am going to … Read more TensorFlow Image Recognition Tutorial using Serverless Architecture — Node JS

Understanding Data Science And It’s Importance In Modern Business

Some researchers identify parameters such as volume, velocity and variety as the main obstacles to handling data. According to Eric Horvitz and Tom Mitchell (2010) and James Manyika et al. (2011), such data, when analysed and used properly, offer a chance to solve problems, accelerates economic growth, and improves quality of life. It is really … Read more Understanding Data Science And It’s Importance In Modern Business