Month: December 2020

The question is whether we can only calculate the convolution with the sparse data efficiently instead of scanning all the image pixels or spatial voxels. One intuitive thinking is, regular image signals are stored as matrix or tensor. And the corresponding convolution was calculated as dense matrix multiplication. The sparse signals are normally represented as … Read more How does sparse convolution work?

Since neural networks are essentially a sequence of operations, one can visualize these operations as nodes on a graph. In Tensorflow 1.x, the way to execute your training was to write the relationships (or edges) of this computational graph (e.g. the layers of a neural net) and then compile it. Once compiled, you would provide … Read more Custom Training Loops for Medical Image Segmentation in Tensorflow 2.x

Inefficiencies and limitations faced by companies using traditional optimization methods and how combinatorial optimization might be the future of logistic industries. Photo by Markus Spiske on Unsplash Title of the article is very oxymoronic: having an optimization and inefficiencies in the same context. But it is very true looking at the trend and current practices … Read more Optimization: A notorious road to Structured Inefficiency and transition to Combinatorial…

Components of a Tree A decision tree has the following components: Node — a point in the tree between two branches, in which a rule is declaredRoot Node — the first node in the treeBranches — arrow connecting one node to another, the direction to travel depending on how the datapoint relates to the rule … Read more Decision Trees — The Maths, The Theory, The Benefits

A step-by-step approach to getting started and developing your skills in this rapidly changing field. Photo by Myriam Jessier on Unsplash For several years, Data Scientist was ranked as the best job in America by Glassdoor. Today it no longer holds the top spot in job rankings but it still ranks near the top of … Read more How to get started with data science in 2021.

Find working codes and trained models here Chinatown @ Singapore (Photo credit to Lily Banse on Unsplash) Introduction In the era of Covid-19, we become more reliant on virtual interactions such as Zoom meetings / Teams chat. These livestream webcam videos have become a rich data source to explore. This article will explore the use … Read more Real-time Age, Gender and Emotion Prediction from Webcam with Keras and OpenCV

Who does the data choose to win the largest international football tournament yet? Image by Michal Jarmoluk from Pixabay. The grandest and most exciting of all football tournaments is still a ways off (2022), but in times like these I find solace in the fact that there are better things (like the next World Cup) … Read more Simulating the FIFA World Cup 2022

setup, debug, version control, and deployment Photo by Greg Rakozy on Unsplash Many data scientists like to use Jupyter Notebook or JupyterLab to do their data explorations, visualizations, and model building. I know some data scientists refuse to use Jupyter Notebook. But, I love to use Jupyter Notebook/Lab to do my experiments and explorations. Here … Read more Jupyter Workflow for Data Scientists

Initialize a Flexdashboard from R Studio using File > New File > R markdown > From Template > Flex Dashboard, save, and knit the document. This creates a static, two-column dashboard with one chart on the left and two on the right: Step 1. flexdashboard “`1. Add runtime: shiny to the YAML header at the … Read more Create an Interactive Dashboard with Shiny, Flexdashboard, and Plotly

This is my go-to method. The main difference between the CatBoost, XGBoost, and H2O versions relate to the ML parameters available for tuning. All functions listed in this blog have working examples in the GitHub README, the R help files (which can be opened in your R session) or the package reference manual. Five feature … Read more The Most Feature-Rich ML Forecasting Methods Available: Compliments of RemixAutoML

From Installation to Implementation: Part 2 Photo by Maarten van den Heuvel on Unsplash In part one, I started building a database to use for a monthlybudgeting application. After installing MongoDB, we discusseda brief overview of how to create a database, our firstcollections, and inserting documents. With some generalknowledge about MongoDB, the goal is to … Read more MongoDB

Through the work of Claude Shannon and many others after him, we can conclude that Markov models: Describe the world in a more realistic way, Are a useful tool to make long-term predictions about a system or process. Realistic tool to describe the world Most real-world systems and phenomena involve multiple parts, which are rarely … Read more Markov models and Markov chains explained in real life: probabilistic workout routine

To approach this analysis, I began by first establishing a null hypothesis as a foundation for further hypothesis testing. In the case of the problem, this is the assertion that both the treatment and control groups would see the same revenue ratio before and after the experiment — in other words, the difference in revenue … Read more Analyzing eBay’s AdWords Spending: Is This Extra Expense Worth It?

As the arsenal of classification algorithms increased dramatically, it became more and more tempting to use several classifiers and then combine their decisions to gain in accuracy and avoid the burden of choosing the right one. Note that a combination of classifiers remains itself a classifier and the no free lunch theorem also applies to … Read more Insights on Classifier Combination

Photo by unsplash.com/@andrewtneel This article will serve as the first part of a potentially ongoing series, looking at the mathematical concepts that drive key parameters in the machine learning algorithms employed in data science. My goal in these posts will be to express key concepts in as simple and non-technical a language as possible, while … Read more Under the Hood: Using Gini impurity to your advantage in Decision Tree Classifiers

A foreign data wrapper is an extension available in PostgreSQL that allows you to access a table or schema in one database from another. Foreign data wrappers can serve all sorts of purposes: Completing a data flow cycle Your data may be segregated across databases, but still related in ways that makes being able to … Read more How to Set Up a Foreign Data Wrapper in PostgreSQL

In this problem, we are given a grid (4 x 4 in this case). The goal is to reach either the top-left or the bottom-right square (gray colored) from any other square on the grid, with maximum reward. You can jump one square in either of the North, South, East, or West directions from any … Read more Dynamic Programming in RL

A practical guide to getting started in Deep Learning Photo by Giorgio Grani on Unsplash What is Deep Learning? Deep learning is a subset of machine learning algorithms that use neural networks to learn complex patterns from large amounts of data. Due to advances in computing and the amount of data being acquired, these algorithms … Read more Using MATLAB’s Deep Learning Toolbox | Part 1: Predicting Cancer Malignancy Using Shallow Neural…

A simple guide to help you create one. No Expertise Required! My Website. BG photo by Daniel Leone on Unsplash A personal portfolio website is like a digital resume. It can make a lot of difference in your career. It can be a great platform for you to publish your work and portfolio, it could … Read more How to create a stunning personal portfolio website for free

Tables with gt and animation with tweenr I recently returned to the R package avr, which runs a range of alternative voting procedures, to add more functionality, and in the process, got to grips with two visualisation packages: gt and tweenr. Each provides a different solution to the problem I had, which was how to … Read more Visualisation of ranked choice voting in R

From individual skills to business development, data professionals have many opportunities in the next few years. Photo by Paweł Czerwiński on Unsplash In response to an atypical year, companies rely on data and analytics leaders to accelerate innovation and create new routes to generate revenue. However, recent research involving business leaders in the U.S., U.K., … Read more Trends in Data Science That Will Change Business Strategies

An Explanation for Splitting Data into Training and Testing Sets Photo by Isaac Smith on Unsplash On my first job out of college, I was tasked with streamlining how a company made purchases. While a big project that encompassed many factors, such as lead times and order quantities, the most challenging part was determining how … Read more The Danger of Overfitting a Model

Data Engineering Books Building foundations and framing your viewpoint towards data engineering Photo by Ahmad Ossayli on Unsplash About 3 years ago, I started my IT career as a Data Engineer and tried to find day-to-day solutions and answers surrounding the data platform. And, I always hope that there are some resources like the university … Read more 5 Books for Data Engineers

This had happened so many times that — machine learning models being naturally lazy — the model gave up trying to learn and instead memorized the data, something a neural network has no trouble doing at all. Hence, when presented with data it had truly never seen before, it flunked. The model had already seen … Read more Be Careful of This Data Science Mistake I Wasted 30 Hours Over

A deep dive into the latest literature in time series forecasting and how you can use them for your business use cases Photo by Brent Ninaber on Unsplash This year at the Neural Information Processing Conference, authors published a number of new papers focusing on time series forecasting and classification. Here I will briefly review … Read more Breakthroughs in Time Series Forecasting at Neurips 2020

The results of GridSearchCV can be somewhat misleading the first time around. The best combination of parameters found is more of a conditional “best” combination. This is due to the fact that the search can only test the parameters that you fed into param_grid. There could be a combination of parameters that further improves the … Read more GridSearchCV for Beginners

UMAP dimension reduction and DBSCAN for clustering MNIST database within KNIME Clustering. Olives and leaves. Shapes and colours. (Image by author) Knime is a free and open-source data analytics, reporting and integration platform. KNIME integrates various components for machine learning and data mining through its modular data pipelining concept. For people like me, who do … Read more Embeeding and clustering combining Knime and Python