The year is coming to an end, and if any of you work in the relevant departments of the company’s financial accounting, you must be busy preparing various annual financial statements. Especially for beginners who have just entered this field, how to design reports to clearly show the financial analysis and business operation status is … Read more 6 Data Analysis Methods to Help You Make Great Financial Statements
What I learned from analyzing 300K German online deals Every year on Black Friday, people try to find the best deals hoping to save a ton of money. In the U.S. the Black Friday craziness has even lead to 12 deaths and 117 injuries thus far. If you are like me and prefer to do … Read more Data Science and Black Friday: When, how and where to find the best deal?
The where() function will return elements from an array that satisfy a certain condition. Let’s explore it with an example. I’ll declare an array of grades of some sort (really arbitrary): You can now use where() to find, let’s say, all grades that are greater than 3: Note how it returns the index position. The … Read more Top 4 Numpy Functions You Don’t Know About (Probably)
You should have a directory for every project, and a virtual environment for every directory. This structure does two important things: It keeps your stuff organized appropriately, which makes it easier to keep projects separate, manage dependencies, and keep out things that shouldn’t be there. (Who likes having to undo git commits?) It lets you … Read more Power up your Python Projects with Visual Studio Code
Data science is a very hands-on and practical field. Data science requires a solid foundation in mathematics and programming. As a data scientist, it is essential that you understand the theoretical and mathematical foundations of data science in order to be able to build reliable models with real-world applications. In data science and machine learning, … Read more Theoretical Foundations of Data Science— Should I Care or Simply Focus on Hands-on Skills?
Comparing models in a social media NLP challenge. Zen and the Art of Motorcycle Maintenance was one of my favorite books in college. Set amidst a father-son motorcycle journey across the United States, the book considers how to lead a meaningful life. Arguably, the key message expounded by the author, Robert Pirsig, is that we … Read more Zen and the Art of Model Optimization: Comparing models in a social media NLP challenge
Artificial Intelligence (AI) is a complex and evolving field. The first challenge an AI aspirant faces is understanding the landscape and how he could navigate through it. Consider this, if you are travelling to a new city, and if you don’t have the map, you will have trouble to navigate the city and you will … Read more How to Navigate Artificial Intelligence Landscape?
Exploring the Definition of Bias in AI When the word ‘data’ is talked about in artificial intelligence ‘bias’ is often mentioned alongside the given discussion. I will have a general discussion of bias first that will be related to subjectivity and objectivity. Whereafter I will return to a summary of things to consider in relation … Read more Subjective and Objective in the Development of Artificial Intelligence
Mean Reversion with Bollinger bands gone wrong Photo by Rick Tap on Unsplash A few years ago we did some work with a Trading simulation. Our strategy was Mean Reversion with Bollinger Bands. Thankfully, it was only a simulation as the losses from share variations and software bugs were horrific. In this article we offer … Read more Algorithmic trading
And build a great mathematical foundation Photo by Antoine Dautry on Unsplash Awhile back I wrote an article on the top books to get started with data science: The Top 3 Books to Get Started with Data Science Right Now And build a great foundation of knowledge towardsdatascience.com One of the readers left a comment … Read more The Top 3 Books to Learn Math for Data Science Right Now
Why does management need to observe data-science-related KPIs? Binoculars Man, Pixabay. Observability for data-science (DS) is a new and emerging field, which is sometimes mentioned in tandem with MLOps or AIOps. New offerings are being developed by young startups to address the lack of monitoring and alerts for everything data-science. However, they are mostly addressing … Read more Data-Science Observability For Executives
Yes. It does. We get a 2x improvement in run time vs. just using the function as it is. So what exactly is happening here? Source: How increasing data size effects performances for Dask, Pandas and Swifter? Swifter chooses the best way to implement the apply possible for your function by either vectorizing your function … Read more Add this single word to make your Pandas Apply faster
In the current business climate, application modernization represents both a significant opportunity and a technological challenge. Image Source: Pixabay In today’s digital economy, organizations that run legacy applications run the risk of being disrupted and putting themselves in a profoundly uncompetitive position. Those with an agile environment, hosting core apps on the hybrid Cloud, are … Read more How and Why Businesses Should Modernize Applications
As usual, let’s first load the data. For this exercise, we will use Boston housing data available in sklearn.datasets. Loading the Boston data and splitting it into training and testing dataframe Now, to accomplish model averaging, we will use RMSE (root mean squared error) as the model fitness. Then we will average them using the … Read more Model Averaging: A Robust Way to Deal with Model Uncertainty
What you must put in place to achieve tangible success using machine learning and artificial intelligence technologies. Image Source: Pixabay AI can provide a host of benefits for the enterprises and organizations of today. From understanding customer behavior to fraud detection, visualizing analytical sentiments, and predicting machine failure. Machine learning, if implemented efficiently, promises to … Read more 4 Ways to Successfully Scale AI and Machine Learning for Businesses
The above dataset contains a daily summary of prices where the CHANGE column is the percentage change of the last price of the day (PRICE) with respect to the first (OPEN). Goal: To make things simple, we’ll focus on predicting if the price will rise (change > 0) or fall (change ≤ 0) the following … Read more Predicting Bitcoin Price with Business News (Python)
Being a travel lover, my next year’s goal is to travel as much as I can. I’m not very picky with destinations; however, I do have a few limitations. First, I can only travel on weekends since I work in an office from Monday to Friday. Second, I want to visit as many different cities … Read more Traveling through Europe (in a nerdy way)
Skepticism The Cambridge dictionary states that skepticism is “an attitude that shows you doubt whether something is true or useful.” Put a highly imbalanced data set, where one target variable represents 97% of the data, into a machine learning model and it will predict with a seemingly almost perfect level of accuracy. If we took … Read more Soft Skills for Data Science
If you are getting a struggle to understand your data, you might want to try this with these super basic data analytics tool e.g. python, SQL and excel pivot table. The ultimate outcome is the dashboard-like table and which enables you to dig actionable insight by just slice and dice. So what to do first? … Read more A simple trick for exploratory data analysis
From Pinterest Statistics can be made to prove anything even the truth! So it is of paramount importance to really understand it. Statistics is dealing with the collection, analysis, interpretation, and presentation of masses of data. Descriptive statistics has to do with methods of summarizing the information we have gathered for analysis. Descriptive statistics can … Read more Telling the full story of Descriptive Statistics with numbers!
Apple may not like NVIDIA cards, the solution is called PlaidML+OpenCL PlaidML is a software framework that enables Keras to execute calculations on a GPU using OpenCL instead of CUDA. This is a good solution to do light ML development on a Mac without a NVIDIA eGPU card. Massively parallel programming is very useful to … Read more GPU-Accelerated Machine Learning on MacOS
Feature Engineering: 4 Discretization Techniques to Learn. Discretization is the process through which we can transform continuous variables, models or functions into a discrete form. We do this by creating a set of contiguous intervals (or bins) that go across the range of our desired variable/model/function. Continuous data is Measured, while Discrete data is Counted. … Read more An Introduction to Discretization in Data Science
AutoML “Automated Machine Learning” is under fire lately by the Data Science Community. I had some criticism published in on Google Cloud ML Google Cloud’s AutoML first look (the Google AutoML Team did answer some of my requests, see **new** things at the end of this article) This post will show it is completely reasonable … Read more Build a useful ML Model in hours on GCP to Predict The Beatles’ listeners
Even if your project is small, YAML files can add a lot to the flexibility and reusability of it. Photo by Max Nelson on Unsplash During my Master in Data Science, no one had told me about the necessity of config files and how much easier my life could have been with them. To be … Read more 5 Reasons To Use YAML Files In Your Machine Learning Projects
Maybe trying to design it isn’t the way… Image by Skeeze, Pixabay Over the last ten years, practitioners of machine learning and deep learning have made stunning progress in narrow AI problems. Computers have gotten very good at a variety of tasks, such as detecting things in images, which were previously only thought possible for … Read more Could We Evolve an Artificial General Intelligence?
Confused why R² is negative? Read below to find out more. Machine learning is continuously growing and is said to affect all domains and bring a radical change in the way human race functions. Few advancements have already started having an impact on the society like the fraud detection systems, the online loan approval systems, … Read more Regression: An Explanation of Regression Metrics And What Can Go Wrong
In this article, we’ll see some of the most useful techniques used to clean and process the data with Pandas library. Photo by Kevin Ku on Unsplash Data Science involves the processing of data so that the data can work well with the data algorithms. Data Wrangling is the process of processing data, like merging, … Read more Data Wrangling using Pandas library
How learning about S3 can help you to design an ideal data lake in AWS If you’ve had any connection to the data world, you’ve probably heard some memorable, often quirky phrase about how valuable data is. I’m thinking of phrases like… “Data is the new oil.” “Without big data, you are deaf and blind … Read more An AWS Data Lake with S3 Explained!
You can download complete file in https://kreilabs.com/wp-content/uploads/2019/12/Metrics_table.pdf Here we can see pyhton implementation for table metrics in matrix_metrix routine: import sklearn.metricsimport mathdef matrix_metrix(real_values,pred_values,beta):CM = confusion_matrix(real_values,pred_values)TN = CM[0][0]FN = CM[1][0] TP = CM[1][1]FP = CM[0][1]Population = TN+FN+TP+FPPrevalence = round(TP / Population,2)Accuracy = round( (TP+TN) / Population,4)Precision = round( TP / (TP+FP),4 )NPV = round( TN / … Read more Metrics and Python II
Photo by Bill Jelen on Unsplash When I work with structured data, Pandas is my number one go-to tool. I think that comes at no surprise as Pandas is the most popular Python library for data manipulation, exploration, and analysis. It offers a lot of functionality out of the box. Additionally, various other modules exist … Read more How to Supercharge your Pandas Workflows
The evolution of Artificial Intelligence and the new wave of “Future AI” Today’s AI Without any doubt, today’s biggest buzzword is Artificial Intelligence or AI. Most prominent research organizations, including Gartner, McKinsey, and PWC, have glorified the future of AI with mind-blowing statistics and future predictions. Here is the PWC’s report (2018), where it predicts … Read more Traditional AI vs. Modern AI.
A lot of the times, face detection is not the primary focus of the application but is an important component and many developers are just overwhelmed with the idea of adding face detection to their applications. If the application is to count the number of people entering and leaving a shop, developers need to detect … Read more Integrate Face Detection in your App
How to choose, leverage, evaluate, and control automated software for debt managers and collectors Image Source: Pixabay Debt collection as an activity dates back to the emergence of the first credit transactions. Even in the primitive barter economies that existed before the rise of money as the medium of exchange, we see the criticality of … Read more How to Choose Automation Services for the Debt Collection Management Industry
Finally a library you can pick up in under 5 minutes Photo by Eftakher Alam on Unsplash The biggest advantage of python is the ease of use and the abundance of libraries for just about anything. With a few lines of code, there is nothing you couldn’t do. As long as your python scripts are … Read more Learn How to Quickly Create UIs in Python
As a data scientist in an organization you frequently find yourself in a couple of situations: you have a dataset, you want to extract some useful information you have a business problem, you want to find a data-driven solution The first situation is actually a common one, basically, this means doing all the things you … Read more A data science project cycle
Will your career be affected by advances in technology, or will AI just make your job easier? Image Source: UnSplash In a recent speech, Forrester vice president and principal consultant Huard Smith said that the human aspect of many professions would be “all gone” by 2030 due to advances in AI and ML technology. In … Read more Seven Jobs That AI Could Replace by 2030
There is a big misconception among the meaning and differences of these terms. Deep Learning (DL) is a subfield of Machine Learning (ML), which is a subfield of Artificial Intelligence (AI). In gross terms: AI incorporates cognitive functions, like learning and problem-solving. It is a system that can think, perceive and derive action from that … Read more Get the DL on AI
We analyse 200k tech news articles with the popular topic modelling algorithm LDA Kristóf Gyódi, Łukasz Nawaro, Michał Paliński, Maciej Wilamowski (DELab UW, University of Warsaw) Image by: DELab UW As part of the NGI Forward project, DELab UW is supporting the European Commission’s Next Generation Internet initiative with identifying emerging technologies and social issues … Read more Mapping the tech world with topic modelling
How to design an algorithm to predict loan default rates source: https://blogs.oracle.com/r/data-science-maturity-model-deployment-part-11 Imagine building a supervised machine learning(ML) model to decide whether a loan application should be approved. With the model confidence level (probability) in successful applications, we can calculate the risk-free loanable amount. The deployment of such ML-model is the goal of this project. … Read more Deployment of Machine learning Model Demystified (Part 1)
Images, Music, Emotion and much more When people hear the phrase, “Deep Learning,” many have only a vague idea of what it really means. Deep Learning is a Machine Learning paradigm that uses an artificial neural network as a learning model. But first, why does it matter? It can help make our lives much easier … Read more It’s Deep Learning Times: A New Frontier of Data
I believe that 0 is one of the most underrated numbers. But I like 0. I like it because it’s a little rebel in Mathematics. Seriously, when it comes to 0, you always have to think twice. You shouldn’t divide by 0 and many times, 0 is different from the other numbers. I also like … Read more 1 Minute Math: Is 0 Even or Odd?
An AI-inspired Revolution in the Life Sciences Source: Shutterstock The scientific revolution was ushered in at the beginning of the 17th century with the development of two of the most important inventions in history — the telescope and the microscope. With the telescope, Galileo turned his attention skyward, and advances in optics led Robert Hooke … Read more Unveiling Biology with Deep Microscopy
If you haven’t seen the video. Check it out. Reinforcement learning has seen a lot of success over the past years. We have seen AIs beat pro players in Go, Dota, and Starcraft all by using reinforcement learning. The former go champion Lee Se-dol even quit the game altogether after his loss against the superior … Read more How OpenAI solved Rubiks Cube with a Robot Hand
Despite their fame as fierce killers, reintroducing wolves in the Yellowstone national park in 1995 brought life back to the region. As the wolves chased the deer and moose away from the rivers and valleys, the vegetation began to grow back. Smaller animals like birds and beavers began to settle next to the water, thanks … Read more The shape of population dynamics
Probabilistic programming is becoming one of the most active areas of development in the machine learning space. What are the top languages we should know about? Probabilistic thinking is an incredibly valuable tool for decision making. From economists to poker players, people that can think in terms of probabilities tend to make better decisions when … Read more A Gentle Introduction to Probabilistic Programming Languages
Don’t let your System 1 judge this post. Here are 5 books that will strengthen your decision-making skills. They will hopefully make your two systems of cognitive processes better, enhancing both your intuitive and deliberate thinking. Below you will find my reasons of why you should reach each one of them together with a quote … Read more 5 Books To Improve Your Fast & Slow Thinking
Overlaying two plots to make one chart http://www.terrystickels.com/math-art/images/synchronized_curves_std.jpg Intro The libraries, code, and visuals will be down below but first I wanted to offer a brief introduction as to why I decided to share this with everyone in this community. If you just want to skip down to the tutorial just skip the intro. While … Read more Combo Charts with Seaborn and Python
In this blog, we’ve highlighted inequalities in access and outcomes within IAPT. This is well-established within the field and discussed within the IAPT manual, which outlines evidence-based guidance for effective and efficient delivery of IAPT services. The longitudinal evidence suggests improvements are being made. However, further progress is needed and while we focus on deprivation, … Read more Inequalities in English NHS talking therapy services: What can the data tell us?
From one side, we had problems that business faced and from another side, the capabilities of AI technologies. To extract the synergy from the technologies and problems, I started thinking about how we can answer the main question: How to copy myself and sell to all the companies in the world for covering repeated work? … Read more Digital Skills as a Service (DSaaS)
My advice to an entry-level programmer would be, to stick with the following 3P strategy which I have formulated and it has really helped me in my career. Perseverance Practice Project work Perseverance “Perseverance is not a long race; it is many races one after another.” — Walter Elliot At the beginning, coding always seems … Read more 3P Strategy to be successful for entry-level programmers