Here is a little recipe for using good old PCA to do some fast and efficient time series anomaly detection.
too obvious? maybe. probably. Recently i had a need to assess streamlit for some internal DS/ML/Data apps i wanted to build in my job. By "i had a need" i mean i heard it was the new cool thing so i wanted to play with it and feel better about myself. Anyway, as part of … Continue reading streamlit multi-page app minimal example
Taken from this great Talk Python Training course - get the lifetime bundle if you can! You have a list of api endpoints you want to pull data from and collect results into some results list or dataframe for further processing. You could just loop over that list and make a load of requests.get() calls … Continue reading Some asyncio fun/pain
I like an excuse to play with fancy things, so when i first learned about the Matrix Profile for time series analysis, particularly around anomaly detection, i was intrigued. When i learned there was a nice python package (STUMPY) i could just pip install i was outright excited, as one thing i like more than … Continue reading Anomaly Detection using the Matrix Profile
always use a meme to kick off a tutorial Here is an anomaly detection tutorial that i created for my boss and the open source community where i work. It's part of some work i have been doing around adding some anomaly detection functionality into our open source monitoring project. Like most ML projects the … Continue reading Anomaly Detection Tutorial
The Setup This is a little one I was surprised to see. Recently I had a need to do some pretty basic feature engineering to a pandas dataframe prior to training some models. Basically I needed to take differences of each column, apply some smoothing, and then add a number of lagged columns for each … Continue reading Numpy Feature Engineering – 2x Speed Up Over Pandas!
An actual market basket I found in my Google photos. tl; dr; if you find yourself doing some association rule mining using mlxtend but finding it a bit slow then checkout PyFIM - here is a colab I made to get you started. I have recently been looking to do some market basket analysis ("Association … Continue reading Market basket analysis in Python
I've recently been playing around with some time series clustering tasks and came across the tslearn library. I was interested in seeing how easy it would be to get up and running some of the clustering functionality that is already built into tslearn, turns out it was quite easy and straight forward, perfect blog post … Continue reading Time series clustering with tslearn
I've been doing some work that necessitated using the same statistical test from spicy lots of times on a fairly wide pandas dataframe with lots of columns. I spent a bit too much time googling around for the most efficient ways to do this, and even more time re-writing things various way before realizing i … Continue reading Premature Optimization
I was looking around a bit and could not really find any datasets behind the daily updates from the Irish government that get posted here. In particular i was thinking the break out tables of numbers by different dimensions might be of use for anyone looking to analyse the data. So here is a python … Continue reading Ireland Covid19 Data