Notes on turning messy data into something you can model, mostly in Python.Follow along with the RSS feed.
pandas treats consecutive rows as consecutive periods, so .diff() and .shift() go wrong when a unit skips one. A fast way to insert the missing periods, including uneven ones such as months. Aug 6, 2024
Cropping a saved figure to its contents can cut off legends and annotations placed outside the axes. Two functions collect them so Matplotlib keeps them in the frame. Aug 6, 2024
A decorator that saves a slow function’s DataFrame to disk and loads it on the next call. It recomputes only when the function’s code changes, ignoring comments and formatting. Apr 1, 2023
A dataset split over many files loads one file at a time. A short function reads them in parallel with Python’s standard library, for any format pandas reads. Aug 4, 2021