Putting notebooks and production in the same sentence is a sure way to trigger a heated online debate. This topic comes up frequently, and it’s common to hear that teams completely dismiss notebooks because they’re not for production. Unfortunately, the discussion often centers on problems already fixed by current tools, but the solutions are still unknown for many practitioners. This post shows how to overcome some of Jupyter notebook’s limitations, proposes a workflow to streamline their use in production, and discusses problems that remain to be solved. PUBLICATION PERMISSIONS:
PyData provided Coding Tech with the permission to republish PyData talks CREDITS:
PyData YouTube channel: https://www.youtube.com/c/PyDataTV https://www.youtube.com/watch?v=PpdeASRwjm8

At Apollo Agriculture, a Kenya based agro-tech startup, one of the challenging problems we face is to predict yields of Kenyan maize farmers. Like almost all data-sets, this data-set has a hierarchical structure: farmers within the same region aren’t independent. By ignoring this fact, a model could predict yields entirely from the region of the farmer, but fails to find any other meaningful insights, and we may not even realize. However, if we “overcorrected,” treating each region as completely separate, each individual analysis could be underpowered. Enter the hero of our story: Bayesian hierarchical modeling. Using a practical example in Pymc3, we’ll follow this hero as they identify and overcome clustered data-sets. Slides: https://pydata.org/london2022/wp-content/uploads/2022/07/Clusterf_ck_-A-practical-guide-to-Bayesian-hierarchical-modeling-in-Pymc3.pdf PUBLICATION PERMISSIONS:
PyData provided Coding Tech with the permission to republish PyData talks. CREDITS:
PyData YouTube channel: https://www.youtube.com/c/PyDataTV https://www.youtube.com/watch?v=3ef-Xe4EOaQ

Writing software is hard. Writing good software is even harder. It requires focus, discipline and a great deal of perseverance. But with the right mindset, the process becomes a whole lot more manageable. If you've been writing code for some time, you know that almost all of the technical difficulties you have faced weren't the technical concepts, but the challenge of maintaining the right mindset. https://www.youtube.com/watch?v=K3gcuBWXmtM