Never heard of algorithmic differentiation? It’s the tool that “underlies modern machine learning”. I learned that from reading the intro to this handbook, btw.

It begins with a calculus-101 style understanding and gradually extends this to build toy implementations of systems similar to PyTorch and TensorFlow.