But what is cross-entropy?
Compression is Intelligence Part 2
The next part in the "compression is intelligence" trilogy is now up. This one is all about cross-entropy, starting with how it naturally arises while studying compression, then going over a few applications to machine learning. Enjoy!
What a crazy coincidence! I’m actually a 3rd y computer science student, but lately I’ve been studying thermodynamics, I'm working on a project for the SoME competition. My project is a model for how solar radiation hits different parts of earth (land, oceans, poles,...) and gets reflected differently. And at the time your post dropped in I was thinking about the physics of entropy! my aim is to build an interactive simulator for this using streamlit, inspired by 'Manim' in explaining the equations.
It’s so fascinating how this concept bridges thermodynamics in physics and information theory in computer science. Talk about a cosmic scientific coincidence!
Compression is intelligence, yeah — somebody call me dmb