Our geometric intuition developed in our three-dimensional world often fails us in higher dimensions. Many properties of even simple objects, such as higher dimensional analogs of cubes and spheres, are very counterintuitive. Below we discuss just a few of these properties in an attempt to convey some of the weirdness of high dimensional space.
This makes a nice follow up to my previous posting about performing gradient decent on the surface of the earth. State spaces for machine learning algorithms tend to have a lot more dimensions than 2. There’s a reasonable chance they have millions, in fact. So while the “surface of the earth” explanation is a really good way of introducing the concept of gradient decent, it might not actually help you when using the algorithm in practice.
Also: higher dimensional spaces are just fun to think about.