in Link Post

“Deep Learning Has Outlived Its Usefulness as a Buzz-Phrase”

Source: Yann LeCun @ www.facebook.com

Yann LeCun, writing as a Facebook status update:

OK, Deep Learning has outlived its usefulness as a buzz-phrase. Deep Learning est mort. Vive Differentiable Programming!

Yeah, Differentiable Programming is little more than a rebranding of the modern collection Deep Learning techniques, the same way Deep Learning was a rebranding of the modern incarnations of neural nets with more than two layers.

I can see what he’s trying to do here, and lord knows he has a hell of a lot more experience in this field than I do. That said: I disagree. “Deep Learning” is a sub area of “Machine Learning”, which contrasts with “Shallow Learning”. The name makes sense, and is somewhat self explanatory. “Differentiable Programming” makes way less sense to me. The key aspect of the field is the learning, not the programming. Plus: what happens if new optimisation schemes which don’t rely on gradients are found1?

Honestly this feels like a nomenclature land grab on the part of LeCunn and his employer. Possibly also a way to move the ownership of the field away from data scientists and toward software engineers.

Additionally, something he said towards the end of the status caught my eye:

People are now actively working on compilers for imperative differentiable programming languages. This is a very exciting avenue for the development of learning-based AI.

On the one hand: interesting. On the other: oh God, not another programming language…