Just jumping off your last sentence: "Simplicity" and "Transparency" are orthogonal.

In the case of deep learning, all current state-of-the-art models can be expressed as nothing more than matrix multiplication, addition, and clipping. That's all you need. It's so simple: multiply, add, clip. Multiply, add, clip.

But the model isn't transparent. It's clear what each step is doing (multiply! add! clip!), but it's not clear to the human why that's necessary.