How to pronounce “sparsity” in English

And by incorporating sparsity into deep learning systems in the way that it seems to be implemented in the neocortex, we've actually been able to recently show

Another property of sparsity is that sparse vectors-- so these are very high-dimensional mathematical vectors that are mostly zero--

by incorporating context and sparsity and updating the learning rules.

I discussed sparsity and how we can use that to get dramatic efficiency gains on hardware and improve robustness.

And so things like sparsity are often brought up.

And there's another type of sparsity, which is called connectivity sparsity.

It turns out when you use sparsity, sparse activations, the brain can represent multiple hypotheses at the same time.

So I'll talk first about sparsity.

Or is there an important benefit to sparsity?

You can think about optimizing for L1 sparsity.

experts with sparsity.

It turns out because the brain works on sparsity that all 10 hypotheses can be processed at the same time.

And they'll need to evolve towards supporting sparsity more inherently.

So kind of summarizing the first aspect of our roadmap-- we think sparsity is going to be critical for scaling AI systems.

And I think implicit in your question is that a lot of people like that sparsity, but not everybody does.

So, one of the words that we like to use is sparsity factor or usage.

But what DeepSeek did that maybe only the leading labs have only just started recently doing is have such a high sparsity factor.

And what Nathan was getting at there was, when you have such a different level of sparsity, you can't just have every GPU have the entire model.

So we think robustness-- when we think about building systems that are not brittle, we think sparsity should be one of the core components of that.

And the last thing I want to talk about, one more essential property-- I'm going even deeper now into neuroscience-- is something called sparsity.

And we're starting to understand at a very detailed circuit level what's required to sort of fulfill kind of efficiency promise with sparsity.