So sparse versus rich first-- so there is a debate between the view that conscious perception is sparse and that conscious perception is rich.I myself think that the resolution of the debate is that it's cognition that's actually sparse -- that's thought, reasoning, problem solving,
But Kimi K2 Thinking and Kimi K2 is a model that is very popular. People say that has very good creative writing and also in doing some software The sparse nature of MoE models makes it much more efficient to do generation, which becomes a big part
It was regarded as a tenure killer among scientists-- that is, if you study it, forget tenure. So sparse versus rich first-- so there is a debate between the view that conscious perception is sparse and that conscious perception is rich.
But in the brain, you don't see that. You see a very sparse connectivity. Now, I mention all this because these are actually the properties we think are absolutely essential for creating intelligent machines
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them This is extremely sparse -- much, much sparser than what we have in typical deep learning systems.
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them When things are sparse , when they're silent, they're not consuming power.
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them When you have sparse representations, they don't collide much with one another.
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them OpenAI had a sparse toolkit.
And I knew I wanted to talk very intimately in great detail about what I was feeling and thinking at the time, and in relatively sparse detail about like who was in my life at the time or what were we doing. So I think knowing like I'm not going to go into a lot of detail about particular relationships, I'm not going to go into detail about like dates
And here are different aspects of Astrobee. One is a general sparse mapping, where it's looking at different features it sees across ISS, mapping that against a known map,
show you a picture because the visual cortex is actually very critical for identifying friend or foe are you in very very sparse data sets and human learning algorithms that are not unlike
So it is a ubiquitous feature of information processing in animals. So that's sparse . Rich is what's in phenomenal consciousness. So that holds many more things.
OK, so that's the kind of thing that suggests rich. What about what suggests sparse ? So here's a-- undoubtedly everybody has seen one of these things.
One account is this inattentional blindness account, which is, you don't consciously see the features that change. That's the sparse view. The other view is the inattentional inaccessibility or sorry, lack of access, not inaccessibility-- lack
There's an attentional bottleneck. And then you have sparse cognition. My opponent's view, because they have to explain Partial Report Superiority, just as I do, their view
is that the bottleneck is between unconscious perception and conscious perception. Conscious perception is sparse , and cognition is sparse . So I'm now going to show you a few pieces of anecdotal evidence for my view.
from hospitals, medical doctors and so on, with a combination of old and new mathematics-- Fourier transform analysis, data processing, so- called sparse methods, and so on, to reduce drastically the time that you need to get a scan. This machine, getting to this.
All those galaxy maps we flew around then are just this tiny part near the center that are well covered. We have some sparse outliers in here. And then there is this cosmic microwave background radiation that we just talked about, which is just a photograph of the outside surface.
a lot about the composition and the structure of our galaxy. And there's a sparse halo of stars. So you're talking about maybe 1% or so of the total number of stars in our galaxy.
and letters gave his cookbook a rave review he says we would have to be grateful for almost any new contribution to the two sparse literature of African food but restaurant CH Pierre Tam's book on the richly appealing food of his native Sagal is beyond the call Fresh Lively and intelligent so without
And so one night, they packed into a classroom-- students, staff, family members, and Sheron in her skirt-- for the monthly school board meeting, which was usually sparsely attended and not at all on students' radars. At the meeting, Sheron presented the petition and made her case.
OK, again, what's this got to do with anything? So its vacuum is sparser than the vacuum outside the International Space Station.
It's about the size of Switzerland and has about maybe a quarter of its population. So it's quite sparsely populated. And as I said earlier, it's the only place where men live as long as women, which is actually extraordinary.
Catherine Hall: I'm driving around looking for people and they are not there. It is so sparsely populated. Oh, by the way, did I mention all the other iconic photographers were like landscape photographers, adventure photographers, like all the things that are perfect for Tasmania?
The brain is constantly processing multiple simultaneous hypotheses at the same time, and nobody gets confused. And it's only because they use sparse representations. So this is like a fundamental information processing idea, like binary digits in computers.
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them Another property of sparsity is that sparse vectors-- so these are very high-dimensional mathematical vectors that are mostly zero--
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them We have a similar kind of sparse toolkit called RigL.
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them LeCun's been doing these dense-to-sparse pruning.
We definitely rely heavily on computer vision here, but we might combine it with inertial sensing, for instance. We build kind of a sparse map of just a couple of landmarks in the 3D scene, and at the same time, try to find out where the robot here represented with this kind of coordinate frame-- where it is located in a kind of global frame.
know that that have a kind of sparse way of living but nonetheless in some of my
So we have sort of sparse sampling of what's out there.
It's probably not too sparse a space.
But somewhere in between the global but sparse data and the very local and high-res but dense data, we need something else to kind of bridge the gap for landing site simulation
high-resolution terrain and the very sparse laser altimetry.
And that also has kind of a sparse mosaic that we put into Mars mode for Google Earth.
They found that concepts are represented sparsely by different sets of neurons in the brain.
And a network which is sparsely knit there are a lot of people in my network and in your
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them So Jeff discussed that the brain is really sparse .
So it works really well when food is really sparse , eating like 15 apples.
OK, so first you have an array of alphanumer-- yeah. Could you just quickly talk about what's the difference between sparse and rich in your terminology is? Because there's the machine learning version of it, which a lot of people here are probably familiar with, but it's probably different.
Great data, but sparse .
But again, sparse mosaics.
low-density operations, sparsely populated areas.
The brain doesn't work like that at all. It turns out when you use sparsity, sparse activations, the brain can represent multiple hypotheses at the same time. So let's say I'm using 200 neurons active to represent something out of 10,000.
It's like what nature did is just discover these neural processes that allow us to build models of the world and then just repackage them And because of that, it actually looks like sparse systems can be far more robust to noise and very robust to perturbations compared to typical DNN systems.
And at the time, Milwaukee Avenue was sparse .
Because in China, that's very sparse .
A locality will be a relatively high-density population area surrounded by a more sparse area around it in a typical town, city arrangement.
different? The talk's going to have three parts. I'll talk about a controversy, about sparse versus rich perception, about concepts. And then I'll end with these methodological breakthroughs that I think have something to say about how to approach the issue experimentally.
like functions. Are the features sparse ?