MoE large language models, inference.
MoE is the shortened version of mixture of experts, is multiple papers old.
giant MoE model, very similar to DeepSeek architecture in December. And then a startup, RCAI, and both Nemotron and NVIDIA have teased MoE
that Moe did about-- I'm actually not going to say what it was because it was really gross.
The sparse nature of MoE models makes it much more efficient to do generation, which becomes a big part
And in the MoE literature, there's something called the auxiliary loss, which helps balance them.
Just give me mellow Moe Dee on the beat with the speech.
OpenAI has also had models that are MoE , and so have all the other labs that are major closed.
One of my friends called Moe Giddings was chased down the seafront with a hammer.
Yeah So going back to "Moesha" a little bit--
Eeny meeny miny moe .
And if you want to use RoPE with a normal MoE , it's a sequential thing.
It's 32 versus 4 for mixed draw and other MoE models that have been publicly released.
And I'm sure all the labs that are training big MoEs are looking at this sort of things, which is getting away from the auxiliary loss.
I have invested in a bar in Brussels called Moeder Lambic.
There have been-- you look at like Doug Moes , the Denver Nuggets team of the early '80s, where they would score 120 points a game or something like that.
So we're going to be hearing Jean Moeremans, who
So when you say you don't remember the "Moesha" years, you were being honest?
Now, which is 120 billion parameter open weight MoE model.
And in addition to that your credits include "Shuffle Along," "Miss Saigon," "Five Guys Named Moe ," "Grease," and the list goes on and on.
This is like calling Moses, Moe ."
So I hired one woman, Moe , in maybe March or April of 2007.
And it's weird because I don't remember a lot of the "Moesha" days, so I'm pretty sure there was a lot of great people on
There's a YouTube video of a conversation that I had at Moeder Lambic which is a place, a home if you will, the tap house that has the
time additionally Bill does such an excellent job of capturing personalities and characters of Tim moer and Todd
So, we don't know the exact extent that the DeepSeek MoE change, which is instead of doing an auxiliary loss, they have an extra parameter in their routing,
And then if you're incompetent and cold, you're Moe .
So what I'll do, I'll slice a few more pieces and then Moe is going to give you some sample plates.
And that was actually one of the most fun times on the site was sitting around mostly with Moe , whose real name is Maureen,
much bigger, and that gives them higher peak performance as MoEs , where a lot of these things that we like a lot, whether it was Gemma and
Even things as sort of well-established as MoEs .
I think about my high school track coach, Dennis Moen, who encouraged me to just-- people who just encourage you to be your best self and don't--
But it was a great experience for me because I got a chance to grow up through the eyes of "Moesha", because I wasn't in
And I make concentric circles on a big plate, and then I'd do eeny, meeny, miny, moe .
But it wouldn't be like eeny, meeny, miny, moe I will eat you, it was eeny, meeny, miny, moe , I will not eat you.
And also, it just happened, like the first person I hired, Moe Tkacik, is very pointed, and biting, and very,
And that we are reluctant to simply roll the die and say, "Eenie, meenie, miney, moe ."
on se serait barées dès le début, parce que ce qu'on voulait, c'était une révolution des sociétés, des moeurs, ne pas vivre et ne pas s'ancrer dans une forme d'hétéronormativité
In the '90s in sitcoms, yeah, like "Sister, Sister," "Moesha," shows-- "Fresh Prince." "Fresh Prince." Those shows, we got to see ourselves a little bit,
And champagne bottles sticks out from there The champagne just opens up on its own In a month, there was a time that we sold 800 bottles of Moet
Definitely. OK, so next we want to move to-- this one's a favorite of mine-- your TV show, "Moesha".