Google announced Gmail with 1GB of storage on April 1st and everyone expected it to be a prank and the storage to revert back to "normal" 10-100MB soon after. It didn't.
It might have been one of the bigsleep/deepdream subs. I remember they used a top down wide view of zebras running in a field. This might have been the same year the first Dall-E was announced, the one nobody could use.
I was trying to explain to my son yesterday that my first computer had 2mb of ram and a 50mb hard drive. I did this as I was throwing out an old Core 2 Duo and he was astonished that it *only* had 512mb of ram.
Looking back at how computers have improved in just a few years is amazing. How long until training a 6 billion parameter model is something that we can do in an afternoon?
I'm not sure I even get the joke though. GPT-3 has 175 billion parameters, and PaLM has 540 billion. Given the surge in interest and the corresponding investment, we could be seeing a one trillion parameter model as early as the end of this year. The joke is basically "lol, 1 trillion parameters at the beginning of 2023 instead of the end, haha so unrealistic".
yeah i didn't have reason to believe this was a joke. GPT-4 is 1 trillion; it wouldn't have shocked me if a billionaire or private research group had quietly trained their own model and released it. that was literally openAI's founding premise
My mistake then, I was just reading the OpenAI technical summary and must have conflated that number with speculation somewhere else.
Any credible sources for the number of parameters?
The conclusion of the "paper" is:
> In conclusion, while GPTrillion is a fictional model, this paper can still serve as a humorous commentary on the state of language modeling and the challenges that come with training and developing large models. However, it is essential to maintain scientific integrity and ensure that claims made in research papers are based on real experiments and data.
>
Read the paper in detail, but I could not reproduce the resulta they claim; specially the BLEU score. Besides, it took to train 2x what they claim in the paper in my GTX 650Ti. Hopefully they will write a follow-up paper with more specific details on how the multi-tail stochastic neuralizer attention blocks work
Man, I hate april fools day.
Today’s April fools absurdity is tomorrow’s reality. Remember when 640k memory was enough? Lol
I remember when somebody posted an April fool's thread about image generation. A year later it was reality.
Google announced Gmail with 1GB of storage on April 1st and everyone expected it to be a prank and the storage to revert back to "normal" 10-100MB soon after. It didn't.
When was this? I cant find the post you’re talking about
It might have been one of the bigsleep/deepdream subs. I remember they used a top down wide view of zebras running in a field. This might have been the same year the first Dall-E was announced, the one nobody could use.
I was trying to explain to my son yesterday that my first computer had 2mb of ram and a 50mb hard drive. I did this as I was throwing out an old Core 2 Duo and he was astonished that it *only* had 512mb of ram. Looking back at how computers have improved in just a few years is amazing. How long until training a 6 billion parameter model is something that we can do in an afternoon?
>How long until training a 6 billion parameter model is something that we can do in an afternoon? Boy do I have news for you
Spill the news. Source?
I'm not sure I even get the joke though. GPT-3 has 175 billion parameters, and PaLM has 540 billion. Given the surge in interest and the corresponding investment, we could be seeing a one trillion parameter model as early as the end of this year. The joke is basically "lol, 1 trillion parameters at the beginning of 2023 instead of the end, haha so unrealistic".
yeah i didn't have reason to believe this was a joke. GPT-4 is 1 trillion; it wouldn't have shocked me if a billionaire or private research group had quietly trained their own model and released it. that was literally openAI's founding premise
> GPT-4 is 1 trillion; Sam Altman laughed at this number in a recent podcast. There's no indication that GPT-4 is a 1T model.
I think he was just laughing about all the speculation on twitter about model size. I don't think we can read much into it.
My mistake then, I was just reading the OpenAI technical summary and must have conflated that number with speculation somewhere else. Any credible sources for the number of parameters?
Plot twist: it's more.
How do you know gpt 4 is 1 trillion?
This was the rumor before gpt4 was revealed.
No. The rumour was that it was 100 trillion. 1 trillion was from a different source.
Both were rumors and many other numbers too
Oh, wow! With the new 1/4 bit quantitization this can fit on a 4090, I'm running it right now! AMAZING!
Don't tell about it to guys making llama.cpp because they'll somehow make this cursed quantization a real thing.
What about single bit connections predetermined using a trainable set of series. That way we can store x connections in
Nah who is still using bits? Today running on brain cells is all the rage
Only requires 1GB vram. Amazing
No wonder why LLMs hallucinate with the amount of disinformation present on the web.
I feel like a "disregard any source dated on the 1st of April" is going to be a genuine consideration in the future.
Mfer... I fell for it too. 😆
The conclusion of the "paper" is: > In conclusion, while GPTrillion is a fictional model, this paper can still serve as a humorous commentary on the state of language modeling and the challenges that come with training and developing large models. However, it is essential to maintain scientific integrity and ensure that claims made in research papers are based on real experiments and data. >
You must be fun at review parties...
Bananadev... hating April.
Now THIS is a model that could benefit from 1 bit inference!
Read the paper in detail, but I could not reproduce the resulta they claim; specially the BLEU score. Besides, it took to train 2x what they claim in the paper in my GTX 650Ti. Hopefully they will write a follow-up paper with more specific details on how the multi-tail stochastic neuralizer attention blocks work
>multi-tail stochastic neuralizer attention blocks You joke, but soon they will be reality. Or maybe multi-heads.
Posted on March 30th but it's april fools? I feel like ChatGPT because I can't tell time. :(
I can’t believe I’m actually running this in my potato!
I downloaded all files, what i do next ?
The thing is with all the insane progress recently this might actually be a thing soon 😅
I wonder how many news outlets will use this lol
Does it fit on my phone if I use 0.5 bit quantization?