• 5 Posts
  • 90 Comments
Joined 11 months ago
cake
Cake day: August 8th, 2023

help-circle


  • We’re close to peak using current NN architectures and methods. All this started with the discovery of transformer architecture in 2017. Advances in architecture and methods have been fairly small and incremental since then. The advancements in performance has mostly just been throwing more data and compute at the models, and diminishing returns have been observed. GPT-3 costed something like $15 million to train. GPT-4 is a little better and costed something like $100 million to train. If the next model costs $1 billion to train, it will likely be a little better.






  • Nah, the politicians and the populace would gain nothing by seceding. The Republicans are trying to push extreme “state’s rights” on all fronts. The goal of this particular fiasco is likely to get a favorable outcome from SCOTUS and to prevent Texas from getting more blue/brown from immigrants’ children. The goal of extreme “state’s rights” is to ensure Republican control over the federal government (there are more red states than blue states, and if states have enough rights to do things like overriding election results, voter suppression, extreme gerrymandering, and other various ways of “rigging” elections, Republicans can ensure control over both state and federal government).




  • Quality. Yeah, using the extra compute to increase speed of development iterations would be a benefit. They could train a bunch of models in parallel and either pick the best model to use or use them all as an ensemble or something.

    My guess is that the main reason for all the GPUs is they’re going to offer hosting and training infrastructure for everyone. That would align with the strategy of releasing models as “open” then trying to entice people into their cloud ecosystem. Or, maybe they really are trying to achieve AGI as they state in the article. I don’t really know of any ML architectures that would allow for AGI though (besides the theoretical, incomputable AIXI).







  • Goes both ways. As someone who has tried bringing new products to market, it’s extremely annoying that nearly everything you can think of already has similar patent. I’ve also reverse engineered a few things (circuits and disassembled code), as a little guy, working for a small business . I don’t think people usually scan patents to learn things, and reverse engineering usually isn’t too hard.

    If I were a capitalist, I’d argue that if a big business “steals” an idea, and implements it more effectively and efficiently than the small business, then the small business should probably fail.




  • He will likely be allowed to run because so much of the country supports him, there is some legal gray area (he has not been convicted), and the courts are “conservative.” I personally think he will win because Biden is getting even worse at speeches, much of the population doesn’t think their personal lives improved under Biden, and a lot of people are upset for how Biden has/is handling the Israel war.

    A lot of things could happen before the election that would hurt Biden as well. A recession, expansion of Israel war, and losses in Ukraine are possibilities that could hurt Biden. I don’t think anything could hurt Trump. I think he could win the election from prison. Trump voters will eagerly buy any conspiracy theory to keep supporting Trump, and they don’t care about democracy or human rights. Democrat and Biden voters are much more critical and fickle.