Once you’ve trained your large language model on the entire written output of humanity, where do you go? Here’s Ilya Sutskever, ex-OpenAI, admitting to Reuters that they’ve plateaued: [Reuters] The…

I’m usually the one saying “AI is already as good as it’s gonna get, for a long while.”

This article, in contrast, is quotes from folks making the next AI generation - saying the same.

  • brucethemoose@lemmy.worldEnglish
    1·
    1 month ago

    I don’t think Qwen was trained with distillation, was it?

    It would be awesome if it was.

    Also you should try Supernova Medius, which is Qwen 14B with some “distillation” from some other models.