• 0 Posts
  • 77 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle





  • I can run 7B models on my laptop with its embedded GPU. Running on a phone or a Pi is possible with smaller models, but very slow. Expect good speed with a desktop Nvidea GPU. Later this year, there should be new computers with an NPU integrated to the CPU which should speed up computers that don’t have a dedicated GPU. (But a GPU will still outperform them by a lot.)

    70B models will run very slowly on even the best consumer hardware due to memory limitations.