Just for reference, a few years back, (ex-Microsoft) David Plummer had this historical dive into the (MIPS) origin of the blue color, and how Windows is not blue anymore: https://youtu.be/KgqJJECQQH0?t=780

How does this analogy work at all? LoRA is chosen by the modifier to be low ranked to accommodate some desktop/workstation memory constraint, not because the other weights are “very hard” to modify if you happens to have the necessary compute and I/O. The development in LoRA is also largely directed by storage reduction (hence not too many layers modified) and preservation of the generalizability (since training generalizable models is hard). The Kronecker product versions, in particular, has been first developed in the context of federated learning, and not for desktop/workstation fine-tuning (also LoRA is fully capable of modifying all weights, it is rather a technique to do it in a correlated fashion to reduce the size of the gradient update). And much development of LoRA happened in the context of otherwise fully open datasets (e.g. LAION), that are just not manageable in desktop/workstation settings.

This narrow perspective of “source” is taking away the actual usefulness of compute/training here. Datasets from e.g. LAION to Common Crawl have been available for some time, along with training code (sometimes independently reproduced) for the Imagen diffusion model or GPT. It is only when e.g. GPT-J came along that somebody invested into the compute (including how to scale it to their specific cluster) that the result became useful.

This is a very shallow analogy. Fine-tuning is rather the standard technical approach to reduce compute, even if you have access to the code and all training data. Hence there has always been a rich and established ecosystem for fine-tuning, regardless of “source.” Patching closed-source binaries is not the standard approach, since compilation is far less computational intensive than today’s large scale training.

Java byte codes are a far fetched example. JVM does assume a specific architecture that is particular to the CPU-dominant world when it was developed, and Java byte codes cannot be trivially executed (efficiently) on a GPU or FPGA, for instance.

And by the way, the issue of weight portability is far more relevant than the forced comparison to (simple) code can accomplish. Usually today’s large scale training code is very unique to a particular cluster (or TPU, WSE), as opposed to the resulting weight. Even if you got hold of somebody’s training code, you often have to reinvent the wheel to scale it to your own particular compute hardware, interconnect, I/O pipeline, etc… This is not commodity open source on your home PC or workstation.

The situation is somewhat different and nuanced. With weights there are tools for fine-tuning, LoRA/LoHa, PEFT, etc., which presents a different situation as with binaries for programs. You can see that despite e.g. LLaMA being “compiled”, others can significantly use it to make models that surpass the previous iteration (see e.g. recently WizardLM 2 in relation to LLaMA 2). Weights are also to a much larger degree architecturally independent than binaries (you can usually cross train/inference on GPU, Google TPU, Cerebras WSE, etc. with the same weights).

GIMP is a special case. GIMP is being getting outdeveloped by Krita these days. E.g.:

https://gitlab.gnome.org/GNOME/gimp/-/issues/9284

Or compare with:

https://www.phoronix.com/news/Krita-2024-GPUs-AI

GIMP had its share of self inflicted wounds starting with a toxic mailing list that drove away people from professional VFX and surrounding FilmGimp/CinePaint. When the GIMP people subsequently took over the GEGL development from Rhythm & Hues, it took literally 15 years until it barely worked.

Now we are past the era of simple GPU processing into diffusion models/“generative AI” and GIMP is barely keeping up with simple GPU processing (like resizing, see above).

AMD’s support for AI is just fine

This is quite untrue, especially if you do actual research and not just run other people’s models. For example, ROCm is missing in many sparse autograd frameworks, e.g. pytorch_sparse, or having a viable alternative to Nvidias MinkowskiEngine. This is needed if you do any state-of-the-art convnets with attention-like sparsity.

Yes. But one should also note that only a limited range of Intel GPU support SR-IOV.

FSFE’s statement:

• https://fsfe.org/news/2023/news-20231011-01.en.html

Some related personal blogs I noticed:

• https://ebb.org/bkuhn/blog/2023/10/11/moglen-sflc.html
• https://mjg59.dreamwidth.org/68004.html

Actually that I doubt it (in the sense you can enter the NCART and move around, like maybe GTA 4).

And a decade ago, Google itself sabotaged XMPP in their version of embrace, extend, and extinguish: https://www.eff.org/deeplinks/2013/05/google-abandons-open-standards-instant-messaging

You know, there are several built-in functions in phones, that are already viable methods to communicate remotely?

There were no discussion about performance and bugs in the video. It is perhaps worthwhile to point out that the PC requirement was raised for Phantom Liberty back in June: https://www.cyberpunk.net/en/news/48271/update-to-pc-system-requirements

A brief summary:

1. The first minutes of the Phantom Liberty mission, interspersed the demonstration of the relic and other revamped skill trees.
2. The revamped police system with someone reaching 5 “wanted” stars, maintaining it/fighting MaxTac for many minutes, then losing it (by hiding cleverly).
3. The new activity loop with stealing vehicles for Muamar “El Capitan” Reyes.
4. Viewer Q&A.

The novel bit of this project is actually the usage of GGML quantization from llama.cpp for Stable Diffusion, which can offer lower RAM usage and faster inference on CPU than all the previous CPU implementations without the benefit of low bit quantization, which was known to make CPU and low RAM LLaMA inference feasible.

The important long term implication is that people have been targeting the incorrectly sized Stable Diffusion model, if the goal is quality on commodity hardware (this includes GPU, too). For example, Stable Diffusion where Stability AI has gloated so much how it fits commodity hardware is slightly less than 1 billion parameters. The smallest LLaMA that people nowadays can happily run on commodity GPU or CPU is already 7 billion parameters. And even OpenAI’s DALL·E 2, which many called prohibitive because “you need a 48 GB GPU” (which is not true, with quantization), is just 3.5 billion parameters.

For additional context, Stable Diffusion using CPU has been done before, though with repurposed frameworks rather than a custom C++ project. Notably, there has been a Q-Diffusion paper (https://github.com/Xiuyu-Li/q-diffusion), but the result was obtained by simulating the quantization, and e.g. the GitHub repo not actually offer an implementation with actual speed-up.

Germany traditionally is quite shocking in their practice of segregating children with disabilities into special Förderschulen. Whereas the U.S. has the Individual’s with Disabilities Education Act since the 1970s, Germany was basically forced into integration recently after the country signed the U.N. Convention on the Rights of Persons with Disabilities in 2009. And even then, they are taking their sweet time to integrate. See e.g. https://www.aktion-mensch.de/inklusion/bildung/hintergrund/zahlen-daten-und-fakten/inklusionsquoten-in-deutschland as how currently, slightly less than half of German students with disabilities go to a regular school (the Inklusionsanteil).

See: https://en.wikipedia.org/wiki/English-language_spelling_reform

English has been the total outlier among (originally) European language with no body of authority over its spelling. Even the “reform” by Noah Webster never really caught on outside North America, nearly 100 years later. And even more curious, the somewhat authoritative Oxford English Dictionary disagrees in their spelling with everybody (https://en.wikipedia.org/wiki/Oxford_spelling).

Nearly every single word in English that starts with a g followed by a soft ih/eh vowel is pronounced as a soft g, just a few:

That is patently not true and blatant cherry picking, e.g. already contradicted by the lexically matching word “gift” (and there are “giggle”, “gild”, “girl”, “git”, “give”, “gizmo”, etc.). See Wikipedia, which referenced linguists studying this:

An analysis of 269 words by linguist Michael Dow found near-tied results on whether a hard or soft g was more appropriate based on other English words; the results varied somewhat depending on what parameters were used.[11] Of the 105 words that contained gi somewhere in the word, 68 used the soft g while only 37 employed its counterpart. However, the hard g words were found to be significantly more common in everyday English; […]

https://en.wikipedia.org/wiki/Pronunciation_of_GIF#Cause

Michael Dow is an associate professor in linguistics with specialization in phonology, by the way.

and if you’re confused why others pronounce it with a soft G, they would seem to be simply more familiar with the English language 🤷‍♂️

Well, clearly you are already not as “familiar with the English language” as you might think.

Just see https://time.com/5791028/how-to-pronounce-gif/ and https://en.wikipedia.org/wiki/Pronunciation_of_GIF#Analysis_of_the_dispute