YES! I study AI, and this is exactly how I feel!

Side note-One of my favorite things to do is ask people what their use case for using AI is, and watch them sputter out “uh…emails and productivity and things.”

I got a laptop back in 2018, and it shipped really fast. It’s not my daily driver, but it works well when I’m on the road, and the battery life is pretty good. Granted, I replaced the OS with a distro I prefer and customized the hell out of it, so that might contribute to my experience. Tbh, I was pretty impressed with it (still am), and I was going to buy a Librem 5 when they came out. I wanted to wait and not just throw money at them because I didn’t want to get burned. After all the horror stories and crap reviews, I passed on that and won’t touch the company with a 10 foot pole, and I thank past me for not throwing money at them.

I think that the company started with noble intentions and made a decent product at first, but they got in way over their heads and now they’re floundering.

However, the name “Arthur Grand Technologies” will always be synonymous to racism.

For now. This will be memory-holed in a couple weeks. A couple grand is nothing, maybe just a slightly down quarter. They should have been completely annihilated. Crimes like this shouldn’t be fixed dollar amounts; it should be percentages of average annual gross earnings, and in this case, something like 300% annual gross earnings. Let them sell everything off.

The original paper itself, for those who are interested.

Overall, this is really interesting research and a really good “first step.” I will be interested to see if this can be replicated on other models. One thing that really stood out, though, was that certain details are obfuscated because of Sonnet being proprietary. Hopefully follow-on work is done on one of the open source models to confirm the method.

One of the notable limitations is quantifying activation’s correlation to text meaning, which will make any sort of controls difficult. Sure, you can just massively increase or decrease a weight, and for some things that will be fine, but for real manual fine tuning, that will prove to be a difficulty.

I suspect this method is likely generalizable (maybe with some tweaks?), and I’d really be interested to see how this type of analysis could be done on other neural networks.

I work/study in AI, and it is completely over-hyped. For one thing, the C-suite can’t wrap it’s head around the fact that AI != LLM; they all seem to think all AI is just LLMs. On top of that, they are way too eager to throw humans out of the loop.

That said, I think LLM applications, even in their current form, are super useful in development and business practices. I myself use it to increase my productivity in coding. But, I use it as an augmentation rather than a replacement. One of my friends put it best the other day, “LLMs are like a junior dev to your senior dev. You need to be hyper-specific, and you need to check it’s output.” In other words, it’s great for off-loading some work, but it isn’t going to completely replace humans.

With that said, I’m a bit annoyed that other AI fields are being over-shadowed by LLMs. There’s a ton of other interesting work being done in those fields that is super useful and important. All of them, though, are not going to replace humans but rather augment and make humans more productive. I’ve found that an AI-Human team is most effective.

I think what they’re saying is that Americans don’t pay attention and forgot how terrible the Trump presidency was because it’s been a few years. Most people think that “we’re better now” and any major issues have abated without understanding that nothing has fundamentally changed. Because of all that, Trump will win the election. The DnD portion of the post is just what got OP to think about this.

Sad thing is that there’s merit to the argument. It’s the old trope of “Americans have short memories.”

Except that scaling alone won’t lead to AGI. It may generate better, more convincing text, but the core algorithm is the same. That “special juice” is almost certainly going to come from algorithmic development rather than just throwing more compute at the problem.

I mean, that’s more-or-less what I said. We don’t know the theoretical limits of how good that text generation is when throwing more compute at it and adding parameters for the context window. Can it generate a whole book that is fairly convincing, write legal briefs off of the sum of human legal knowledge, etc.? Ultimately, the algorithm is the same, so like you said, the same problems persist, and the definition of “better” is wishy-washy.

Cool, Bill Gates has opinions. I think he’s being hasty and speaking out of turn and only partially correct. From my understanding, the “big innovation” of GPT-4 was adding more parameters and scaling up compute. The core algorithms are generally agreed to be mostly the same from earlier versions (not that we know for sure since OpenAI has only released a technical report). Based on that, the real limit on this technology is compute and number of parameters (as boring as that is), and so he’s right that the algorithm design may have plateaued. However, we really don’t know what will happen if truly monster rigs with tens-of-trillions of parameters are used when trained on the entirety of human written knowledge (morality of that notwithstanding), and that’s where he’s wrong.

Congratulations on making the switch! I remember when I switched full time almost 10 years ago. It always feels like there’s something new to explore or to try with your computer. One of the most freeing things I learned was that most things are within my grasp if I put in the effort to learn about it. There’s nothing quite as fun as whittling the day away going down a configuration rabbit-hole to make something just right.

It depends on the terms of employment. If they are salaried, then there are no real work hours and just work to do. In general, if someone is salaried, they’re paid to do a job not when they do it.

This is a much better article. OP’s article just shows the author’s surface understanding of how coding works and how well an LLM can actually code. There’s way more that goes into a programming task than just coding.

I see LLMs as having the potential of being almost like a super library. I can prompt GPT, Claude, etc. to write me a custom function that I copy, paste, test, scrutinize, and almost certainly change. It’s a tool that will make someone a more productive programmer. It won’t completely subsume a human’s ability to be creative and put the pieces together.

At the absolute worst over the next decade, I could see programming changing from writing and debugging code to prompting, stitching together, and debugging.

It works well if you know how to prompt well. LLMs are able to do so much, but a user just needs to know how to use it correctly. The technology is still in its infancy, so it’s a bit difficult to use well.

It’s funny you say that. I find the Linux way of getting software way more intuitive. Just hop in the terminal and use the package manager. When I used Windows, I always felt like I was doing something shady when I was getting a .exe. With drivers, I’ve only had an issue once; everything else was pre-compiled into the kernel. On Windows, I had driver issues a lot. For those reasons (and others), I switched full time to Linux almost a decade ago.

Totally anecdotal, of course, but I just thought it was funny how our experiences were complete opposites and sent us in complete opposite directions for the same reason.

This used to be me while I distrohopped. Now, I’ve settled on which distro I will use. I use Arch, btw.

Exactly. I didn’t learn multiplication tables, but instead, my teachers taught what multiplication was and how to do it. Sure, after practicing how to do it I memorized a lot of basic math facts, but it was more important that I understand what I was doing rather than just knowing that 9×5=45. Then again, my parents saw the sorry state of public schools (both teachers) and made sure that I was sent to a decent private school after seeing the travesty of the local public school’s reading curriculum.