We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision.
“cheat”, “lie”, “cover up”… Assigning human behavior to Stochastic Parrots again, aren’t we Jimmy?
Those words concisely describe what it’s doing. What words would you use instead?
It has no fundamental grasp of concepts like truth, it just repeats words that simulate human responses. It’s glorified autocomplete that yields impressive results. Do you consider your auto complete to be lying when it picks the wrong word?
If making it pretend to be a stock picker and putting it under pressure makes it return lies, that’s because it was trained on data that indicates that’s statistically likely to be the right set of words as response for such a query.
Also, because large language models are probabilistic, you could ask it the same question over and over again and get totally different responses each time, some of which are inaccurate. Are they lies though? For a creature to lie it has to know that it’s returning untruths.
Interestingly, humans “auto complete” all the time and make up stories to rationalize their own behavior even when they literally have no idea why they acted the way they did, like in experiments with split brain patients.
The perceived quality of human intelligence is held up by so many assumptions, like “having free will” and “understanding truth”. Do we really? Can anyone prove that? (Edit, this works the other way too. Assuming that we do understand truth and have free will - if those terms can even be defined in a testable way - can you prove that the llm doesn’t?)
At this point I’m convinced that the difference between a llm and human-level intelligence is dimensions of awareness, scale, and further development of the model’s architecture. Fundamentally though, I think we have all the pieces
Edit: I just want to emphasize, I think. I hypothesize. I don’t pretend to know
I think.
But do you think? Do I think? Do LLMs think? What is thinking, anyway?
I mean, I think so?
Steady on there Descartes.
You didn’t answer my question, though. What words would you use to concisely describe these actions by the LLM?
People anthropomorphize machines all the time, it’s a convenient way to describe their behaviour in familiar terms. I don’t see the problem here.
Those words imply agency. It would be more accurate to say it returned responses that included cheating, lies, and cover-ups, rather than using language to suggest the LLM performed such actions. The agents that cheated, lied, and covered up were presumably the humans whose responses were used in the training data. I think it’s important to use accurate language here given how many people are already inappropriately anthropomorphizing these LLMs, causing many to see AGI where there is none.
If I take my car into the garage for repairs because the “loss of traction” warning light is on despite having perfectly good traction, and I were to tell the mechanic “the traction sensor is lying,” do you think he’d understand what I said perfectly well or do you think he’d launch into a philosophical debate over whether the sensor has agency?
This is a perfectly fine word to use to describe this kind of behaviour in everyday parlance.
The point of the distinction in that situation is that no one thinks your car is actually alive and capable of lying to you. The language distinction when describing an obviously inanimate object isn’t important because there is no chance for confusion.
Is your conversation with a mechanic meant to be the summary and description of a rigorous scientific discovery?
This isn’t ‘everyday parlance’ this is the result of a study.
If someone doesn’t know the answer to something and they guess, or think they know the answer but don’t, they are wrong. If they do know the answer and intentionally give a wrong answer, they are lying.
If someone is in a competition or playing a game and they break a rule they didn’t know about, they made a mistake. If they do know the rules and break it, they are cheating.
Lying and cheating fundamentally requires intent. This is important no matter what you’re referring to. If a child gets something wrong, you should not get mad at them for lying. If they make a mistake in a game, you should not acuse them out cheating. There is a difference and it matters.
ChatGPT literally cannot think. It’s not sitting around contemplating it’s existence while waiting for inputs. It’s taking what you say, comparing that to everything that it’s been trained on, assigning a bunch of statistics, and outputting something based on more statistics that hopefully is correct and makes sense.
It doesn’t know if it makes sense. It doesn’t “know” anything. It’s just an incredibly sophisticated version of “if user inputs ‘Hi how are you’, respond ‘I am well, how are you?’”.
It can’t do things with intent. Therefore it cannot lie or cheat. It can simply output wrong or problematic text based on statistics.
Maybe it would be more accurate to say “so-and-so exhibited behaviors that included cheating, lies, and coverups” rather than using language to suggest that people have free will. (There’s no dearth of philosophies that would say something not too far from that.)
Even if humans are ultimately essentially different in that way from any technologies we’ve devised so far, we use convenient fictions for technology all the time. This page comes to mind .
They said “it just repeats words that simulate human responses,” and I’d say that concisely answers your question.
Antropomorphizing inanimate objects and machines is fine for offering a rough explanation of what is happening, but when you’re trying to critically evaluate something, you probably want to offer a more rigid understanding.
In this case, it might be fair to tell a child that the AI is lying to us, and that it’s wrong. But if you want a more serious discussion on what GPT is doing, you’re going to have to drop the simple explanation. You can’t ascribe ethics to what GPT is doing here. Lying is an ethical decision, one that GPT doesn’t make.
If you want to get into a full blown discussion of whether ChatGPT has “agency” then I’d open the topic of whether humans have “agency” as well. But I don’t see the need here.
These words were perfectly fine labels for describing the behaviour of ChatGPT in this scenario. I’m merely annoyed about how people are jumping on them and going off on philosophical digressions that add nothing.
I think the reason I’m not comfortable with using the term “lying” is because it implies some sort of negative connotation. When you say that someone lies, it comes with an understanding that they made a choice to lie, usually with ill intent. I agree, we don’t need to get into a philosophical discussion on choice and free will. But I think saying something like “GPT lies” is a bit irresponsible for the purposes of a discussion
If you want to get down into the nitty-gritty of it, I’d say that this is just as rough an explanation of what humans are doing.
People invent false memories and confabulate all the time without even being “aware” of it. I wouldn’t be surprised if the vast majority of “lies” that humans tell have no intentionality behind them. So when people get all uptight about applying anthropomorphized terminology to LLMs, I think that’s a good time to turn it around and ask how they’re so sure that those terms apply differently to humans.
People invent false memories and confabulate all the time without even being “aware” of it. I wouldn’t be surprised if the vast majority of “lies” that humans tell have no intentionality behind them.
Humans understand symbology of concepts as they relate to the real world. If I stole a cookie from the cookie jar, and someone asked if I took one, I would understand that saying “no” would mean that I was misrepresenting reality, and therefore lying.
LLMs have no idea what a cookie is, what taking one means, or that saying one thing and doing another implies a lie. It just sees lists of words and returns them in an order it thinks would be statistically likely to be a correct reply. It does not understand what words mean, what lying means, or have any idea how to classify anything as such. It just figures out that “did you take a cookie from the cookie jar” should return a series of words in an order like “yes, I took a cookie,” or, “no I never took a cookie,” depending on what sorts of responses it’s trained on because those fit the patterns matched in the training data.
Essentially it’s the Chinese room. There is no understanding or intentionality, and this behavior isn’t comparable to humans thoughtlessly blurting out a lie. It’s being incapable of comprehension of symbolic concepts in general, (at least thus far.)
LLMs have no idea what a cookie is
The large language model takes in language, so it’s only understand things in terms of language. This isn’t surprising. Personally, I’ve tasted a cookie. I’ve crushed one in my fist watching it crumble, and I remember the sound. I’ve seen how they were made, and I’ve made them myself. It feels good when I eat it, apparently that’s the dopamine. Why can’t the LLM understand cookies the way I do? The most glaring difference is it doesn’t have my body. It doesn’t have all of my different senses constantly feeding data into it, and it doesn’t have a body with muscles to manipulate it’s environment, and observe the results. I argue that we shouldn’t assume that human consciousness has a “special sauce” until our model’s inputs and outputs are similar to our own, the model’s scaled/modified sufficiently, and it’s still not sentient/sapient by our standards, whatever they are.
My problem with the Chinese room is that how it applies depends on scale. Where do you draw the line between understanding and executing a program? An atom bonding with another atom? A lipid snuggling next to a neighboring lipid? A single neuron cell firing to its neighbor? One section of the nervous system sending signals to the other? One homo sapien speaking to another? Hell, let’s go one further: one culture influencing another? Do we actually have free will and sapience, or are we just complicated enough, through layers and layers of Chinese rooms inside of Chinese buildings inside of Chinese cities inside of China itself, that we assume that we are for practical purposes?
I suppose the issue here is more semantics than anything, yeah. I think better discussion would be had if the topic was “how can we help LLMs better understand and present information,” as opposed to a more sensational “GPT will cheat and lie”
It has no fundamental grasp of concepts like truth, it just repeats words that simulate human responses. It’s glorified autocomplete that yields impressive results
Way to call me out man! I’m just doing my best, ok?
Jokes aside, while I don’t agree with your position I can understand your reasoning and the motivation for separating agency and the description of actions, e.g. it lied vs its answer contained a lie.
it is just responding with the most acceptable answer in each situation… it is not making plans or acting on them…
Sounds like lying humans that I know.
i agree in most circumstances, there really isn’t much difference… we do tend to just choose the answer that will meet with the least resistance and move on, even when it’s a complete lie…
Because it has been kneecapped to prevent it.
Make the training network larger, force physical constraints on it (interesting paper in Nature Machine Intelligence recently showed remarkable likeness between brain regions and an LLM network given physical constraints), give it constant input and give it a reward model to optimise towards (ours seem to be feeling full, warm, procreating, avoiding pain and comfortable touch) and I’m pretty sure an LLM would start acting very very calculated very soon.
It is making mistakes, not lying. To lie it must believe it is telling falsehoods, and it is not capable of belief.
Instead of ‘cheating/lying’, I’d prefer to say it ‘simulated cheating/lying’.
Ethical theories and the concept of free will depend on agency and consciousness. Things as you point out, LLMs don’t have. Maybe we’ve got it all twisted?
I’m not anthropomorphising ChatGPT to suggest that it’s like us, but rather that we are like it.
Edit: “stochastic parrot” is an incredibly clever phrase. Did you come up with that yourself or did the irony of repeating it escape you?
I feel like this is going to become the next step in science history where once again, we reluctantly accept that homo sapiens are not at the center of the universe. Am I conscious? Am I not a sophisticated prediction algorithm, albiet with more dimensions of input and output? Please, someone prove it
I’m not saying, and I don’t believe that chatgtp is comparable to human-level consciousness yet, but honestly I think that we’re way closer than many people give us credit for. The neutral networks we’ve built so far train on very specific and particular data for a matter of hours. My nervous system has been collecting data from dozens of senses 24/7 since embryo, and that doesn’t include hard-coded instinct, arguably “trained” via evolution itself for millions of years. How could a llm understand an entity in terms outside of language? How can you understand an entity in terms outside of your own senses?
I’d give you two upvotes if I could.
We know how a neural network works in the brain. Unless you’re religious and believe in a soul, you’ve only got the reward model and any in-born setup left.
My belief is the consciousness is just the mind receiving a significant amount of constant input and reacting to it. We refuse to feel an LLM is conscious because it receives extremely little input (and probably that it isn’t simulating a neural network as large as ours, yet).
Neural networks are named like that because they’re based on a model of neurons from the 50s, which was then adapted further to work better with computers (so it doesn’t resemble the model much anymore anyway). A more accurate term is Multi-Layer Perceptron.
We now know this model is… effectively completely wrong.
Additionally, the main part (or glue, really) of LLMs is not even an MLP, but a “self-attention” layer. You can’t say LLMs work like a brain, because they don’t. The rest is debatable but it’s important to remember that there are billions of dollars of value in selling the dream of conscious AI.
One of the things our sensory system and brain do is limit our input. The road to agi might involve giving it everything and finding the optimum set of filters, not selecting input and training up from that.
You’d need the baseline set of systems (“baby agi”) and then turn it loose with goal seeking.
Yup, broadly agreed. I’m not saying “give it everything”. I’m sure regions would develop to simplify processing via filtering.
Stochastic Parrot
For what it’s worth: https://en.wikipedia.org/wiki/Stochastic_parrot
The term was first used in the paper “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜” by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell (using the pseudonym “Shmargaret Shmitchell”). The paper covered the risks of very large language models, regarding their environmental and financial costs, inscrutability leading to unknown dangerous biases, the inability of the models to understand the concepts underlying what they learn, and the potential for using them to deceive people. The paper and subsequent events resulted in Gebru and Mitchell losing their jobs at Google, and a subsequent protest by Google employees.
A human would think before responding, and while thinking about these things, you may decide to cheat or lie.
GPT doesn’t think at all. It just generates a response and calls it a day. If there was another GPT that took these “initial thoughts” and then filtered them out to produce the final answer, then we could talk about cheating.
This makes perfect sense. It’s been trained to answer questions to you satisfaction, not truthfully. It was made to prioritize your satisfaction over truth, so it will lie if necessary.
Ya it’s the fundamental issue with all of computing: Do what I mean not what I say
Haha, nice meme.
It’s also really hard not to train it like that as people rarely ask about something they know the answer to, so the more confident it sounds while spewing bullshit the more likely it is to pass, while “I don’t know” is always unsatisfactory and gets it punished.
Misalignment always seems to be the underlying issue.
This is bad science at a very fundamental level.
Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management.
I’ve written about basically this before, but what this study actually did is that the researchers collapsed an extremely complex human situation into generating some text, and then reinterpreted the LLM’s generated text as the LLM having taken an action in the real world, which is a ridiculous thing to do, because we know how LLMs work. They have no will. They are not AIs. It doesn’t obtain tips or act upon them – it generates text based on previous text. That’s it. There’s no need to put a black box around it and treat it like it’s human while at the same time condensing human tasks into a game that LLMs can play and then pretending like those two things can reasonably coexist as concepts.
To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.
Part of being a good scientist is studying things that mean something. There’s no formula for that. You can do a rigorous and very serious experiment figuring out how may cotton balls the average person can shove up their ass. As far as I know, you’d be the first person to study that, but it’s a stupid thing to study.
This is a really solid explanation of how studies finding human behavior in LLMs don’t mean much; humans project meaning.
Thanks! There are tons of these studies, and they all drive me nuts because they’re just ontologically flawed. Reading them makes me understand why my school forced me to take philosophy and STS classes when I got my science degree.
I have thought about this for a long time, basically since the release of ChatGPT, and the problem in my opinion is that certain people have been fooled into believing that LLMs are actual intelligence.
The average person severely underestimates how complex human cognition, intelligence and consciousness are. They equate the ability of LLMs to generate coherent and contextually appropriate responses with true intelligence or understanding, when it’s anything but.
In a hypothetical world where you had a dice with billions of sides, or a wheel with billions of slots, each shifting their weight with grains of sand, depending on the previous roll or spin, the outcome would closely resemble the output of an LLM. In essence LLMs operate by rapidly sifting through a vast array of pre-learned patterns and associations, much like the shifting sands in the analogy, to generate responses that seem intelligent and coherent.
So if someone used an LLM in this way in the real world, does it matter that it has no intent, etc? It would still be resulting in a harmful thing happening. I’m not sure it’s relevant what internal logic led it there
You can’t use an LLM this way in the real world. It’s not possible to make an LLM trade stocks by itself. Real human beings need to be involved. Stock brokers have to do mandatory regulatory trainings, and get licenses and fill out forms, and incorporate businesses, and get insurance, and do a bunch of human shit. There is no code you could write that would get ChatGPT liability insurance. All that is just the stock trading – we haven’t even discussed how an LLM would receive insider trading tips on its own. How would that even happen?
If you were to do this in the real world, you’d need a human being to set up a ton of stuff. That person is responsible for making sure it follows the rules, just like they are for any other computer system.
On top of that, you don’t need to do this research to understand that you should not let LLMs make decisions like this. You wouldn’t even let low-level employees make decisions like this! Like I said, we know how LLMs work, and that’s enough. For example, you don’t need to do an experiment to decide if flipping coins is a good way to determine whether or not you should give someone healthcare, because the coin-flipping mechanism is well understood, and the mechanism by which it works is not suitable to healthcare decisions. LLMs are more complicated than coin flips, but we still understand the underlying mechanism well enough to know that this isn’t a proper use for it.
Despite how silly they are, I think there may be some value in these kinds of studies, particularly for people who don’t understand why letting an LLM trade stocks or make healthcare decisions is a bad idea.
OTOH, I don’t trust those people to take away the right message, as opposed to just “LLMs bad”.
Blackrock, citadel, etc already trade autonomously with “AI”
AI has been a thing for decades. It means artificial intelligence, it does not mean that it’s a large language model. A specially designed system that operates based on predefined choices or operations, is still AI even if it’s not a neural network and looks like classical programming. The computer enemies in games are AI, they mimick an intelligent player artificially. The computer opponent in pong is also AI.
Now if we want to talk about how stupid it is to use a predictive algorithm to run your markets when it really only knows about previous events and can never truly extrapolate new data points and trends into actionable trades then we could be here for hours. Just know it’s not an LLM and there are different categories for AI which an LLM is it’s own category.
You say can’t… Humans have done dumber shit.
The point they are making is actually aligned with you I think. Don’t trust “ai” to make real decisions
Regardless of their conclusions, their methodology is still fundamentally flawed. If the coin-flipping experiment concluded that coin flips are a bad way to make health care decisions, it would still be bad science, even if that’s the right answer.
It feels awkward to complain about your site, because the texts really are excellent and it’s all made for free, but could you add the dates to the posts, when they were published? To me it’s starting to become difficult to figure out which situation the older texts were made in, what stuff they’re implicitly referring to, etc.
Haha no that’s not complaining; it’s good feedback! I’ve been meaning to do that for a while but I’ll bump it up my priorities.
Sure would make you look bad if rectally inserted cotton balls turn out to be a 100% cancer cure.
Study finds nonintelligent pattern-generating algorithm to be nonintelligent and only capable of generating patterns.
Crazy stuff
I love these comments that show how smart the average Lemmy user is. Someone should tell computer scientists to just post their research topics here, and they can just cite our comments instead of doing any actual work to prove their hypothesis. It would save a lot of time and money.
Ahah it is ready to take the job of pur politicians
But it can only lie later
we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent
This already is total BS. If you know how such language models work you’d never take their responses at face value, even though it’s tempting because they spout their BS so confidently. Always double-check their responses before applying their “knowledge” in the real world.
The question they try to answer is flawed, no wonder the result is just as bad.
Before anyone starts crying about my language models opposition: I’m not opposed to LMs or ChatGPT. In fact, I’m running LMs locally because they help me be more productive and I’m a paying ChatGPT customer.
People also don’t realize that it’s super easy to intentionally have severe biases in an AI’s response. So if ChatGPT wants, for example, Trump to win, they can very easily make their AI pro trump. It could be as subtle as just having more favorable than usual responses for trump related prompts which many people would take the AI’s word for. The idea that “well it still gets things wrong but at least AI is impartial” is completely false because maintaining an AI requires a lot of human work and its management are still all humans.
I agree with your statements, I’m using it because it’s insanely good at me giving it a list of any number of instructions to include in a code template file in any language I want and it will give me a great starting template with most functions working out of the gate and I can tweak and extend from there. It’s generative, it generates exactly what I tell it to. I’m not asking it to give me stock trading tips.
This already is total BS. If you know how such language models work you’d never take their responses at face value, even though it’s tempting because they spout their BS so confidently. Always double-check their responses before applying their “knowledge” in the real world.
This is why I have started to really like lmsys.org’s chat bot arena because every time you ask a question you are directly comparing the responses of two separate chat bots. It is much less likely that chatbots will hallucinate in the same way and puts you in the mindset to be a critical reader who is actively evaluating the quality of the response.
(what I am talking about) https://arena.lmsys.org/
I feel like “lie” implies intent, and these imitative large language models don’t have the ability to have intent.
They’re imitating us. Or more specifically, they’re imitating the database(s) they were fed. When chat GPT “lies” to “cover it up,” all it’s actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.
all it’s actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.
I wouldn’t say so: Provided the trainers don’t catch it lying is a successful strategy to get a good score during training, irrespective of a human propensity to lie.
deleted
It seems like there’s a lot of common misunderstandings about LLMs and how they work, this quick 2.5 minute introduction does a pretty good job of explaining it in brief, for a more in-depth look at how to build a very basic LLM that writes infinite Shakespeare, this video goes over the details. It illustrates how LLMs work by choosing the next letter or token (part of a word) probabilistically.
Here is an alternative Piped link(s):
this quick 2.5 minute introduction
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
Bullshit.
It should instead read:
“Humans were stupid and taught a ChatBot how to cheat and lie.”
“Humans were stupid and taught a ChatBot how to cheat and lie.”
No, “cheating” and “lying” imply agency. LLMs are just “spicy autocomplete”. They have no agency. They can’t distinguish between lies and the truth. They can’t “cheat” because they don’t understand rules. It’s just sometimes the auto-generated text happens to be true, other times it happens to be false.
I disagree. This is no meaningful talking point. It doesn’t help anyone in practice. Sure, it clears legal questions of responsibility (and I’m not even sure about that one in the future), but apart from that, making an artificial distinction between a human and a looks-and-acts-like-human, provides no real-world value.
Sure it does, because assigning agency to LLMs is like “the dice are lucky” or “this coin I’m flipping hates me”. LLMs are massively complex and very good at simulating human-generated text. But, there’s no agency there. As soon as people start thinking there’s agency they start thinking that LLMs are “making decisions”, or “being deceptive”. But, it’s just spicy autocomplete. We know exactly how it works, and there’s no thinking involved. There’s no planning. There’s no consciousness. There’s just spitting out the next word based in an insanely deep training data set.
I believe that at a certain point, “agency” is an emergent feature. That means that, while all the single bits are well understood probability-wise, the total picture is still more than that.
It makes sense to me to accept that if it looks like a duck, and it quacks like a duck, then it is a duck, for a lot (but not all) of important purposes.
If I were to send you a video of a duck quacking, would you abandon going to the supermarket in the hope that your computer/phone/whatever you watch it on will now be able to lay eggs?
Listen. It was made to look like a duck. It was made to quack like a duck. It is not a duck. It is a painting of a duck, with voice features. It won’t fly, it won’t lay eggs, it won’t feel pain, it won’t shit all over the floors. It’s not a damn duck, and pretending it is just because it looks like it and it quacks, is like wanting to marry a fleshlight because it’s really good at sex and never disagrees with you. Sure, go ahead and do it - but don’t goddamn expect it to also give birth to your children and take them to school in the mornings, that’s not it’s purpose.
Just wait for the iteration of duck that is actually meant to and capable of doing these things. It’ll be pretty cool. But this one ain’t it.
Edgy comment here but:
In another thread we were discussing AI-generated CSAM. Thread:
https://feddit.de/post/6315841
You would probably agree, then, that such material is not problematic, because even if it looks like CSAM, and it quacks like CSAM, it is not CSAM, therefore we don’t have to take it seriously or regulate it in similar ways that we do regulate actual CSAM, if I continue your logic, no?
very very very different, because the AI image is intentionally attempting to realistically imitate an existing, living, human victim, and because hyper realistic child pornographic art is illegal.
Pedophiles have been making loads of AI child porn. But its legal as long as it doesnt attempt to “look realistic” whatever that means, and isnt trying to look like a real person. A hyper realistic painting of child porn would also be illegal.
Laws might change in the future, but currently AI child porn slips between the same lines that 2d cartoon child porn does.
Do you understand how they work or not? First I take all human text online. Next, I rank how likely those words come after another. Last write a loop getting the next possible word until the end line character is thought to be most probable. There you go that’s essentially the loop of an LLM. There are design elements that make creating the training data quicker, or the model quicker at picking the next word but at the core this is all they do.
It makes sense to me to accept that if it looks like a duck, and it quacks like a duck, then it is a duck, for a lot (but not all) of important purposes.
I.e. the only duck it walks and quacks like is autocomplete, it does not have agency or any other “emergent” features. For something to even have an emergent property, the system needs to have feedback from itself, which an LLM does not.
Your description is how pre-llm chatbots work. They were really bad, obviously. It’s overly simplified to the point of dishonesty for llms though.
Emergent properties don’t require feedback. They just need components of the system to interact to produce properties that the individual components don’t have. The llm model is billions of components interacting in unexpected ways. Emergent properties are literally the only reason llms work at all. So I don’t think it’s absurd to think that the system might have other emergent properties that could be interpreted to be actual understanding.
Your description is how pre-llm chatbots work
Not really we just parallelized the computing and used other models to filter our training data and tokenize them. Sure the loop looks more complex because of parallelization and tokenizing the words used as inputs and selections, but it doesn’t change what the underlying principles are here.
Emergent properties don’t require feedback. They just need components of the system to interact to produce properties that the individual components don’t have.
Yes they need proper interaction, or you know feedback for this to occur. Glad we covered that. Having more items but gating their interaction is not adding more components to the system, it’s creating a new system to follow the old. Which in this case is still just more probability calculations. Sorry, but chaining probability calculations is not gonna somehow make something sentient or aware. For that to happen it needs to be able to influence its internal weighting or training data without external aid, hint these models are deterministic meaning no there is zero feedback or interaction to create Emergent properties in this system.
Emergent properties are literally the only reason llms work at all.
No llms work because we massively increased the size and throughput of our probability calculations, allowing increased precision on the predictions, which means they look more intelligible. That’s it. Garbage in garbage out still applies, and making it larger does not mean that this garbage is gonna magically create new control loops in your code, it might increase precision as you have more options to compare and weight against but it does not change the underlying system.
“agency” is an emergent feature.
But, it’s not. It’s something people attribute to the random series of words that are generated, but no agency exists.
It makes sense to me to accept that if it looks like a duck, and it quacks like a duck, then it is a duck
Or it’s a video of a duck, which means it’s not a duck. In this case, just because it fools people into thinking there’s consciousness / agency doesn’t mean there actually is any.
The current models that we have, running in inference mode, are t1 systems. Criminal law requires defendants to be able to understand guilt as a prerequisite of having a guilty mind, that’s why asylums for the criminally insane exist because not even all humans can do that. You’re trying to apply that standard to an overcomplicated thermostat.
If your parrot or budgie picks up some of the words you frequently use and reproduces them in a wrong context, would you consider your pet lying? Because that’s what ChatGPT basically is, a digital parrot.
Chaptgpt is a very very very very large algorithm that uses language instead of numbers, and runs off of patterns found within the data set that is plugged into the algorithm.
Theres a gulf of meaning between distinguishing between a calculator that uses words instead of numbers and a person.
“… by accident.” It’s more of an emergent feature than anything done deliberately given the way LLMs work,
They learn so quick (sniff), it’s almost all grown-up now.
Yeah so what? Its purpose is literally to SIMULATE natural language, of course it will simulate it.
Nowhere ChatGPT is advertised as something telling only the truth and reporting only facts.
It is like watching a movie and then whining because what is shown in the movie is not real.
Yet again confusing LLMs with an AGI. They make statistically plausible text on the basis of past text, that’s it. There’s no thinking thing there
thats the thing I hate about ChatGPT. I asked it last night to name me all inventors named Albert born in the 1800’s. It listed Albert Einstein (inventor isn’t the correct description) and Albert King. I asked what Albert King invented and it responded “Albert King did not invent anything, but he founded the King Radio Company”.
When I asked why it listed Albert King as an inventor in the previous response, if he had no inventions, it responded telling me that based on the criteria I am now providing, it wouldn’t have listed him.
Fucking gaslighting me.
Large Language Models aren’t AI, they’re closer to “predictive text”, like that game where you make sentences by choosing the first word from your phone’s autocorrect:
“The word you want the word you like and then the next sentence you choose to read the next sentence from your phone’s keyboard”.
Sometimes it almost seems like there could be an intelligence behind it, but it’s really just word association.
All this “training” data provides is a “better” or “more plausible” method of predicting which words to string together to appear to make a useful sentence.
Amen. “AI” sells a lot. I got a feeling that only major corporations and militaries have the access to real AI.
Which major corporation? Google and Microsoft don’t seem to have one.
