- cross-posted to:
- technology@lemmy.world
- cross-posted to:
- technology@lemmy.world
It’s clear that companies are currently unable to make chatbots like ChatGPT comply with EU law, when processing data about individuals. If a system cannot produce accurate and transparent results, it cannot be used to generate data about individuals. The technology has to follow the legal requirements, not the other way around.
You must log in or register to comment.
I have an unusual name. There is one other person in the U.S. with my name and there is something even more unique about them. I typed “Tell me about [MY NAME].” into ChatGPT, including my middle initial just to be sure and got this back:
[MY NAME] is an American entrepreneur, author, and business consultant known for his expertise in marketing and business strategy. He has authored several books on entrepreneurship, marketing, and personal development, aiming to empower individuals to achieve success in their ventures.
[MY NAME] is recognized for his practical approach to business growth and his ability to simplify complex concepts into actionable steps. He often emphasizes the importance of innovation, adaptability, and effective communication in building successful businesses.
While specific details may vary depending on recent developments, [MY NAME]'s work typically revolves around helping entrepreneurs and small business owners navigate the challenges of starting and growing their ventures in today’s competitive market landscape. His insights and strategies often focus on leveraging technology, understanding consumer behavior, and developing effective branding and marketing strategies.
[MY NAME] may also engage in public speaking engagements, workshops, and consulting services to share his knowledge and insights with a wider audience. His work likely continues to inspire and empower aspiring entrepreneurs and business professionals to pursue their goals with confidence and determination.
Not one bit of that is true either for me or for the other person who shares my first and last name but not my middle initial.
This is the problem with training LLMs on Reddit. It doesn’t know how to say “I don’t know”. So, like Redditors…. It just makes shit up.
It’s not that it doesn’t know how to say “I don’t know”. It simply doesn’t know. Period. LLMs are not sentient and they don’t think about the questions they are asked, let alone if the answer they provide is correct. They string words together. That’s all. That we’ve gotten those strings of words to strongly resemble coherent text is very impressive, but it doesn’t make the program intelligent in the slightest.
What amazes me is that people don’t find it significant that they don’t ask questions. I would argue there is no such thing as intelligence without curiosity.
They’re trained on far more than reddit. But it’s not a training data problem, it’s a wrong tool problem. It’s called “generative AI” for a reason: it generates text, same way a Markov chain does. You want it to tell you something, it’ll tell you. You want factual data, don’t ask a storyteller.
What I think is especially funny though is that both the other person and myself have done enough (not horrific) things in our lives to have things like mainstream media mentions but it still got it entirely wrong.
I’m not famous but it definitely should have known who I am.
How can a Flying Squid not be famous? Haven’t the tonight show contacted you about doing aerobatics?
We’re far more common than you’d think.
I thought I knew you from somewhere. That was gonna bother me all day.
But we know everything, why would we say otherwise when we are always the smartest person in every room we’ve ever walked into? What even is this foreign tongue ‘I don’t know’. I’ve never heard of it before. Is it latin?
So your work revolves around bringing entrepreneurs down?
In the sense that it would bring them down if they found out that I couldn’t spend money on their business because I’m not working? I suppose.
I congratulate you, and think you should be proud of overcoming your inherent invertebrate self, to not only be a prolific poster on Lemmy, but also to be an entrepreneur, author, and business consultant.
Truly you are one in a squidillion.
Thank you. You can take my new business course for only $399.95 and a bucket full of any small species of saltwater fish you can find.
I am also unique-except-one. Mine is similarly unrecognizable.
Which version of ChatGPT?
Whichever free one you can use by going to their website, but considering anything it would know about me would come from at least 13 or 14 years ago, that shouldn’t be an issue.
If you search my name on pretty much any search engine, a bunch of links come up.
I just checked and someone by my unusual name apparently retired in 1986 after a storied career. I was in about 8th grade. When I provided more particulars, it just said I’m too obscure. Which isn’t terribly surprising. I should turn up on searches, but I’m a fairly private person and avoid any sort of publicity.
I couldn’t find any specific information about a [redacted] born in [redacted]. It’s possible there may be limited public information available about this individual. If you have any additional details or context that could help narrow down the search, please let me know!
That being said, it was running Bing searches on me so that’s probably on the search engine and not the AI.
I did run into someone with my exact same name and married to a woman with my wife’s first name at an out of state niche conference of maybe 300 people. It caused quite a bit of confusion with the hotel booking. That was surreal because it was the first time ever running into someone with my last name that wasn’t family. Anyway, apparently both of us are completely off the radar, which is good because I’d hate for him to have turned into a career criminal or something.
ChatGPT is not an information repository.
ChatGPT is not an information repository.
ChatGPT is not an information repository.
The correct answer to this problem is not “we can’t correct it”; it is “this class of task is completely out of scope for ChatGPT, and we will do everything we can to make sure users understand that”. Unfortunately, OpenAI knows damn well this is how the public perceives and uses its product and seems happy to let this misconception persist.
We do need laws to curb this, but it’s really more a marketing issue than a technological issue. The underlying technology is amazing; the applications built around it are mostly garbage. What we have here is a hype trainwreck.
Yet, LLMs are trained on data - an information repository. They are capable of accessing and recalling the contents of that information repository, and relaying information from that repository to an end user. It may not be an information repository functionally, but it legally seems to have the capabilities to be classified as one. (I am neither a lawyer nor a programmer, and I am not in the EU.)
The software breaks the law, and the people who built it knew that this was likely the case. It was developed as a research project, which has very different legal requirements from a consumer product. They might not outright ban the software, but they might issue some hefty fines, etc. Banning a product is not the only recourse of the courts.
They are capable of accessing and recalling the contents of that information repository, and relaying information from that repository to an end user.
This is not correct based on my understanding of LLMs, but I am certainly not an expert. As I understand it, it’s basically a statistics exercise in how they determine what order to put words into. They don’t ‘look stuff up’ in their training data. They probably don’t even have access to their training data once the model is complete. These models are trained on terabytes of data but are small enough to fit in memory, so it’s impossible for them to still have access to all that. But it wouldn’t matter if they did, because that’s not how they work.
To me that sounds like a distinction without a difference. A jpeg is not an image, but a set of data that can be algorithmically processed and rendered as an image - which is why it can fit in a smaller space than a bmp. Despite the technical differences, a jpg and a bmp are legally equivalent. If something is illegal in a bmp, it’s also illegal in a jpg. The same laws apply to EVs and gas vehicles. The same laws apply to vinyl records and cassette tapes. The law does not care about the mechanism.*
*for the most part
You’re illustrating the issue so many people have with this technology. Without a fundamental understanding of how it works, people will attempt to use it in ways it shouldn’t be used, and won’t understand why it isn’t giving them correct information. It simply doesn’t have the ability to do anything but put words in an order that statistically will resemble how a human might answer the question.
LLMs don’t know anything. They can’t tell fact from fiction (and are incapable of even trying), and don’t understand concepts such as verifying info when requested. That’s the problem, they don’t ‘understand’ anything, including what they are telling you. But they do spit out words in a statistically probable order, even if the result is complete bullshit. They do it so well that they can fool most people into thinking the computer actually knows what it’s telling you.
LLM isn’t a compilation of its training data, anymore than a cake is a pile of eggs, flower and sugar.
Here’s a better metaphor because yours completely misses the mark when it comes to the difference between an LLM and an actual encyclopedia.
A painter will spend years honing his craft by studying other paintings as well as photos and real life. If you ask him to paint you a house from memory and try to build it with what he gives you, that just makes you an idiot, it doesn’t make him a bad architect.
Chatgpt is not an encyclopedia and any thing it says that is remotely important to your personal or work life should be verified. They explicitly tell you it can and will give false responses.
Jpg is a lossy compression algorithm. Statistical probability of words occuring in sequence is not compression. That’s like saying generative images are compression, they aren’t. It’s not producing blurry matches of images, it’s producing something “novel”. Otherwise, that would be considered over fitting the data.
To clarify, I mean to say that users should not consider it an information repository, because it does not function as one, by design. Whether it should be classified as such under the law is another matter, one on which I do not have enough knowledge to comment. I do think OpenAI is presenting ChatGPT inappropriately, and I hope they will be held accountable for that.
I’m sure in the future we will see true databases built on the same technology (and they will be awesome, if implemented properly). But that’s not what ChatGPT is (or, as far as I know, any other existing LLM-based application). Any information it is able to “recall” is almost a coincidence of how it was trained. You can sort of think of it like lossy compression. The LLM gets all of its information from its training set, but it is not designed to retain any specific information from the training set in full. In cases where it does, that usually means one of two things:
- The information appeared many times in the training set, enough prevent it from being washed out.
- The model is far bigger than it should be, and is overfitted to its training data.
OpenAI openly admits that it is unable to correct incorrect information on ChatGPT. Furthermore, the company cannot say where the data comes from or what data ChatGPT stores about individual people. The company is well aware of this problem, but doesn’t seem to care.
Wow. Where are all the news stories about THIS?
If you try to start learning how they work, the first thing you realize is that hallucinations are fundamental to how the technology works. Of course they are unfixable. That’s literally how they work.
They’re broken clocks that happen to be right more than just twice a day, but still broken nonetheless.
Exactly like humans.
It’s an inherent issue with deep learning. Awareness of this among people who are regularly using these tools is very low, which is troubling.
https://umdearborn.edu/news/ais-mysterious-black-box-problem-explained
You’re reading one right now?
The technology has to follow the legal requirements, not the other way around.
Given the possibility that this is a general problem of AI that simply cannot be corrected, the law could end up meaning that LLMs are outright forbidden in the EU. If that’s true then the legal requirements will have to be changed, there’s no way the EU would actually ban them. It’d be like opting out of the internal combustion engine due to some detail of an old law that they happened to violate.
they would not be banned outright. They just can’t be used to process data about customers.
But an ai furry porn generator doesn’t necessarily process customer data
Not unless you want furry porn about your… taxes?
That would get it banned in the US, not the eu.
The economic effects would still be enormous. You can amend my analogy to “banning internal combustion engines when their services are being sold to customers”, leaving them free for individuals to use to carry themselves around, and it’d still have a massive impact.
Europe’s not going to kneecap themselves over this.
-
If the world had opted out of the ICE early, maybe we wouldn’t be in quite the global warming situation we’re in.
-
LLMs are still a novelty product that can barely perform their novelty. Comparing them to the wildly useful and game changing ICE is not terribly accurate.
It’s not the world that would be opting out of the internal combustion engine in this analogy, it would be Europe. There rest of the world would go on industrializing while Europe remains in the 19th century. It would be an insane act of self-destruction.
Or European websites will suddenly be the only ones worth visiting because they aren’t buried under mountains of LLM garbage text.
-
The technology has to follow the legal requirements, not the other way around.
This is something that really needs to be taught better, at least in the US.
GDPR doesn’t mean that LLMs are forbidden in the EU, but it does mean that the companies that create them may be liable for damages. That said, the damages must be real. Actual damages is somewhat cut and dry (e.g., ChatGPT publishes defamatory information about you, and someone relies on it to your detriment), but GDPR also contemplates damages for distress (e.g., emotional).
If that’s true then the legal requirements will have to be changed …
I think this position needs to be rejected in the strongest possible terms. Our response to any emerging technology should not be “It’s too good not to have, so who cares if people lose their rights?” The right to privacy and with it the right to control one’s likeness, name, and personal data is a much easier right to conceptually trade away than, say, the right to bodily integrity, but I think we’ve seen enough dystopian sci-fi at this point to understand where the intersections might lie between other rights and correspondingly miraculous technologies. [And after all, without the combustion engine we probably wouldn’t be staring down the barrel of climate change right now.]
Should we, for instance, do away with the right to bodily integrity if it means everyone gets chipped shortly after birth? [The analogy to circumcision is unintentional but not lost on me.] After all, the chips mean that we can locate missing and abducted children easily and at trivial cost. They also mean that we no longer need to carry money or proxies for money. Crime is at an all-time low. Worth it, right? After all, the procedure is “minimally invasive.”
The point is, rights have to be sacrosanct. They need to be the first consideration, and they need to be non-negotiable. If a technology needs those rights to bend or give way in order to exist, then it should not exist. If it’s of sufficient benefit to society, then it can be made to exist in a way that preserves those rights, and those who are unwilling to create it in such a way should suffer the sanction of law.
Or on the other hand, maybe we have to admit that these technologies were released before they were finished, and that was a dangerous decision. It’s now been well documented that chat gpt and similar technologies were rushed to the public against the advice of some of their developers.
The developers will need to devise ways for the LLMs to understand their own training data.
Llm tech is not rushed. The models are not for accurate information and trying to use them this way is out of their scope. What’s rushed is corpos trying to use them for searches
It so lol as it say hubert manne is vampire lizard technomancer from alpha centauri. so much I laugh because it is so not truth. fun funny it is.
This stuff is literally a bullshit(1) machine. How can you fix it without making something else entirely?
