I am an LLM researcher at MIT, and hopefully this will help.

As others have answered, LLMs have only learned the ability to autocomplete given some input, known as the prompt. Functionally, the model is strictly predicting the probability of the next word⁺, called tokens, with some randomness injected so the output isn’t exactly the same for any given prompt.

The probability of the next word comes from what was in the model’s training data, in combination with a very complex mathematical method to compute the impact of all previous words with every other previous word and with the new predicted word, called self-attention, but you can think of this like a computed relatedness factor.

This relatedness factor is very computationally expensive and grows exponentially, so models are limited by how many previous words can be used to compute relatedness. This limitation is called the Context Window. The recent breakthroughs in LLMs come from the use of very large context windows to learn the relationships of as many words as possible.

This process of predicting the next word is repeated iteratively until a special stop token is generated, which tells the model go stop generating more words. So literally, the models builds entire responses one word at a time from left to right.

Because all future words are predicated on the previously stated words in either the prompt or subsequent generated words, it becomes impossible to apply even the most basic logical concepts, unless all the components required are present in the prompt or have somehow serendipitously been stated by the model in its generated response.

This is also why LLMs tend to work better when you ask them to work out all the steps of a problem instead of jumping to a conclusion, and why the best models tend to rely on extremely verbose answers to give you the simple piece of information you were looking for.

From this fundamental understanding, hopefully you can now reason the LLM limitations in factual understanding as well. For instance, if a given fact was never mentioned in the training data, or an answer simply doesn’t exist, the model will make it up, inferring the next most likely word to create a plausible sounding statement. Essentially, the model has been faking language understanding so much, that even when the model has no factual basis for an answer, it can easily trick a unwitting human into believing the answer to be correct.

—-

⁺more specifically these words are tokens which usually contain some smaller part of a word. For instance, understand and able would be represented as two tokens that when put together would become the word understandable.

The victims have beed identified as the owner of the flight school, a flight instructor, and a student.

https://www.wtnh.com/new-england-news/no-survivors-after-plane-crash-in-greenfield/

I hate that I am defending Israel when I say this because what is occurring in Gaza is tragic, but a lot of people are confusing “Genocide” for perceived “War Crimes” as defined by international law and also confusing “Hamas” for “Palestine” or the “Palestinian Authority”.

Hamas is terrorist government (similar in nature to the Taliban) that receives a lot of external funding from countries that actively wish to see the death of Israel and all Jews, making Hamas the chief perpetrators of Genocide in this conflict despite how ineffective they have been in their goals.

Israel was attacked by this terrorist government, and is now defending itself with the expressed war goal of destroying Hamas. While Israel has had a tenuous relationship with the Palestinian people (namely the government’s active efforts to limit the Palestinian Authority and drag their feet on grant the PA more autonomy and their own state which is deplorable and inexcusable), they do not and have not wished to kill an entire culture of people.

Complicating matters, Hamas commonly employs warfare techniques that go against the Geneva Convention like placing government and military headquarters in basements of protected buildings like Hospitals and places of worship. The moment they do that, and abuse those international recognized sanctuaries, they become legitimate military targets leading to the tragic deaths of unwitting civilians.

People can object to the war on the grounds that war is tragic and results in many civilian casualties, but to make meritless claims is detrimental to both international institutions and to the definition of a Genocide. South Africa calls what Israel is doing a genocide, but also explicitly looks the other way with Ukraine and continues to forge close ties with Putin? (For the record, Russia’s actions in Ukraine are also not considered genocide under it’s strict international definition, but they have been found guilty of war crimes).

Israel has an internationally recognized right to defend itself, and it is doing that by dismantling Hamas through force. The Palestinian people are unfortunately caught in the crossfire. With that said, Israel’s methods to this end are not above criticism, and they have faced pressure from the US and Biden to limit civilian casualties wherever possible, and use ground forces to directly attack Hamas rather than relying on airstrikes that have resulted in many innocent deaths.

For those reading who think all war is bad, I’ll leave you with this quote from John Stuart Mills:

War is an ugly thing, but not the ugliest of things: the decayed and degraded state of moral and patriotic feeling which thinks that nothing is worth a war, is much worse. When a people are used as mere human instruments for firing cannon or thrusting bayonets, in the service and for the selfish purposes of a master, such war degrades a people. A war to protect other human beings against tyrannical injustice; a war to give victory to their own ideas of right and good, and which is their own war, carried on for an honest purpose by their free choice, — is often the means of their regeneration. A man who has nothing which he is willing to fight for, nothing which he cares more about than he does about his personal safety, is a miserable creature who has no chance of being free, unless made and kept so by the exertions of better men than himself. As long as justice and injustice have not terminated their ever-renewing fight for ascendancy in the affairs of mankind, human beings must be willing, when need is, to do battle for the one against the other.

It’s literally an area chart. There’s no way to “skew” it in this format. $800B of $6,300B in spending is always going to be 12.6% no matter how you slice it.

The actual study claims that top 10% is $41k and accounts for 50% of carbon emissions. No where does it normalize incomes for those from Kenya as the article claims. So these incomes are viewed globally. If you are in the US and make more than $20/hr hours a week, you are top 10%.

$67/hr makes you top 1%.

Others are calling to eat the rich without realizing that the global rich includes low wage earners flipping burgers at McDonald’s (I’m in Boston and minimum wage is $15/hr and an assistant manager can be hired for $22/hr).

https://oxfamilibrary.openrepository.com/bitstream/10546/621551/2/cr-climate-equality-201123-en.pdf

I’m an AI researcher at one of the world’s top universities on the topic. While you are correct that no AI has demonstrated self-agency, it doesn’t mean that it won’t imitate such actions.

These days, when people think AI, they mostly are referring to Language Models as these are what most people will interact with. A language model is trained on a corpus of documents. In the event of Large Language Models like ChatGPT, they are trained on just about any written document in existence. This includes Hollywood scripts and short stories concerning sentient AI.

If put in the right starting conditions by a user, any language model will start to behave as if it were sentient, imitating the training data from its corpus. This could have serious consequences if not protected against.

Not quite, this was made with a ControlNet. A hybrid image wouldn’t work as well as this does. But the underlying visual phenomena is the same.

This is done by combining a Diffusion model with ControlNet interface. As long as you have a decently modern Nvidia GPU and familiarity with Python and Pytorch it’s relatively simple to create your own model.

The ControlNet paper is here: https://arxiv.org/pdf/2302.05543.pdf

I implemented this paper back in March. It’s as simple as it is brilliant. By using methods originally intended to adapt large pre-trained language models to a specific application, the author’s created a new model architecture that can better control the output of a diffusion model.

I am a satellite software engineer turned program manager. This is not unexpected in this current environment, however the conditions that created the environment are abnormal.

This solar cycle is much stronger than past cycles. I’m on mobile, so I can’t get a good screenshot, but you can go here to see this cycle and the last cycle, as well as an overlay of a normal cycle https://www.swpc.noaa.gov/products/solar-cycle-progression

As solar flux increases, the atmosphere expands considerably, causing more drag than predicted. During periods of solar minimum, satellites can remain in a very low orbit with minimal station keeping. However, at normal levels of solar maximum, 5 year orbits can easily degrade to 1 year orbits. Forecasters says we are still a year away from solar maximum, and flux is already higher than last cycle’s all time high (which was also an anomalously strong cycle). So it will get worse before it gets better.

TLDR: Satellites are falling out of the sky because the sun is angy

Nothing about that comment was pretentious. I’m surrounded at work by people with estimated IQ greater than 140, and undoubtedly a few north of 150 (think top academic institutions in the world). That estimation is based on GRE scores and the prestige of the institution.

If anything it makes me doubt those estimations. I can’t remember what Derek said his SATs were but I was also surprised because I felt they were low for someone like him. All this just further drives home the idea that IQ does not correlate with success, maybe even in science communication.

But apparently making an observation is seen as pretentious and boastful to you.

deleted by creator

Do you always reduce everything you read to a false dichotomy or do you simply like being abrasive?

Frankly, it was much lower than I expected. As a PhD Physicist who leads a very successful career in science education, I expected him to score at least 140, and would not have been surprised to see 150.

A lot of people in this situation have already tried everything, and they are complaining on the internet because they feel like there is nothing else they can do. Most people won’t even complain about it with their friends. I know I didn’t until one day I got drunk camping. That was the trigger for me to do something about it, because it was clearly eating away at me.

I had many conversations with my wife following that, and how much our sexless marriage really bothered me, but that I am still completely and totally in love with her. We don’t have kids, so there’s not anything keeping us together. We agreed to an open relationship with our rules, and each of us has a veto which could stop the arrangement at any time. We are still completely committed to one another and love spending time together. Things haven’t been better, now when I think about being with another woman, I don’t need to feel guilty about it.

And I think this is the natural state of humanity. Everyone needs a long-term partner for stability and to care for one another, but people always felt the need to cheat as well. Monogamy is as core to our DNA and survival as is Adultery. But because we’ve talked about it, there is no sneaking around necessary, no lying. We are completely honest with each other on everything.

The same argument could be said for an apartment building too. We need to collectively realize that Single Family Houses are a luxury that most of us will never see in our lifetimes. Our grandparents were able to enjoy them at low prices because the US had half the population it does today.

Restrictive building codes that only permit building SFH is the cause of our housing shortage and not short term rentals that consist of 0.2%-1% of all dwellings.

Many leading economist argue for land value tax only as a way to incentivize the most efficient use for our most valuable resource. If land tax was used instead of property tax, a multi-acre plot in a dense urban would be taxed just as much a multi-story apartment building that takes up the same amount of space.

See the Strongtowns article on the subject. https://www.strongtowns.org/journal/2019/3/8/if-the-land-tax-is-such-a-good-idea-why-isnt-it-being-implemented

At it’s core, this is the root cause of the housing crisis. We do not have enough supply. The amount of Airbnb’s that exist is extremely miniscule and the targeting of Airbnbs is an intentional distraction tactic.

Depending on the source, 1% to 0.2% of all dwellings are listed for short-term rental in the US. That’s crazy small and has very little impact on housing prices overall.

The fact of the matter is that Single Family Homes are an incredible luxury that our parents and grandparents were able to enjoy when the country had half as many people as it does now. It is no longer sustainable to expect a SFH in the US, and the American public continuing to cling to that dream and restrictive zoning practices are really what is driving up prices.

If you want an affordable house you will need to move to a rural area where land and labor are cheap. If you want to live near any reasonably sized city, you better be upper middle class to even think about buying a SFH.

We used to run an Airbnb out of the spare rooms in our house. It was very cheaply priced, and we were always booked out for months. Super host status and everything. It was clear most people just look at the price and never the description or rules. We rented two bedrooms with a shared bathroom, and the amount of complaints we received because they had to share a bathroom with someone else was obnoxious.

We closed up shop during the pandemic and just used those rooms as guest rooms instead. In hindsight it wasn’t worth the hassle of dealing with self-centered people who expect an experience superior to that hotels at a quarter the price. We also had some fantastic guests that we loved having stay with us, but the few bad experiences dramatically overshadowed all the good decent people.

Airbnb’s are so shitty today because their customers are just as equally shitty on aggregate.

Cab prices tend to be more consistent across all times of day and location. In Boston an Uber 10 miles west from the airport can cost upwards of $120 depending on the time. A cab would be $60.

The same Uber to the airport is typically much cheaper always at $40.

You are absolutely correct! I just couldn’t think of a way to further dive into that nuance, but I also wanted the example to be relatable and tangible. Thank you!