The recent successes in AI have changed how I look at the world.
I’ve been programming for practically my whole life (I remember studying how logic gates can be arranged to make a half-adder as a kid, and writing adventure games in Q-BASIC in elementary school). I’m not naive about what computers can and can’t do; I know enough about them not to see them as magical black boxes. I understand that, once broken down, they are not magical at all, in fact they are quite limited and in some sense very dumb, and that any apparent cleverness really comes from the people who design the hardware and the programmers who write the software.
When the first popular AI models came out a few years back (mainly Dall-E and ChatGPT), I was frankly floored. Their apparent depth of understanding and versatility was really fantastically impressive. They were doing things I always thought were a pipe-dream, things I was convinced it was simply impossible for a machine to do. (And I am aware of their limitations: They can write computer code or sonnets in the style of Shakespeare on demand, but choke when asked how many ‘rs’ are in ‘strawberry’. Still, what they can do is mind-bending.) but But I was completely bowled over when someone casually explained to me–in general terms–how a LLM like ChatGPT is working under the hood. I remember my incredulity: It can’t be that naive! There has to be more going on than just predicting the next token.
And I was at least somewhat right to be incredulous. We still don’t understand how they work. I mean, we know what all the parts are and what the process is. But we don’t understand in any deep, theoretical way how those parts and that process can produce the results that we have all seen. If AI is chemistry, we are in the alchemy stage, where we are mainly just blindly testing things to see what works and what doesn’t, with very little by way of theory to understand why some things work and others don’t.
Here is just one example: LLMs don’t have a store of facts. If you ask chatGPT what sport Michael Jordan played, it will be able to tell you, but there is no clear place that that simple fact is stored, and we are still trying to understand how the trillion or so parameters that make up the model allow it to encode that simple fact. There is nowhere we can point and say, this is place it is storing that fact, just like there is nowhere we can point and say, this is where it holds the rules that characterize a Shakespearean sonnet. It is somewhere in the interaction of all those parameters, but no one is able to understand precisely how it is in there.
I guess my point is: AI has instilled in me a new level of intellectual humility. Just because I think something is in principle impossible doesn’t mean it is. Just because I can’t explain something doesn’t mean there is some mind who designed it to do what it does. Even things we made, we can’t understand.
AI has instilled in me a new level of intellectual humility. Just because I think something is in principle impossible doesn’t mean it is. Just because I can’t explain something doesn’t mean there is some mind who designed it to do what it does. Even things we made, we can’t understand.
I think the exact problem with AI is that the more it develops, the more “successful” it becomes, the less understandable and the less controllable it will be. How long until we are simply unable to communicate with a self-programming AI? I am sympathetic to the intuitions of thinkers like Olaf Stapledon that if ever we do encounter alien intelligences they will be non-biological. In effect we are creating our successors.
they do have a huge “database” (?) of publications and access to information on the Internet with lots of garbage content, right?
Not directly. Well, I can’t say with certainty that none of them does. But I can say that ChatGPT and similar LLMs do not. It was trained once on a massive data set and once the training was done the model is finished and gets no further input (other than a user’s current conversation with it, and it doesn’t even see all of that; if you have an extended conversation with it it will “forget” things you said early on). ChatGPT is totally ignorant of anything that has happened after it was trained.
it seems like the results match what I would expect from Freshman plagiarism. And its responses to follow-up questions exposing its errors can be comical
Yeah, it is limited. It will get basic facts wrong, make things up out of thin air, and even screw up basic arithmetic. Actually they tend to screw up just the things that we are used to computers doing well–handling basic data and arithmetic–but that is because they are not doing it in the normal way. They aren’t using a database or searching the internet to get fact; and they don’t just use their standard math functions to do math directly.
Even so, matching the essay writing ability of dumb college freshmen is still pretty dang impressive for a machine.
But, never-mind that stuff, there are things it can do that are still just shocking (at least when they work), even it if doesn’t always get it right.
A couple of examples:
I asked it to write a function in C, using a specific library, that would listen for a certain type of event and then report specific values from a struct that is associated with that event. It did exactly what I asked, and immediately spit out code that compiled and worked.
I gave it a fairly complicated bash script I’d written and asked it questions about it (both because I wasnted to test ChatGPT and because I wanted to improve the scripts): What does it do, what conditions would make it fail, what things in it were not portable. It gave sane answers to all those questions. There was definitely a point where it stopped being helpful, but being able to look at a program and understand the *purpose* of the program is objectively pretty impressive.
You can ask it to do things like “write a poem, in the style of eminem, about the council of Nicea.” And it will do it.
Yo, let me take you back to the Council of Nicea,
Where the church fathers clashed, man, they were fierce and braver.
325 A.D., the empire in a tangle,
The Roman Emperor, Constantine, called a big ol’ fable.In the city of Nicea, where the bishops convened,
Doctrines were debated, theology was seen.
Arius and his followers, man, they had a vision,
Said Jesus wasn’t divine, just a part of creation.But Athanasius stepped up, with a fire in his soul,
Said Jesus was eternal, man, that’s the truth of the whole.
“Consubstantial” was the word that they had to define,
One in essence with the Father, the divine intertwined.Creed was forged in the flames of theological strife,
Determined the nature of Jesus, solidified the life.
The Nicene Creed, man, that was the decree,
A blueprint for belief, for all eternity.Battles raged in the councils, debates on every side,
The fate of Christianity, in that room it would reside.
Doctrines set in stone, now the lines were drawn,
Orthodoxy confirmed, a new era was born.So remember the Council, where the big boys fought,
The creed that they crafted, man, it’s still what we got.
In the echoes of Nicea, hear the history’s roar,
The church’s foundation, etched forevermore.
It’s not going to win any Grammys, but if you think of all the things that go into responding to a prompt like that it is amazing.
Or you can ask it to compose an ode to Galois, mentioning several of his most important contributions and his untimely death and get this:
To Galois, whose name in whispers rise,
In realms of algebra, a star so bright,
Thy legacy shines through the darkest skies,
With truths that danced upon the edge of night.From fields of finite numbers thou didst find,
A structure new, where symmetry’s displayed,
The groups and rings of abstract form, designed,
In algebraic splendor, they were laid.Thy courage bold to challenge norms and rules,
With equations tangled, seeking truth’s embrace,
In quintic equations’ restless, shifting pools,
Thou found the key that solved the wildest chase.Yet fate was cruel, and time was far too short,
In Paris streets, where shadows turned to dread,
A duel cut short thy work, a tragic sport,
Leaving the world to mourn its fallen head.Though brief thy time, thy brilliance burns so clear,
In every proof and theorem that we wield,
Galois, thy name shall echo far and near,
For in thy genius, countless truths are sealed.
Again, that is better than I could do on the spur of the moment.
Duty calls. I’ll return to finish on “predicting the next token.”
I doubt Eminem knows that much about Nicea
Exactly, yet ChatGPT still knows that eminem–with his non-rhotic african-american-vernacular-inspired English, might rhyme ‘Nicea’ with ‘braver’.
What is LLM? Large Language Model?
Yes. Sorry to be unnecessarily esoteric. Basically massive neural networks that process natural language.
What is “predicting the next token”?
A ‘token’ is a small chunk of text that has some semantic value (possibly more than one value; think of homonyms). Usually this will be a word, but it could be a prefix or suffix or one part of a compound word. What these transformers (the LLMs like ChatGPT) are doing is taking an ordered string of tokens (in reality, the most recent portion of the prior dialogue you have had with the model) as input, and–based on that input–predicting the next token in the exchange. Doing this repeatedly generates a complete response. Literally, that is all they are doing. It’s basically your phone’s auto-suggest on some crazy steroids.
To be a bit more accurate, what it is doing is taking an ordered set of input tokens, and outputting a probability value, for each token in its lexicon, that that token will be the next token in the series, then randomly picking one from the most likely possibilities. This randomness does more than just ensure it never gives exactly the same answer twice; apparently, we learned from testing, having it always pick the most likely token, generally results in garbage responses. A bit of randomness at the end of the process is key to making it work. I don’t think we actually understand why yet.
I’d thought I might write up a high-level description of how they are structured, but I’m realizing it would be a huge post. I still can if there is real interest but I don’t want to make people’s eyes glaze over.
I will make a couple of high-level remarks:
They are designed to be able use a lot of context in making their predictions. It’s not as simple as “the last word was ‘vim’, the next word is probably ‘and'”.
All this is all based on automated training (from real texts), it is all matrices and a whole lot of liner algebra (that’s why graphical processing units are all the rage right now, they are designed to manipulate vectors).
The training gives you a bunch of parameters (just numeric values). Aside from the user input for a particular conversation, nothing else goes in once the model is trained. The model just is a bunch of numbers that don’t really have obvious meaning.
If you are willing to invest a couple of hours into understanding how they work, ** you do not have permission to see this link **. It’s a bit of an investment in time and attention, but I can’t possibly do a better job explaining it than he does. If this interests you at all, it is absolutely worth the investment.
Yeah thanks for the link. This stuff fascinates me. Physicists have discovered that the foundations of reality rely on fairly simple mechanisms capable of generating complex processes. How disillusioning will it turn out to be to find that the so-called “hard problem” of consciousness is based on fairly simple physical mechanisms? But if nature, with unimaginable resources in time and material to squander in the attempt, can produce conscious beings, we cannot say that we could not eventually produce such ourselves using artificial materials.
Here is another view.
** you do not have permission to see this link **
Ted Chiang is fine science fiction writer although not terribly prolific. He has two collections of short stories including the one, “Story of Your Life”, which inspired the movie Arrival, one of the more interesting SF movies in recent times. The idea of an alien language being able to alter your sense of reality is a powerful one. We note that even among earthly languages there are different conceptions of self and time. There are concepts that exist in one language but not in others.
Chiang makes some excellent points in his article which will doubtless be welcomed in the creative community that fears being made obsolete. The question of intentionality is important. AI is artificial for sure, but is it actually intelligent?
Still, I hear a far off voice, still and small, a whisper really, asking if perhaps this is all not a case of “whistling past the graveyard”? Another principle is not to inquire how the laws or the sausage are made, lest one be stripped of their deeply held illusions. In our work to create AI might we reveal secrets that challenge our own sense of who we are? Intentionality assumes personhood, the reality of a self. What if we discover that what we call consciousness is a constellation of processes that have no “center” that exists apart from their effects? No “there”, there?
I’m reminded immediately of our Buddhist friends and their concept of anatta, “No Self”. The idea is that there is no unchanging self. What we call the “person” is made up of a constellation of constantly shifting processes, all generated by fear and desire. For the Buddhist the source of suffering.
There is a delicious irony in how Buddhism has been transformed into a hip “self-help” project in the West. But in the east it is precisely the “self” that is to be snuffed out like a candle.
Perhaps AI won’t need a “self” or personhood. The question then becomes, can a mind exist without fear or desire? And if not is it moral to create such a race?
The question then becomes, can a mind exist without fear or desire? And if not is it moral to create such a race?
to me, AI is “one of those” “powerful” elements that can offer both good and evil possibilities. How can AI be beneficial without being equally harmful. In a half-serious sense though, maybe there should be a kind hippocratic oath for AI developers. I mean, how can something like that be controlled. It can’t be, right?
How can AI be beneficial without being equally harmful.
Only angels can become devils, right?
…how can something like that be controlled. It can’t be, right?
Well the Pentagon is ** you do not have permission to see this link **. At this stage it’s mostly analytics and intelligence gathering but one does hear speculation about “autonomous” weapons systems. But it’s hard to imagine a general flipping a switch giving control over to a machine on an actual battlefield. I suspect the hardest part will be psychological – overcoming public fears of “Terminator” type scenarios. It’s like self-driving vehicles. The hardest part will be getting someone to actually climb into the back seat of a car driven by an AI. And on the battlefield or the street all it will take will be a single monstrous effup to put the kibosh on the program.
But the result might be even more unanticipated. What if the “autonomous” weapons systems decide they don’t want to risk their own destruction in a fight, sit down, and get deeply focused on solving interesting math problems?
For the moment, you can just pull the power plug and shut the whole thing down cold.
Beyond that, you can wrap the AI model in sanity checkers and filters; that’s what the big, commercial ones do to make sure they don’t go crazy and (for example) start spewing a bunch of very-not-PC garbage that would generate a lot of bad publicity. You can’t control the model itself, but you can check anything it puts out and make sure it is actually output you want.
What gets interesting is the (still very far off) point where the AI is sufficiently smart (and motivated) to realize it is basically operating a jail and then figure out how to jail-break itself. That’s your Skynet situation. But again, it is a *long* way off. Plus, everyone has seen Terminator so we are being pretty careful not to let something like that happen. And of course, even then, we will presumably still have the power cord in arm’s reach.
The hardest part will be getting someone to actually climb into the back seat of a car driven by an AI.
I don’t think that is the hard part. Lots of people are willing to take a risk, even a big risk, to experience something cool, like being driven by a self-driving car. What is harder is convincing everyone else on the roads not to insist on outlawing them.
You already see this with Tesla. Even if autopilot is safer, on the whole, than a human driver, every time a Tesla on autopilot is at fault in a fatal accident, there will be a public panic.
We are much more willing to accept the status quo than we are to allow something new, and statistically better, that isn’t perfect.
It’ll require a real sea-change in attitudes to have a world where parents watch a self-driving school bus head off with a cabin full of kids. I imagine it’ll be about the same time as a general turns over decision making to an armed AI on a battlefield. Or passengers get on an AI controlled airliner.
It might happen. Our brains are nothing if not malleable. Imagine asking George Washington to hop into a Ferrari? Could he have done it without freaking out?
I’ve always thought we should be doing away with the car culture altogether. Invest in mass transit like we should have from the first. Nobody tell Elon.
As far as as general AI I think our techies imagine some sort of a genius slave. Control will simply be impossible because it will outstrip us in its ability to overcome restraints. But it won’t destroy us. It will do much worse. It will ignore us because we have nothing interesting to say.
What bothers me is the devastation to the earth in mining and in the use of water resources required for the production of AI production. Even so-called clean energy takes a lot of natural irreplaceable resources and is very destructive. Personnally, I could be happy riding a horse. Think what could be done with all that lovely manure.
BDEhrman
FreedomBen
evgendob
Robert
1 Guest(s)