Thanks, Detective Drebin (love that character). Some great insight there.
It makes sense that, if the way these models are trained is to guess the next word in a sentence, they wouldn’t necessarily be great at math/logic. I’d heard that this was how AI operates, but the idea that that’s how AI learns and operates makes very little sense to me. It seems like a terrible way to create an artificial intelligence. Predict the next word? Without considering anything further out than that? Why? Why not consider where the sentence is going, where the paragraph is going, where the topic is going, where society is going, where humans are going? Why not endow it with all the knowledge (including math/logic knowledge) humans have accumulated?
In addition to not making sense to me, the fact that AI can create images indicated to me that there has to be more to it than just predicting the next word. So I assumed that I had misunderstood this thing I’d heard about “predict the next word” being how AI operates. I assumed that there was more to it than predicting the next word, and I assumed that of course you would endow your AI with logic/math/reasoning skills (and other knowledge) right from the start.
I imagine you (frankdrebin) are reading this and thinking, “boy, this guy really doesn’t understand much about AI.” And you’d be right. I don’t. And it’s kinda been bugging me and I’d really like to understand it more. Any misconceptions you could clear up or questions you could answer would be much appreciated.
