Conversation

Notices

1. Zach Weinersmith (zachweinersmith@mastodon.social)'s status on Saturday, 27-Jul-2024 15:00:09 UTC Zach Weinersmith

   Are there any LLMs yet that are able to kick questions over to a physics model? Like, it seems that at least for some questions, the way we get an answer isn't by thinking about what we've seen or learned or said before, but literally imagining the world. For kids, this seems to include things like finger counting for addition.

   • Ricardo (rgbes@mastodon.social)'s status on Saturday, 27-Jul-2024 15:00:07 UTC Ricardo

     in reply to

     @ZachWeinersmith as I understand it, LLMs interpolate while humans have the ability to extrapolate. In order to extrapolate from a limited dataset, you need a model of reality, and LLMs don't have one, they just look for the most likely token to drop next to the previous one. They may (just may) be able to find interesting connections between data points, but they cannot go outside their training box.

   • Zach Weinersmith (zachweinersmith@mastodon.social)'s status on Saturday, 27-Jul-2024 15:00:08 UTC Zach Weinersmith

     in reply to

     Like, GPT-3 failed questions of the form "I'm in the basement and look at the sky. What do I see?" GPT-4 fixed this by having humans correct its mistakes. I imagine if I were a kid getting this question for the first time, especially in a place where there aren't typically basements, what I'd do is probably imagine being in a basement.

   • Zach Weinersmith (zachweinersmith@mastodon.social)'s status on Saturday, 27-Jul-2024 15:00:08 UTC Zach Weinersmith

     in reply to

     And the model I use could be fairly stupid. Just a sort of underground box. No need for deep physics or even an understanding of what the point of a basement is.

   • Ricardo (rgbes@mastodon.social)'s status on Saturday, 27-Jul-2024 15:12:21 UTC Ricardo

     in reply to

     @ZachWeinersmith and of course, LLMs do what they do while consuming ridiculous amounts of energy and water, while physicists only need coffee and a slice of pizza.