Intial Post from Linkedin:
https://www.linkedin.com/posts/matthewwallaceco_anthropic-is-on-but-the-interesting-activity-7255240609672671232-5qB_?utm_source=share&utm_medium=member_desktop
Anthropic is on 🔥 🔥 🔥 🔥
But the interesting story here is NOT the tool. ChatGPT has had an analytics sandbox for ages, and certainly having it write code is useful - I've intentionally forced it to prototype to iterate w/out me having to test the code. However, there's a connection at the FM level. Every major effort on foundational models is using the models to drive data:- Meta used Llama 2 to help build Llama 3, significantly improving the data pipeline. It wasn't an architectural change that made 3 much better than 2, it was the data.
- Meta used Llama 2 to help build Llama 3, significantly improving the data pipeline. It wasn't an architectural change that made 3 much better than 2, it was the data.
- Meta used Llama 2 to help build Llama 3, significantly improving the data pipeline. It wasn't an architectural change that made 3 much better than 2, it was the data.
- Anthropic has the double whammy here of the code sandbox and computer control; which will allow them potentially to do a lot, like: drive around the interactive web, learning from changing pages during interactions; getting data from unusual sources; and, via a code sandbox, improve the model's ability to write code by adding to and improving the data.
Of course, Anthropic and others were probably already doing this for code - certainly in research there's a good amount of effort to generate synthetic data, and moreso, there is an enormous corpus between sites like leetcode and repositories like GitHub, to let LLMs dabble with. But papers have already shown that with the right focus on data LLMs can become extremely strong in areas they were historically terrible at - for example, Math directly from inference - and it is fascinating to think about how these capabilities - code and browser use - can, at maturity, then allow them to potentially ferret out and grow synthetic data for future iteration.
Whether models can "think" originally or not remains a red herring question - models as tools to accelerate human discovery is a paradigm shift.