So one definition of intelligence is sample efficiency.
That is to say, how much data do you need in a given domain to operate fluently and competently?
And it's actually not clear that we've made that much progress in training sample efficiency over the last few years.
It seems like more so we've just dramatically widened and improved the data distribution.
The main way that AI has been getting better is from adding more and better
data and scaling the compute required to develop that data in the first place.
Obviously, RL is the main way that this has happened.
You can think of RL as basically a kind of synthetic data generation
where you dump a ton of compute against a verifier or a rubric if you have an LLM as a judge.
And you do this in order to find out what the good data is in the first place.
And then you train your model to predict these correct rollouts
much in the same way that you might train that model to predict the next word in internet text.
For this process to work, the model must have at least some prior probability to anticipate
the correct solution in the first place, which is why you need mind stretching amounts of human expert
trajectories in every single field and skill that you want the model to eventually be competent in.
It's hard to overstate how task specific and bespoke this human expert data is.
If you want some intuition, I recommend checking out the job descriptions on McCore or Surge's websites.
There are listings for word specialists who will convert legacy documents into polished word files and legal experts
who will write realistic M&A deligences or securities filings
and management consultants who will write up template market research.