So I've explained elsewhere why I think actual continual learning is needed.
I don't think you can have AIs that perform whole jobs as competently as humans
if they are forced to just write marked-on files from session to session.
Just to give an illustrative example, imagine if this is the way that students learn to play the saxophone.
So you have one student, he's never played the saxophone before, he goes into the music hall,
he tries to play it, of course it's his first time, so he fails.
And he writes down a bunch of notes about what went wrong.
And there's a next student who's waiting outside the music hall.
He comes in, he reads all these notes.
He's also never played, so of course he messes up.
And he continues to add on to these notes.
And you have an infinity of students who are outside the music hall who keep writing notes to the next person.
I don't think there's any sequence of text they could write to each other that would allow the subsequent
student to just nail the saxophone from the first try.
At some point, you actually have to accumulate the relevant experience into your brain.
I think the same thing will be true for a lot of skills
that we want AIs to actually accumulate from all the different workplaces in which they're deployed.
Okay, so what changes once we have actual continual learning?
One, I think that a lot of proposals that have been put forward about regulating AI assume
that you train a model and then you deploy.