It has been a few years since I wrote the first blog article and I think the website has been broken for much of that time. Anyway, I am now resuscitating my writing and hope to do some daily.
Over that time I have been thinking about the control problem and have come to the conclusion that it is not soluble and I can’t even think of how to solve it. Some big problems of the day are actually solvable, eg climate change. There are number of solutions, not necessarily good ones but none the less possible solutions. Eg we could put up a parasol to shade use from the sun. It might cost 10’s of trillions of pounds but it is conceivable at £1000 per person on the plant. However the control problem of controlling a super intelligence actually seems insuperable.
I am not the only or first person to think this. I have just finished Mo Gawdat “Scary Smart: The Future of Artificial Intelligence and How You Can Save Our World”. He is clear that we can’t stop our journey to a super intelligence we can just hope to persuade it to be nice to us. Eliezer Yudkowsky has a clear rant in a Time Op-ed to argue for a total halt.
The question if we are already too late is probably touch and go. 25 years ago I thought you would need a peta flop for a human level intelligence (10^10 Neurons, 10^3 connections 10^2 per second). So potentially one of Nvidias A100 AI chips at 5 petaflops would be eouugh. But the problem was always what was the program going to be. Now we are a little clearer that we can get to the program by training on vast sets of data. This requires orders of magnituted more computer power so we might be able to shut them down.
There are only a smallish number of advanced semiconductor plants so if we shut them all down we could buy ourselves some time.