Debugging super-intelligent AI
2000-11-14 04:00:00 UTC
Joy Reid directed me to Eliezer S. Yudkowsky's Frequently Asked Questions about the Meaning of Life. In general I don't think much of these ideas, but one thought in particular is bothering me. How does one debug super-intelligent AI?
Yudkowsky says: "We're fundamentally unable to predict the actions of anything smarter than we are - after all, if we could do so, we'd be that smart ourselves." Given that this is the case, and given that we have to write the AI programs for these super-intelligences, our standard techniques of software debugging would be inadequate. What is the proposed solution?