Wednesday, September 28, 2005

Information theory negates AI hopes?

Douglas S. Robertson's excellent book, "Phase Change: The Computer Revolution in Science and Mathematics" published in 2003 points out various moments of phase change in six science disciplines. According to Robertson, phase changes occur following the invention of a novel technology for collecting information. The obvious examples are the telescope in astronomy and the microscope in biology. However, greater than any other phase change trigger has been the computer which caused (and continues to cause) phase changes of unprecedented magnitude in all science areas.

Robertson goes on to cite IBM information theorist Gregory Chaitin's claim that compressed information (the shortest form of information) is governed by a conservation law, similar to the conservation of energy in physics. "Just as the quantity of energy is unchanged under physical operations, the quantity of compressed information is unchanged under logical operations...the conservation of compressed information implies that the quantity of information output from any computer program must be less than or equal to the quantity that was input."

If the same or less information comes out, it suggests that intelligence or consciousness cannot be emergent. The artificial intelligence idea that massive information input to a computer, even self-learning algorithms over the Internet, will evenutally result in intelligence as an output may be wrong. Intelligence would seem to be an emergent property which would exist on top of the underlying data inputs, as more than the sum of the parts.

Emergent intelligence might work with Chaitin's conservation law if quantity stays the same but quality increases. But is intelligence more information (quantity) or better information (quality) or information at all? The key take away is probably that intelligence cannot be an emergent property of computing and that other approaches to artificial intelligence may be more promising.