Recent searches
Search options
Traditional computer software tools resemble the standard mathematical concept of a function 𝑓:𝑋→𝑌: given an input 𝑥 in the domain 𝑋, it reliably returns a single output 𝑓(𝑥) in the range 𝑌 that depends on 𝑥 in a deterministic fashion, but is undefined or gives nonsense if fed an input outside of the domain. For instance, the LaTeX compiler in my editor will take my LaTeX code, and - provided that it is correctly formatted, all relevant packages and updates have been installed, etc. - return a perfect PDF version of that LaTeX every time, with no unpredictable variation. On the other hand, if one tries to compile some LaTeX with a misplaced brace or other formatting problem, then the output can range from compilation errors to a horribly mangled PDF, but such results are often obvious to detect (though not always to fix).
#AI tools, on the other hand, resemble a probability kernel μ:𝑋→Pr(𝑌) instead of a classical function: an input 𝑥 now gives a random output sampled from a probability distribution μₓ that is somewhat concentrated around the perfect result 𝑓(𝑥), but with some stochastic deviation and inaccuracy. In many cases the inaccuracy is subtle; the random output superficially resembles 𝑓(𝑥) until inspected more closely. On the other hand, such tools can handle noisy or badly formatted inputs 𝑥 much more gracefully than a traditional software tool.
Because of this, it seems to me that the way AI tools would be incorporated into one's workflow would be quite different from what one is accustomed to with traditional tools. An AI LaTeX to PDF compiler, for instance, would be useful, but not in a "click once and forget" fashion; it would have to be used more interactively.
@tao - nice thought! And a suffficiently complicated deterministic function 𝑓:𝑋→𝑌 may be most easily modeled as a probability kernel μ:𝑋→Pr(𝑌) - e.g. one where you use a deterministic but chaotic process to determine the output from the input. It doesn't matter much that the AI is really deterministic if the deterministic function it's computing is too complicated to understand.
@johncarlosbaez @tao Modeling AI as μ:𝑋→Pr(𝑌) is very natural at least in the context of chess and game theory (As shown in the attached image from https://arxiv.org/pdf/2302.13937.pdf ) In chess, the quality of a move x is measured by f(x)-E(μ(x)), and the sum of all moves gives us a measure of the game's quality: : $\sigma_{x\in X’} f(x)-E(μ(x))$.
@johncarlosbaez @tao However, in human vs. human chess, playing the "engine-optimal" move may not always be humanly optimal because the engine-optimal move may lead to an easy-to-play position for humans. This is why some top players regularly play sub-optimal engine moves to obtain an edge. To get a more accurate measure of play, we can subtract the above quality measure from the actual score s a player obtains in a game: s - $\sigma_{x\in X’} f(x)-E(μ(x))$
@johncarlosbaez @tao Apologies for intruding your post 
, but I thought modeling AI in chess could be of interest to your discussion.
@johncarlosbaez @tao +1 that similar workflow changes are required for chaotic functions (as often the input will slightly change), and they are also required for very expensive functions. In both cases the remedy is to cache the output, ideal in a nonchaotic form that is likely to be useful even across slight changes.
@tao in game development, a naive POV is that AI will make levels and art. The more realistic POV is that AI can be a "magic paintbrush" to help content creators work more efficiently, in line with what you are saying.
@tao "somewhat concentrated around the perfect result 𝑓(𝑥)" is a huge assumption given the way these systems work. Both in the sense that the mean might resemble the "perfect result" and that the metric for "concentrated" is related to meaning rather than statistical distribution of words. For example, LLMs are notoriously bad at handing negation and will routinely contradict themselves. Extreme ill conditioning is also important here.
@tao Well put. That's exactly the point of my current course on human-centered AI: https://niloufar.org/human-centered-ai-course/
@tao I tried to say something similar, but one of my friends here got upset.
But here is another thought. Floating point computation, used as an approximation of real-number computation, has a similar defect: we can't read off the correctness of the answer from the answer itself. It can be completely wrong, and often is, and often is not, but we don't know when. Unless we have *proved* something about the floating-point program, which is possible, but which most of the time people don't do.
@MartinEscardo @tao
yes-but -- for machine FP computation, there is the possibility of a proof of error bounds, and in many of the frequently-used mathematical packages have had that work done and/or provide the option of an estimated error.
(We did these proofs as exercises in undergrad and graduate numerical analysis classes, I've worked with people designing and implementing the libraries, and I've validated accuracy of some FP libraries and conversions).
Machine learning, who can say?
@tao Now I am picturing a dialogue between D. Knuth and chatTeX, a language model designed to typeset the remaining chapters of The Art of Programming.
@tao
The way I think about it, when trying out prompts etc, is that the model samples possible texts that a person might write, and which relate to some key words in the question.
The trick to get good answers from GPT is to change the underlying distribution: the average person, or even an average highly-intelligent intellectual, wouldn't write correct mathematical ideas. By changing the prompt we change the distribution the model samples from, essentially changing the language from English to the jargon used by some community, and then sampling texts related to the question is likely to produce the right answer.
Presumably using RLHF would give more accurate distributions than prompting, but that's both very expensive and cannot be done without OpenAI's proprietary datasets.
@tao I wish there was such an interactive compiler that would let me improve equation or figure layout, without having to fiddle with spacing and such…
@tao 从数学角度可以得到aigc接近本质的解释
@tao I'm sure you know this already, but your description of traditional software tools is somewhat optimistic. The vast, vast majority of software can only be thought of as a function 𝑓:𝑋→𝑌 if 𝑋 is pretty close to being the set of states of the whole universe, at least on any physical computer.
Which makes me wonder how far away an AI tool is that is able to carry out an instruction like "compile this LaTeX file" from essentially zero and do so successfully relatively often.