Back In The USSR
https://youtu.be/nS5_EQgbuLc

Maxwell Silver Hammer
https://youtu.be/3HuXFfq79I8

Unexplainability and Incomprehensibility of Artificial Intelligence
Roman V. Yampolskiy

Computer Engineering and Computer Science
University of Louisville
roman.yampolskiy@louisville.edu, @romanyam
June 20, 2019

"If a lion could speak, we couldn't understand him"
Ludwig Wittgenstein

“It would be possible to describe everything scientifically, but it would make no sense. It would
be a description without meaning - as if you described a Beethoven symphony as a variation of
wave pressure.”
Albert Einstein

“Some things in life are too complicated to explain in any language.. … Not just to explain to
others but to explain to yourself. Force yourself to try to explain it and you create lies.”
Haruki Murakami

“I understand that you don’t understand”
Grigori Perelman

Abstract
Explainability and comprehensibility of AI are important requirements for intelligent systems
deployed in real-world domains. Users want and frequently need to understand how decisions
impacting them are made. Similarly it is important to understand how an intelligent system
functions for safety and security reasons. In this paper, we describe two complementary
impossibility results (Unexplainability and Incomprehensibility), essentially showing that
advanced AIs would not be able to accurately explain some of their decisions and for the decisions
they could explain people would not understand some of those explanations.
Keywords: AI Safety, Black Box, Comprehensible, Explainable AI, Impossibility, Intelligible,
Interpretability, Transparency, Understandable, Unserveyability.

1. Introduction
For decades AI projects relied on human expertise, distilled by knowledge engineers, and were
both explicitly designed and easily understood by people. For example, expert systems, frequently
based on decision trees, are perfect models of human decision making and so are naturally
understandable by both developers and end-users. With paradigm shift in the leading AI
methodology, over the last decade, to machine learning systems based on Deep Neural Networks
(DNN) this natural ease of understanding got sacrificed. The current systems are seen as “black
boxes” (not to be confused with AI boxing [1, 2]), opaque to human understanding but extremely
capable both with respect to results and learning of new domains. As long as Big Data and Huge
Compute are available, zero human knowledge is required [3] to achieve superhuman [4]
performance.

With their new found capabilities DNN-based AI systems are tasked with making decisions in
employment [5], admissions [6], investing [7], matching [8], diversity [9], security [10, 11],
recommendations [12], banking [13], and countless other critical domains. As many such domains
are legally regulated, it is a desirable property and frequently a requirement [14, 15] that such
systems should be able to explain how they arrived at their decisions, particularly to show that they
are bias free [16]. Additionally, and perhaps even more importantly to make artificially intelligent
systems safe and secure [17] it is essential that we understand what they are doing and why. A
particular area of interest in AI Safety [18-25] is predicting and explaining causes of AI failures
[26].

A significant amount of research [27-41] is now being devoted to developing explainable AI. In
the next section we review some main results and general trends relevant to this paper.

2. Literature Review
Hundreds of papers have been published on eXplainable Artificial Intelligence (XAI) [42].
According to DARPA [27], XAI is supposed to “produce more explainable models, while
maintaining a high level of learning performance … and enable human users to understand,
appropriately, trust, and effectively manage the emerging generation of artificially intelligent
partners”. Detailed analysis of literature on explainability or comprehensibility is beyond the scope
of this paper, but the readers are encouraged to look at many excellent surveys of the topic [43-
45]. Miller [46] surveys social sciences to understand how people explain, in the hopes of
transferring that knowledge to XAI, but of course people often say: “I can’t explain it” or “I don’t
understand”. For example, most people are unable to explain how they recognize faces, a problem
we frequently ask computers to solve [47, 48].

Despite wealth of publications on XAI and related concepts [49-51], the subject of unexplainability
or incomprehensibility of AI is only implicitly addressed. Some limitations of explainability are
discussed: “ML algorithms intrinsically consider high-degree interactions between input features,
which make disaggregating such functions into human understandable form difficult. … While a
single linear transformation may be interpreted by looking at the weights from the input features
to each of the output classes, multiple layers with non-linear interactions at every layer imply
disentangling a super complicated nested structure which is a difficult task and potentially even a
questionable one [52]. … As mentioned before, given the complicated structure of ML models,
for the same set of input variables and prediction targets, complex machine learning algorithms
can produce multiple accurate models by taking very similar but not the same internal pathway in
the network, so details of explanations can also change across multiple accurate models. This
systematic instability makes automated generated explanations difficult.” [42].

Sutcliffe et al. talk about incomprehensible theorems [53]: “Comprehensibility estimates the effort
required for a user to understand the theorem. Theorems with many or deeply nested structures
may be considered incomprehensible.” Muggleton et al. [54] suggest “using inspection time as a
proxy for incomprehension. That is, we might expect that humans take a long time … in the case
they find the program hard to understand. As a proxy, inspection time is easier to measure than
comprehension.”

The tradeoff between explainability and comprehensibility is recognized [52], but is not taken to
its logical conclusion. “[A]ccuracy generally requires more complex prediction methods [but]
simple and interpretable functions do not make the most accurate predictors'' [55]. “Indeed, there
are algorithms that are more interpretable than others are, and there is often a tradeoff between
accuracy and interpretability: the most accurate AI/ML models usually are not very explainable
(for example, deep neural nets, boosted trees, random forests, and support vector machines), and
the most interpretable models usually are less accurate (for example, linear or logistic regression).”
[42].

Incomprehensibility is supported by well-known impossibility results. Charlesworth proved his
Comprehensibility theorem while attempting to formalize the answer to such questions as: “If [full
human-level intelligence] software can exist, could humans understand it?” [56]. While describing
implications of his theorem on AI, he writes [57]: “Comprehensibility Theorem is the first
mathematical theorem implying the impossibility of any AI agent or natural agent—including a
not-necessarily infallible human agent—satisfying a rigorous and deductive interpretation of the
self-comprehensibility challenge. … Self-comprehensibility in some form might be essential for a
kind of self-reflection useful for self-improvement that might enable some agents to increase their
success.” It is reasonable to conclude that a system which doesn’t comprehend itself would not be
able to explain itself.
Hernandez-Orallo et al. introduce the notion of K-incomprehensibility (a.k.a. K-hardness) [58].

“This will be the formal counterpart to our notion of hard-to-learn good explanations. In our sense,
a k-incomprehensible string with a high k (difficult to comprehend) is different (harder) than a k-
compressible string (difficult to learn) [59] and different from classical computational complexity
(slow to compute). Calculating the value of k for a given string is not computable in general.
Fortunately, the converse, i.e., given an arbitrary k, calculating whether a string is k-
comprehensible is computable. … Kolmogorov Complexity measures the amount of information
but not the complexity to understand them.” [58].

Yampolskiy addresses limits of understanding other agents in his work on the space of possible
minds [60]: “Each mind design corresponds to an integer and so is finite, but since the number of
minds is infinite some have a much greater number of states compared to others. This property
holds for all minds. Consequently, since a human mind has only a finite number of possible states,
there are minds which can never be fully understood by a human mind as such mind designs have
a much greater number of states, making their understanding impossible as can be demonstrated
by the pigeonhole principle.” Hibbard points out safety impact from incomprehensibility of AI:
“Given the incomprehensibility of their thoughts, we will not be able to sort out the effect of any
conflicts they have between their own interests and ours.”

On Wednesday, November 2, 2022 at 12:47:28 PM UTC-7, LowRider44M wrote:
> Back In The USSR
> https://youtu.be/nS5_EQgbuLc
>
> Maxwell Silver Hammer
> https://youtu.be/3HuXFfq79I8
>
>
>
>
>
> Unexplainability and Incomprehensibility of Artificial Intelligence
> Roman V. Yampolskiy
>
> Computer Engineering and Computer Science
> University of Louisville
> roman.ya...@louisville.edu, @romanyam
> June 20, 2019
>
> "If a lion could speak, we couldn't understand him"
> Ludwig Wittgenstein
>
> “It would be possible to describe everything scientifically, but it would make no sense. It would
> be a description without meaning - as if you described a Beethoven symphony as a variation of
> wave pressure.”
> Albert Einstein
>
> “Some things in life are too complicated to explain in any language. … Not just to explain to
> others but to explain to yourself. Force yourself to try to explain it and you create lies.”
> Haruki Murakami
>
> “I understand that you don’t understand”
> Grigori Perelman
>
>
> Abstract
> Explainability and comprehensibility of AI are important requirements for intelligent systems
> deployed in real-world domains. Users want and frequently need to understand how decisions
> impacting them are made. Similarly it is important to understand how an intelligent system
> functions for safety and security reasons. In this paper, we describe two complementary
> impossibility results (Unexplainability and Incomprehensibility), essentially showing that
> advanced AIs would not be able to accurately explain some of their decisions and for the decisions
> they could explain people would not understand some of those explanations..
> Keywords: AI Safety, Black Box, Comprehensible, Explainable AI, Impossibility, Intelligible,
> Interpretability, Transparency, Understandable, Unserveyability.
>
> 1. Introduction
> For decades AI projects relied on human expertise, distilled by knowledge engineers, and were
> both explicitly designed and easily understood by people. For example, expert systems, frequently
> based on decision trees, are perfect models of human decision making and so are naturally
> understandable by both developers and end-users. With paradigm shift in the leading AI
> methodology, over the last decade, to machine learning systems based on Deep Neural Networks
> (DNN) this natural ease of understanding got sacrificed. The current systems are seen as “black
> boxes” (not to be confused with AI boxing [1, 2]), opaque to human understanding but extremely
> capable both with respect to results and learning of new domains. As long as Big Data and Huge
> Compute are available, zero human knowledge is required [3] to achieve superhuman [4]
> performance.
>
> With their new found capabilities DNN-based AI systems are tasked with making decisions in
> employment [5], admissions [6], investing [7], matching [8], diversity [9], security [10, 11],
> recommendations [12], banking [13], and countless other critical domains. As many such domains
> are legally regulated, it is a desirable property and frequently a requirement [14, 15] that such
> systems should be able to explain how they arrived at their decisions, particularly to show that they
> are bias free [16]. Additionally, and perhaps even more importantly to make artificially intelligent
> systems safe and secure [17] it is essential that we understand what they are doing and why. A
> particular area of interest in AI Safety [18-25] is predicting and explaining causes of AI failures
> [26].
>
> A significant amount of research [27-41] is now being devoted to developing explainable AI. In
> the next section we review some main results and general trends relevant to this paper.
>
> 2. Literature Review
> Hundreds of papers have been published on eXplainable Artificial Intelligence (XAI) [42].
> According to DARPA [27], XAI is supposed to “produce more explainable models, while
> maintaining a high level of learning performance … and enable human users to understand,
> appropriately, trust, and effectively manage the emerging generation of artificially intelligent
> partners”. Detailed analysis of literature on explainability or comprehensibility is beyond the scope
> of this paper, but the readers are encouraged to look at many excellent surveys of the topic [43-
> 45]. Miller [46] surveys social sciences to understand how people explain, in the hopes of
> transferring that knowledge to XAI, but of course people often say: “I can’t explain it” or “I don’t
> understand”. For example, most people are unable to explain how they recognize faces, a problem
> we frequently ask computers to solve [47, 48].
>
> Despite wealth of publications on XAI and related concepts [49-51], the subject of unexplainability
> or incomprehensibility of AI is only implicitly addressed. Some limitations of explainability are
> discussed: “ML algorithms intrinsically consider high-degree interactions between input features,
> which make disaggregating such functions into human understandable form difficult. … While a
> single linear transformation may be interpreted by looking at the weights from the input features
> to each of the output classes, multiple layers with non-linear interactions at every layer imply
> disentangling a super complicated nested structure which is a difficult task and potentially even a
> questionable one [52]. … As mentioned before, given the complicated structure of ML models,
> for the same set of input variables and prediction targets, complex machine learning algorithms
> can produce multiple accurate models by taking very similar but not the same internal pathway in
> the network, so details of explanations can also change across multiple accurate models. This
> systematic instability makes automated generated explanations difficult.” [42].
>
> Sutcliffe et al. talk about incomprehensible theorems [53]: “Comprehensibility estimates the effort
> required for a user to understand the theorem. Theorems with many or deeply nested structures
> may be considered incomprehensible.” Muggleton et al. [54] suggest “using inspection time as a
> proxy for incomprehension. That is, we might expect that humans take a long time … in the case
> they find the program hard to understand. As a proxy, inspection time is easier to measure than
> comprehension.”
>
> The tradeoff between explainability and comprehensibility is recognized [52], but is not taken to
> its logical conclusion. “[A]ccuracy generally requires more complex prediction methods [but]
> simple and interpretable functions do not make the most accurate predictors'' [55]. “Indeed, there
> are algorithms that are more interpretable than others are, and there is often a tradeoff between
> accuracy and interpretability: the most accurate AI/ML models usually are not very explainable
> (for example, deep neural nets, boosted trees, random forests, and support vector machines), and
> the most interpretable models usually are less accurate (for example, linear or logistic regression).”
> [42].
>
> Incomprehensibility is supported by well-known impossibility results. Charlesworth proved his
> Comprehensibility theorem while attempting to formalize the answer to such questions as: “If [full
> human-level intelligence] software can exist, could humans understand it?” [56]. While describing
> implications of his theorem on AI, he writes [57]: “Comprehensibility Theorem is the first
> mathematical theorem implying the impossibility of any AI agent or natural agent—including a
> not-necessarily infallible human agent—satisfying a rigorous and deductive interpretation of the
> self-comprehensibility challenge. … Self-comprehensibility in some form might be essential for a
> kind of self-reflection useful for self-improvement that might enable some agents to increase their
> success.” It is reasonable to conclude that a system which doesn’t comprehend itself would not be
> able to explain itself.
> Hernandez-Orallo et al. introduce the notion of K-incomprehensibility (a.k.a. K-hardness) [58].
>
> “This will be the formal counterpart to our notion of hard-to-learn good explanations. In our sense,
> a k-incomprehensible string with a high k (difficult to comprehend) is different (harder) than a k-
> compressible string (difficult to learn) [59] and different from classical computational complexity
> (slow to compute). Calculating the value of k for a given string is not computable in general.
> Fortunately, the converse, i.e., given an arbitrary k, calculating whether a string is k-
> comprehensible is computable. … Kolmogorov Complexity measures the amount of information
> but not the complexity to understand them.” [58].
>
> Yampolskiy addresses limits of understanding other agents in his work on the space of possible
> minds [60]: “Each mind design corresponds to an integer and so is finite, but since the number of
> minds is infinite some have a much greater number of states compared to others. This property
> holds for all minds. Consequently, since a human mind has only a finite number of possible states,
> there are minds which can never be fully understood by a human mind as such mind designs have
> a much greater number of states, making their understanding impossible as can be demonstrated
> by the pigeonhole principle.” Hibbard points out safety impact from incomprehensibility of AI:
> “Given the incomprehensibility of their thoughts, we will not be able to sort out the effect of any
> conflicts they have between their own interests and ours.”
>
> We are slowly starting to realize that as AIs become more powerful, the models behind their
> success will become ever less comprehensible to us [61]: “… deep learning that produces
> outcomes based on so many different variables under so many different conditions being
> transformed by so many layers of neural networks that humans simply cannot comprehend the
> model the computer has built for itself. … Clearly our computers have surpassed us in their power
> to discriminate, find patterns, and draw conclusions. That’s one reason we use them. Rather than
> reducing phenomena to fit a relatively simple model, we can now let our computers make models
> as big as they need to. But this also seems to mean that what we know depends upon the output of
> machines the functioning of which we cannot follow, explain, or understand. … But some of the
> new models are incomprehensible. They can exist only in the weights of countless digital triggers
> networked together and feeding successive layers of networked, weighted triggers representing
> huge quantities of variables that affect one another in ways so particular that we cannot derive
> general principles from them.”
>
> “Now our machines are letting us see that even if the rules are simple, elegant, beautiful and
> rational, the domain they govern is so granular, so intricate, so interrelated, with everything
> causing everything else all at once and forever, that our brains and our knowledge cannot begin to
> comprehend it. … Our new reliance on inscrutable models as the source of the justification of our
> beliefs puts us in an odd position. If knowledge includes the justification of our beliefs, then
> knowledge cannot be a class of mental content, because the justification now consists of models
> that exist in machines, models that human mentality cannot comprehend. … But the promise of
> machine learning is that there are times when the machine’s inscrutable models will be far more
> predictive than the manually constructed, human-intelligible ones. In those cases, our
> knowledge — if we choose to use it — will depend on justifications that we simply cannot
> understand. … [W]e are likely to continue to rely ever more heavily on justifications that we
> simply cannot fathom. And the issue is not simply that we cannot fathom them, the way a lay
> person can’t fathom a string theorist’s ideas. Rather, it’s that the nature of computer-based
> justification is not at all like human justification. It is alien.” [61].
>
> 3. Unexplainability
> A number of impossibility results are well-known in many areas of research [62-70] and some are
> starting to be discovered in the domain of AI research, for example: Unverifiability [71],
> Unpredictability1 [72] and limits on preference deduction [73] or alignment [74]. In this section
> we introduce Unexplainability of AI and show that some decisions of superintelligent systems will
> never be explainable, even in principle. We will concentrate on the most interesting case, a
> superintelligent AI acting in novel and unrestricted domains. Simple cases of Narrow AIs making
> decisions in restricted domains (Ex. Tic-Tac-Toe) are both explainable and comprehensible.
> Consequently a whole spectrum of AIs can be developed from completely
> explainable/comprehensible to completely unexplainable/incomprehensible. We define
> Unexplainability as impossibility of providing an explanation for certain decisions made by an
> intelligent system which is both 100% accurate and comprehensible.
>
>
> 1 Unpredictability is not the same as Unexplainability or Incomprehensibility, see ref. 72. Yampolskiy, R.V.,
> Unpredictability of AI. arXiv preprint arXiv:1905.13053, 2019. for details.
> Artificial Deep Neural Networks continue increasing in size and may already comprise millions
> of neurons, thousands of layers and billions of connecting weights, ultimately targeting and
> perhaps surpassing the size of the human brain. They are trained on Big Data from which million
> feature vectors are extracted and on which decisions are based, with each feature contributing to
> the decision in proportion to a set of weights. To explain such a decision, which relies on literally
> billions of contributing factors, AI has to either simplify the explanation and so make the
> explanation less accurate/specific/detailed or to report it exactly but such an explanation elucidates
> nothing by virtue of its semantic complexity, large size and abstract data representation. Such
> precise reporting is just a copy of trained DNN model.
>
> For example, an AI utilized in the mortgage industry may look at an application to decide credit
> worthiness of a person in order to approve them for a loan. For simplicity, let’s say the system
> looks at only a hundred descriptors of the applicant and utilizes a neural network to arrive at a
> binary approval decision. An explanation which included all hundred features and weights of the
> neural network would not be very useful, so the system may instead select one of two most
> important features and explain its decision with respect to just those top properties, ignoring the
> rest. This highly simplified explanation would not be accurate as the other 98 features all
> contributed to the decision and if only one or two top features were considered the decision could
> have been different. This is similar to how Principal Component Analysis works for dimensionality
> reduction [75].
>
> Even if the agent trying to get the explanation is not a human but another AI the problem remains
> as the explanation is either inaccurate or agent-encoding specific. Trained model could be copied
> to another neural network, but it would likewise have a har
> 104. Yampolskiy, R.V. and M. Spellchecker, Artificial Intelligence Safety and Cybersecurity: a
> Timeline of AI Failures. arXiv preprint arXiv:1610.07997, 2016.
hot rod charlie runs tomorrow
last race of his career

On Friday, November 4, 2022 at 4:57:53 PM UTC-7, chris rodgers wrote:
> On Wednesday, November 2, 2022 at 12:47:28 PM UTC-7, LowRider44M wrote:
> > Back In The USSR
> > https://youtu.be/nS5_EQgbuLc
> >
> > Maxwell Silver Hammer
> > https://youtu.be/3HuXFfq79I8
> >
> >
> >
> >
> >
> > Unexplainability and Incomprehensibility of Artificial Intelligence
> > Roman V. Yampolskiy
> >
> > Computer Engineering and Computer Science
> > University of Louisville
> > roman.ya...@louisville.edu, @romanyam
> > June 20, 2019
> >
> > "If a lion could speak, we couldn't understand him"
> > Ludwig Wittgenstein
> >
> > “It would be possible to describe everything scientifically, but it would make no sense. It would
> > be a description without meaning - as if you described a Beethoven symphony as a variation of
> > wave pressure.”
> > Albert Einstein
> >
> > “Some things in life are too complicated to explain in any language. … Not just to explain to
> > others but to explain to yourself. Force yourself to try to explain it and you create lies.”
> > Haruki Murakami
> >
> > “I understand that you don’t understand”
> > Grigori Perelman
> >
> >
> > Abstract
> > Explainability and comprehensibility of AI are important requirements for intelligent systems
> > deployed in real-world domains. Users want and frequently need to understand how decisions
> > impacting them are made. Similarly it is important to understand how an intelligent system
> > functions for safety and security reasons. In this paper, we describe two complementary
> > impossibility results (Unexplainability and Incomprehensibility), essentially showing that
> > advanced AIs would not be able to accurately explain some of their decisions and for the decisions
> > they could explain people would not understand some of those explanations.
> > Keywords: AI Safety, Black Box, Comprehensible, Explainable AI, Impossibility, Intelligible,
> > Interpretability, Transparency, Understandable, Unserveyability.
> >
> > 1. Introduction
> > For decades AI projects relied on human expertise, distilled by knowledge engineers, and were
> > both explicitly designed and easily understood by people. For example, expert systems, frequently
> > based on decision trees, are perfect models of human decision making and so are naturally
> > understandable by both developers and end-users. With paradigm shift in the leading AI
> > methodology, over the last decade, to machine learning systems based on Deep Neural Networks
> > (DNN) this natural ease of understanding got sacrificed. The current systems are seen as “black
> > boxes” (not to be confused with AI boxing [1, 2]), opaque to human understanding but extremely
> > capable both with respect to results and learning of new domains. As long as Big Data and Huge
> > Compute are available, zero human knowledge is required [3] to achieve superhuman [4]
> > performance.
> >
> > With their new found capabilities DNN-based AI systems are tasked with making decisions in
> > employment [5], admissions [6], investing [7], matching [8], diversity [9], security [10, 11],
> > recommendations [12], banking [13], and countless other critical domains. As many such domains
> > are legally regulated, it is a desirable property and frequently a requirement [14, 15] that such
> > systems should be able to explain how they arrived at their decisions, particularly to show that they
> > are bias free [16]. Additionally, and perhaps even more importantly to make artificially intelligent
> > systems safe and secure [17] it is essential that we understand what they are doing and why. A
> > particular area of interest in AI Safety [18-25] is predicting and explaining causes of AI failures
> > [26].
> >
> > A significant amount of research [27-41] is now being devoted to developing explainable AI. In
> > the next section we review some main results and general trends relevant to this paper.
> >
> > 2. Literature Review
> > Hundreds of papers have been published on eXplainable Artificial Intelligence (XAI) [42].
> > According to DARPA [27], XAI is supposed to “produce more explainable models, while
> > maintaining a high level of learning performance … and enable human users to understand,
> > appropriately, trust, and effectively manage the emerging generation of artificially intelligent
> > partners”. Detailed analysis of literature on explainability or comprehensibility is beyond the scope
> > of this paper, but the readers are encouraged to look at many excellent surveys of the topic [43-
> > 45]. Miller [46] surveys social sciences to understand how people explain, in the hopes of
> > transferring that knowledge to XAI, but of course people often say: “I can’t explain it” or “I don’t
> > understand”. For example, most people are unable to explain how they recognize faces, a problem
> > we frequently ask computers to solve [47, 48].
> >
> > Despite wealth of publications on XAI and related concepts [49-51], the subject of unexplainability
> > or incomprehensibility of AI is only implicitly addressed. Some limitations of explainability are
> > discussed: “ML algorithms intrinsically consider high-degree interactions between input features,
> > which make disaggregating such functions into human understandable form difficult. … While a
> > single linear transformation may be interpreted by looking at the weights from the input features
> > to each of the output classes, multiple layers with non-linear interactions at every layer imply
> > disentangling a super complicated nested structure which is a difficult task and potentially even a
> > questionable one [52]. … As mentioned before, given the complicated structure of ML models,
> > for the same set of input variables and prediction targets, complex machine learning algorithms
> > can produce multiple accurate models by taking very similar but not the same internal pathway in
> > the network, so details of explanations can also change across multiple accurate models. This
> > systematic instability makes automated generated explanations difficult..” [42].
> >
> > Sutcliffe et al. talk about incomprehensible theorems [53]: “Comprehensibility estimates the effort
> > required for a user to understand the theorem. Theorems with many or deeply nested structures
> > may be considered incomprehensible.” Muggleton et al. [54] suggest “using inspection time as a
> > proxy for incomprehension. That is, we might expect that humans take a long time … in the case
> > they find the program hard to understand. As a proxy, inspection time is easier to measure than
> > comprehension.”
> >
> > The tradeoff between explainability and comprehensibility is recognized [52], but is not taken to
> > its logical conclusion. “[A]ccuracy generally requires more complex prediction methods [but]
> > simple and interpretable functions do not make the most accurate predictors'' [55]. “Indeed, there
> > are algorithms that are more interpretable than others are, and there is often a tradeoff between
> > accuracy and interpretability: the most accurate AI/ML models usually are not very explainable
> > (for example, deep neural nets, boosted trees, random forests, and support vector machines), and
> > the most interpretable models usually are less accurate (for example, linear or logistic regression).”
> > [42].
> >
> > Incomprehensibility is supported by well-known impossibility results. Charlesworth proved his
> > Comprehensibility theorem while attempting to formalize the answer to such questions as: “If [full
> > human-level intelligence] software can exist, could humans understand it?” [56]. While describing
> > implications of his theorem on AI, he writes [57]: “Comprehensibility Theorem is the first
> > mathematical theorem implying the impossibility of any AI agent or natural agent—including a
> > not-necessarily infallible human agent—satisfying a rigorous and deductive interpretation of the
> > self-comprehensibility challenge. … Self-comprehensibility in some form might be essential for a
> > kind of self-reflection useful for self-improvement that might enable some agents to increase their
> > success.” It is reasonable to conclude that a system which doesn’t comprehend itself would not be
> > able to explain itself.
> > Hernandez-Orallo et al. introduce the notion of K-incomprehensibility (a.k.a. K-hardness) [58].
> >
> > “This will be the formal counterpart to our notion of hard-to-learn good explanations. In our sense,
> > a k-incomprehensible string with a high k (difficult to comprehend) is different (harder) than a k-
> > compressible string (difficult to learn) [59] and different from classical computational complexity
> > (slow to compute). Calculating the value of k for a given string is not computable in general.
> > Fortunately, the converse, i.e., given an arbitrary k, calculating whether a string is k-
> > comprehensible is computable. … Kolmogorov Complexity measures the amount of information
> > but not the complexity to understand them.” [58].
> >
> > Yampolskiy addresses limits of understanding other agents in his work on the space of possible
> > minds [60]: “Each mind design corresponds to an integer and so is finite, but since the number of
> > minds is infinite some have a much greater number of states compared to others. This property
> > holds for all minds. Consequently, since a human mind has only a finite number of possible states,
> > there are minds which can never be fully understood by a human mind as such mind designs have
> > a much greater number of states, making their understanding impossible as can be demonstrated
> > by the pigeonhole principle.” Hibbard points out safety impact from incomprehensibility of AI:
> > “Given the incomprehensibility of their thoughts, we will not be able to sort out the effect of any
> > conflicts they have between their own interests and ours.”
> >
> > We are slowly starting to realize that as AIs become more powerful, the models behind their
> > success will become ever less comprehensible to us [61]: “… deep learning that produces
> > outcomes based on so many different variables under so many different conditions being
> > transformed by so many layers of neural networks that humans simply cannot comprehend the
> > model the computer has built for itself. … Clearly our computers have surpassed us in their power
> > to discriminate, find patterns, and draw conclusions. That’s one reason we use them. Rather than
> > reducing phenomena to fit a relatively simple model, we can now let our computers make models
> > as big as they need to. But this also seems to mean that what we know depends upon the output of
> > machines the functioning of which we cannot follow, explain, or understand. … But some of the
> > new models are incomprehensible. They can exist only in the weights of countless digital triggers
> > networked together and feeding successive layers of networked, weighted triggers representing
> > huge quantities of variables that affect one another in ways so particular that we cannot derive
> > general principles from them.”
> >
> > “Now our machines are letting us see that even if the rules are simple, elegant, beautiful and
> > rational, the domain they govern is so granular, so intricate, so interrelated, with everything
> > causing everything else all at once and forever, that our brains and our knowledge cannot begin to
> > comprehend it. … Our new reliance on inscrutable models as the source of the justification of our
> > beliefs puts us in an odd position. If knowledge includes the justification of our beliefs, then
> > knowledge cannot be a class of mental content, because the justification now consists of models
> > that exist in machines, models that human mentality cannot comprehend. … But the promise of
> > machine learning is that there are times when the machine’s inscrutable models will be far more
> > predictive than the manually constructed, human-intelligible ones. In those cases, our
> > knowledge — if we choose to use it — will depend on justifications that we simply cannot
> > understand. … [W]e are likely to continue to rely ever more heavily on justifications that we
> > simply cannot fathom. And the issue is not simply that we cannot fathom them, the way a lay
> > person can’t fathom a string theorist’s ideas. Rather, it’s that the nature of computer-based
> > justification is not at all like human justification. It is alien.” [61].
> >
> > 3. Unexplainability
> > A number of impossibility results are well-known in many areas of research [62-70] and some are
> > starting to be discovered in the domain of AI research, for example: Unverifiability [71],
> > Unpredictability1 [72] and limits on preference deduction [73] or alignment [74]. In this section
> > we introduce Unexplainability of AI and show that some decisions of superintelligent systems will
> > never be explainable, even in principle. We will concentrate on the most interesting case, a
> > superintelligent AI acting in novel and unrestricted domains. Simple cases of Narrow AIs making
> > decisions in restricted domains (Ex. Tic-Tac-Toe) are both explainable and comprehensible.
> > Consequently a whole spectrum of AIs can be developed from completely
> > explainable/comprehensible to completely unexplainable/incomprehensible.. We define
> > Unexplainability as impossibility of providing an explanation for certain decisions made by an
> > intelligent system which is both 100% accurate and comprehensible.
> >
> >
> > 1 Unpredictability is not the same as Unexplainability or Incomprehensibility, see ref. 72. Yampolskiy, R.V.,
> > Unpredictability of AI. arXiv preprint arXiv:1905.13053, 2019. for details.
> > Artificial Deep Neural Networks continue increasing in size and may already comprise millions
> > of neurons, thousands of layers and billions of connecting weights, ultimately targeting and
> > perhaps surpassing the size of the human brain. They are trained on Big Data from which million
> > feature vectors are extracted and on which decisions are based, with each feature contributing to
> > the decision in proportion to a set of weights. To explain such a decision, which relies on literally
> > billions of contributing factors, AI has to either simplify the explanation and so make the
> > explanation less accurate/specific/detailed or to report it exactly but such an explanation elucidates
> > nothing by virtue of its semantic complexity, large size and abstract data representation. Such
> > precise reporting is just a copy of trained DNN model.
> >
> > For example, an AI utilized in the mortgage industry may look at an application to decide credit
> > worthiness of a person in order to approve them for a loan. For simplicity, let’s say the system
> > looks at only a hundred descriptors of the applicant and utilizes a neural network to arrive at a
> > binary approval decision. An explanation which included all hundred features and weights of the
> > neural network would not be very useful, so the system may instead select one of two most
> > important features and explain its decision with respect to just those top properties, ignoring the
> > rest. This highly simplified explanation would not be accurate as the other 98 features all
> > contributed to the decision and if only one or two top features were considered the decision could
> > have been different. This is similar to how Principal Component Analysis works for dimensionality
> > reduction [75].
> >
> > Even if the agent trying to get the explanation is not a human but another AI the problem remains
> > as the explanation is either inaccurate or agent-encoding specific. Trained model could be copied
> > to another neural network, but it would likewise have a har
> > 104. Yampolskiy, R.V. and M. Spellchecker, Artificial Intelligence Safety and Cybersecurity: a
> > Timeline of AI Failures. arXiv preprint arXiv:1610.07997, 2016.
> hot rod charlie runs tomorrow
> last race of his career
>
> and joe biden should tap out
> and Harris? she can sell hot dogs in front of the White House

> > hot rod charlie runs tomorrow
> > last race of his career
> >
> > and joe biden should tap out
> > and Harris? she can sell hot dogs in front of the White House
> poor hot rod charlie got beat yesterday.
> didn't even finish 'in the money'. oh well.
> they had some fast horses in that race.

I'd bet a 20 on this dog!
https://twitter.com/i/status/1589242912109301760