The problem stems from the desire to define ethics. What is ethics? From my point of view, ethics is a set of rules describing how ethical is a given behaviour in a given situation. The definition might seem somewhat circular, but lets put this aside. The important aspect is that ethics is something allowing comparison of different choices and marking some as "better" and some as "worse", doesn't matter in what sense.

In order to discuss ethics, we must understand the set of choices available to a given person at a given situation. The fact that choice exists at all relies on the notion of "free will" which, on the surface, contradicts determinism.

What is determinism? Determinism means that given complete knowledge of the initial conditions it is possible to predict what would happen at any time in the future. This is a principle that holds in classical physics. The real world is better described by quantum physics, in which the situation is more complicated, but lets leave this for later.

In practice, such predictions are limited by 3 factors:

In practice, such predictions are limited by 3 factors:

- Incomplete knowledge of the initial conditions. Indeed it is difficult to know everything about the entire universe.
- Incomplete knowledge of the laws of nature. Our model of reality is imperfect, and in my opinion, will always remain so.
- Limited information processing power. Even if you know the initial conditions and the laws of nature sufficiently precisely to give a prediction with the accuracy you need, doing so might require a complicated computation which would take lots of "CPU clock ticks" and perhaps lots of "RAM" to complete.

*impossible*for X to know it. That is, we have the following

*principle of self-unpredictability*:

An intelligent being X can never have a future-prediction capacity, accounting for the 3 factors above, that would allow it to predict her own behaviour.

If the principle is violated, we would get something like the grandfather paradox: if X knows she is going to do Y, what prevents her from doing Z which is different from Y? The principle also makes sense physically, as far as my intuition goes: to produce better predictions we need a more powerful "brain" which would have to be more complicated and thus more difficult to predict. It's sort of a "bootstrap".

To give an example, suppose X is a human. Certainly X is not able to use her knowledge of biology, chemistry, quantum physics and what else to predict the workings of her own brain. Now, suppose X recruits a computer Y to help her. It might appear that now her task is more realistic. However, by using the computer she made it

*a part of the system*. That is, the required prediction now involves the joint dynamics of X + Y and thus remains out of reach.It is fascinating to me whether the self-unpredictability principle can be reformulated as a theorem in physics about abstract information-processing systems.

Thus, the space of possible choices of X can be defined to be the space of things X can do

*as far as X can tell*. Ethics would have to operate on this space.It might appear that the indeterminism of quantum mechanics provides some kind of an alternative solution to the problem. However, I deem it is not so. The "freedom" allowed for by quantum mechanics is pure random. It is not consistent with the sort of choice that involves ethical judgement, which is the sort of choice we are targeting here.

Moreover, consider a person X observed by a superintelligent being Y. Y can predict X's behaviour up to quantum indeterminism. Y knows X is going to A with propability

pA = 1 - 1e-100 and B with probability pB = 1e-100. Thus B is a highly unlikely choice. However, from the point of view of X both of the choices might be equally legitimate, a priori (before ethical judgement is applied). Moreover, the small probability pB might stem from something like a lightning striking X's head, which is an artifact completely irrelevant to the ethical dilemma at hand.

Moreover, consider a person X observed by a superintelligent being Y. Y can predict X's behaviour up to quantum indeterminism. Y knows X is going to A with propability

pA = 1 - 1e-100 and B with probability pB = 1e-100. Thus B is a highly unlikely choice. However, from the point of view of X both of the choices might be equally legitimate, a priori (before ethical judgement is applied). Moreover, the small probability pB might stem from something like a lightning striking X's head, which is an artifact completely irrelevant to the ethical dilemma at hand.

## 9 comments:

Thanks. I hadn't been able to clearly formulate what bothers me.

That sounds very like what we discuss in our philosophers club :)

A few notes:

1) I'm not a follower, but Penrose, in his "Emperor" etc plays with the idea that quantum collapse is not a probabilistic but rather a non computational feature, in the sense, that when a brain collapses to a classical state (on some level or another, leave the quantumness of the brain aside for the moment), it can solve unsolvable problems, like the halt problem. In this view, our decisions are essentially unpredictable. You can possibly simulate them on a machine using the same quantum twist, but as Penrose connects it to the consciousness itself, that wouldn't be a simulation, that will be you making the decision once again.

2) Regarding the X girl using the Y computer. The computer doesn't have to simulate itself. It can simulate the girl, given the answers it provides, and see if the girls' decisions diverse. There's a good possibility, that if will be able to predict, for every practical use, the decision made. e.g it says "A", girls says "what the hell, I chose B", then it says "ok, B", the girl says "f* machine, I'd just do what _I_ want" and chooses A.

3) I think that ethics is not well defined on that microscopical level. It is a generalized coordinate, like pressure or temperature. It's really funny to notice, that ethics is best understood in quasistatical situations, just as thermodynamical variables. In that terminology, you take a "situation" , composed of two coupled systems "me" and "world", where world is the ethics bath, and the system "me" thrives to get into states with equal ethics potential with the world. That's all good in the subjective perception of ethics, but can we agree on a uniform "ethics scale"? There's an obvious empirical evidence of ethics existence.

4) I do agree that "ethical, as far as X can tell" is a good definition to avoid caveats.

1. I never understood the claim that people are better than computers because people supposedly can solve the halting problem. In my opinion, people cannot solve the halting problem. The illusion that they can stems from thinking about very simple halting problems.

Consider the following halting problem. The Turing machine T, loops over all even numbers 2k for k > 1 and checks whether 2k is a sum of two prime numbers. If it is, it goes on, if it isn't, it halts. Does it ever halt or not? This is an open problem: http://en.wikipedia.org/wiki/Goldbach's_conjecture.

Moreover, it is in principle possible that this cannot be either proved or disproved. Even if this particular statement can be either proved or disproved, it is possible to show that a there is a statement S(n) checkable for every natural n exists which cannot be either proved or disproved. In fact it follows from the halting problem! Indeed, consider statements of the form:

S_T(n) = "The Turing machine T halts after n steps".

Now consider the Turing machine P whose input is a Turing machine T. What P does is a brute-force search in the space of formal logical deductions of either a proof that T halts or a proof that T doesn't halt. Since P cannot be the solution to the halting problem (there is no such solution), it follows that T exists such that is impossible to either prove or disprove it halts.

Moreover, all known physics including superstring theory is computable, as far as I can tell. So, with all due respect to Penrose, it sounds like a lot of wishful thinking.

2. To set the rules of the game, lets suppose the only output allowed from Y is the predicted decision. If we allow scenarios like yours, the notion of "prediction" becomes ill-defined since Y provided several conflicting predictions. Now, Y can compute the function D(P) where D is the decision X and P is the prediction provided by Y. It can then search for roots of the equation

D(P) = P

Such roots may or may not exist, as far as I can tell.

3. Hmm, I don't quite follow you. What is the "ethics potential" and why does it equilibrates?

1)Agree with you here, I do not see an overwhelming proof either. ( = I

'm not a follower )

Penrose doesn't rely on superstrings, he develops an "objective reduction" concept based on quantum gravity. I'm not sure I want to get into the details now, but supposing it is possible, there may be new non-computable horizons for the consciousness.

2) D(P)=P may very well be solvable for any practical purpose, and then the determinism will celebrate. However, will it discard ethics? I think, still the answer will be "NO", as the external computer Y is an oracle beyond X's understand.

It's like another external oracle, named muezzin, doesn't discard ethics, IMHO. So what I say is that is ethics only about something you "understand", not something you "believe".

3)"Ethics" is some sort of macroscopic (subjective) variable, which describes how "ethical" is (state,action) in the given subspace of known state and available actions.

So far so good?

Btw, thx for your commments!

1. It is very interesting to me how can one construct a reasonable

non-computable model.

3. Ok, maybe now I get what you're saying. You're saying that a person X immersed in an environment E is influenced by the kind of ethics E has and acquires it for herself.

Also, ethics is "macroscopic" in the sense that people have ethics (and perhaps, societies have ethics?) but electrons don't have ethics.

Sorry for disappearing, busy week.

1) Basically, every non-computable model is in particular a computable model, with non-computable elements. So the question is not "how" but "and what". For example, add to a Turing machine, a zero-time halt problem oracle. What could you get from it? This particular example is bad - the answer is that you get instantaneous answers for all your problems.

But maybe, with some strong constraints, we manage to stay in the real world, but add "god","belief","intuition" to our vocabulary.

2) correct

1. This I know. However, does Penrose (or anyone) has an example of at least remotely reaslitic

non-computable physics? Stuff like halt problem oracles appears difficult to insert into feasible mathematical models of reality.

Sorry, didn't get that deep into it. But I know a right person to ask.

That’s Too nice, when it comes in india hope it can make a Rocking place for youngster.. hope that come true.

Post a Comment