Free will, ethics and determinism

Lev has recently brought up the question of free will vs. determinism. I have spent some time thinking about these issues and came up with the ideas I want to lay out here.

The problem stems from the desire to define ethics. What is ethics? From my point of view, ethics is a set of rules describing how ethical is a given behaviour in a given situation. The definition might seem somewhat circular, but lets put this aside. The important aspect is that ethics is something allowing comparison of different choices and marking some as "better" and some as "worse", doesn't matter in what sense.

In order to discuss ethics, we must understand the set of choices available to a given person at a given situation. The fact that choice exists at all relies on the notion of "free will" which, on the surface, contradicts determinism.

What is determinism? Determinism means that given complete knowledge of the initial conditions it is possible to predict what would happen at any time in the future. This is a principle that holds in classical physics. The real world is better described by quantum physics, in which the situation is more complicated, but lets leave this for later.
In practice, such predictions are limited by 3 factors:

1. Incomplete knowledge of the initial conditions. Indeed it is difficult to know everything about the entire universe.
2. Incomplete knowledge of the laws of nature. Our model of reality is imperfect, and in my opinion, will always remain so.
3. Limited information processing power. Even if you know the initial conditions and the laws of nature sufficiently precisely to give a prediction with the accuracy you need, doing so might require a complicated computation which would take lots of "CPU clock ticks" and perhaps lots of "RAM" to complete.

Thus, even if a "bird's-view" observer knows precisely what person X is going to do, X doesn't know it. Moreover, I claim it is impossible for X to know it. That is, we have the following principle of self-unpredictability:

An intelligent being X can never have a future-prediction capacity, accounting for the 3 factors above, that would allow it to predict her own behaviour.

If the principle is violated, we would get something like the grandfather paradox: if X knows she is going to do Y, what prevents her from doing Z which is different from Y? The principle also makes sense physically, as far as my intuition goes: to produce better predictions we need a more powerful "brain" which would have to be more complicated and thus more difficult to predict. It's sort of a "bootstrap".

To give an example, suppose X is a human. Certainly X is not able to use her knowledge of biology, chemistry, quantum physics and what else to predict the workings of her own brain. Now, suppose X recruits a computer Y to help her. It might appear that now her task is more realistic. However, by using the computer she made it a part of the system. That is, the required prediction now involves the joint dynamics of X + Y and thus remains out of reach.

It is fascinating to me whether the self-unpredictability principle can be reformulated as a theorem in physics about abstract information-processing systems.

Thus, the space of possible choices of X can be defined to be the space of things X can do as far as X can tell. Ethics would have to operate on this space.

It might appear that the indeterminism of quantum mechanics provides some kind of an alternative solution to the problem. However, I deem it is not so. The "freedom" allowed for by quantum mechanics is pure random. It is not consistent with the sort of choice that involves ethical judgement, which is the sort of choice we are targeting here.
Moreover, consider a person X observed by a superintelligent being Y. Y can predict X's behaviour up to quantum indeterminism. Y knows X is going to A with propability
pA = 1 - 1e-100 and B with probability pB = 1e-100. Thus B is a highly unlikely choice. However, from the point of view of X both of the choices might be equally legitimate, a priori (before ethical judgement is applied). Moreover, the small probability pB might stem from something like a lightning striking X's head, which is an artifact completely irrelevant to the ethical dilemma at hand.