
Goal content (p. 272-274) What objective should the ai pursue? How should a description of this objective be interpreted? Should the objective include giving special rewards to those who contributed to the projects success?
Decision theory (p. 274-275) Should the ai use causal decision theory, evidential decision theory, update less decision theory or sth else? direct vs. indirect decision theory (eg. "that decision theory d which we would have wanted it to use had we thought long and hard about the matter." using d` as an interim decision throw on its search for d. seed ai might be endowed with a form of restricted rationality, a simplified decision theory, ignoring esoteric considerations and designed to replace itself with a more advanced form of theory (indirect) once certain criteria are met.
Epistemology (p. 275-276) What should the ais prior probability function be and what other explicit or implicit assumptions about the world should it make? What theory of anthropics should it use? "the point of superintelligence is not to pander to human preconceptions but to make mincemeat out of our ignorance and folly."
Ratification (p. 277-278) Should the ais plan be subject to human review before being put into effect? If so what is the protocol for that review process?