AI Control Problem

Transcript

1. AI Control Problem David Evans University of Virginia aipavilion.github.io AI

   Pavilion 29 October 2018

2. Schedule Reminders “Final” Papers are Due 4:59pm Thursday, Nov 1

   Email [email protected], subject line: [AI Pavilion] Paper Title Include in the email body: 1. What is the purpose of your paper? (one sentence answer) 2. Who is your intended audience? (one sentence answer) 3. If you decided not to follow advice from the first draft, explain why. (It is okay to not follow advice, but you need to make it clear that you understood the advice and justify why you didn’t follow it.) 4. Do you want to continue with this topic, or start on a new topic for the “final” paper? (Yes/no answer is fine, but feel free to explain more if helpful) 1

3. Next Class: Short Presentation You should prepare a short presentation

   (no more than 5 minutes) about your paper 2

4. Crash Course in Machine Learning 3

5. Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious

   / Benign Operational Data Trained Classifier Training (supervised learning) Statistical Machine Learning

6. Learning a Function 5 !: ℝ$ ⟶ {1, 2, …

   , +} !: ℝ$ ⟶ ℝ !: ℤ$ ⟶ ℤ$

7. Learning a Function 6 !: ℝ$ ⟶ {1, 2, …

   , +} !: ℝ$ ⟶ ℝ !: ℤ$ ⟶ ℤ$ Classifier: ⟶ “panda” Regression: Profile ⟶ risk of default Translation:

8. How to Learn Functions Linear Regression: learn a function of

   type: 7 ! = #$ %$ + #' %' + … + #) %)

9. How to Learn Functions Linear Regression: learn a function of

   type: 8 ! " = $% &% + $( &( + … + $* &* Training: given set of labeled points, { ,% , "% , ,( , "( , … , ,. , ". } Find the values for weights 0 that minimize prediction error: Mean squared error: 123 = % . (∑ 6 7 ("6 − "6 )(

10. How well does this work? Generalization: easy to learn a

    function that predicts the training data a simple lookup table does perfectly! goal is to find a function that generalizes: produces correct prediction for unseen inputs Capacity: ability to learn a large variety of functions linear regression can only learn linear functions too high capacity: overfits training data (poor generalization) 9

11. Perceptron 10 !" !# !$ … % = '(!" )"

    + !# )# + … + !$ )$ )

12. How powerful is a perceptron? 11 !" !# !$ …

    % = '(!" )" + !# )# + … + !$ )$ )

13. Trick-or-Treat Protocols 12

14. “Trick or Treat” 13 !"#$% ∨ !"'() Tricker initiates the

    protocol by making a threat and demanding tribute Victim either pays tribute (usually in the form of sugary snack) or risks being tricked

15. Illogical Threat 14 !"#$% ∨ !"'()

16. “Trick xor Treat” 15 !"#$% ⊕ !"'() Tricker initiates the

    protocol by making a threat and demanding tribute Victim either pays tribute (usually in the form of sugary snack) or risks being tricked Tricker must convince Victim that she poses a credible threat: prove she is a qualified tricker

17. Trick-or-Treat Trickers? “Trick xor Treat?” “Prove it!” “The magic word

    is: squamish ossifrage” Victim 16 Any problems with this?

18. Proof without Disclosure How can the tricker prove their trickability,

    without allowing the victim to now impersonate a tricker? 17

19. Challenge-Response Protocol 18 Prover: proves knowledge of ! by revealing

    "(!, %) . Verifier: convinces prover knows !, but learns nothing useful about !. Verifier: picks random %. Need a one-way function: hard to invert, but easy to compute.

20. Example: RSA 19 E e (M ) = Me mod

    n D d (C ) = Cd mod n Correctness property: E e (D d (.)) = .

21. Trick-or-Treat Trickers? “Trick xor Treat?” “Prove it! Challenge = !”

    Response: " = $ % ! = !%mod ) Victim 20

22. Trick-or-Treat Trickers? “Trick xor Treat?” “Prove it! Challenge = !”

    Response: " = $ % ! = !%mod ) Victim 21 How does victim know e and n? Verify: *+ " = "+mod ) = !

23. “Trick xor Treat?” “What is your Tricker ID?” 22 “Elsa

    #253224”, ! = 3482..., " = 1234... signed by Tricker’s Buroo “Prove it! Challenge = #” Response: $ = & ' # = #'mod " Verify: +, $ = $,mod " = # Verify Tricker’s Buroo signature on certificate

24. “Trick xor Treat?” “Hello" 23 “virginia.edu”, ! = … "

    = ... signed by Certificate Authority “Prove it! Decrypt E$ (&) channel encrypted using & Verify and Decrypt: () *+ (&) = & Verify signature on certificate Server

25. 24

26. How powerful is a perceptron? 25 !" !# !$ …

    % = '(!" )" + !# )# + … + !$ )$ )

27. Deep Learning 26 Connect multiple layers of perceptrons In theory,

    one hidden layer is enough to match any training set In practice, more layers often works better

28. 27 Inception v3 23M parameters,

29. Training a DNN • Loss function: measure of how close

    output are to desired outputs • Backpropagation: update weights throughout network to minimize loss function 28 When Bostom talks about reward functions, this is what it means (for todays ML)

30. Some examples… 29 https://www.youtube.com/watch?v=GdTBqBnqhaQ

31. GANs 30 Generative Adversarial Networks [Ian Goodfellow, et al. 2014]

32. https://github.com/znxlwm/tensorflow-MNIST-GAN-DCGAN 31

33. 32

34. 33 https://github.com/robbiebarrat/art-DCGAN

35. 34 Generator

36. 35 Robot Box Game: +1 reward for pushing box into

    black square -0.01 penalty for each step Camera observer: Shuts down robot once one box is pushed

37. 36 https://www.youtube.com/watch?v=sx8JkdbNgdU

38. Reading Discussion • Capability Contol: – Boxing (limit access) –

    Incentive (cryptotoken rewards) – Stunting (constraints on abilities) – Tripwires (diagnostics) • Motivation Selection – Direct specification – Domesticity (limit scope) – Indirect normativity – Augmentation 37 For your topic: 1. Explain what it is 2. Why Bostrom doesn’t think it is sufficient 3. Why it could work 4. Argue for or against its effectiveness

39. https://www.youtube.com/watch?v=3TYT1QfdfsM 38