EECS 349 Problem Set 4

Due 11:59PM Tuesday, November 23

v1.0 Fri Nov 12 19:06:38 CST 2010

Instructions

1. Answer the questions below in a single text or PDF file.
2. Name the file PS4-<first-name>-<last-name>.txt (or .pdf).
3. Attach the file in an email with the subject EECS349-PS4-<first-name>-<last-name>.
4. Send the email to both of these addresses:
   • zhiyaoduan00 at gmail dot com
   • arefinhuq2013 at u dot northwestern dot edu

Questions

1. Consider six binary variables related to scoring well on an exam: G (got a good night's sleep), S (studied a lot), I (find the material interesting), E (exam is easy), A (got an A on exam), R (recommend class to friends at end of quarter).
   1. (1 point) Draw a Bayes Net representing this situation. There are multiple different reasonable networks for this domain. You should try to exploit conditional independencies and end up with relatively few edges.
   2. (0.5 points) What is the minimum number of probabilities you would need in order to specify all the conditional probability tables for your Bayes Net?
   3. (0.5 points) How many probabilities would you need to specify for the full joint distribution over the six variables?
   4. (0.5 points) Using your network, is E (whether the exam is easy) independent of G (whether you got a good night's sleep) when we are not given the value of any other variables? In a sentence, justify why or why not.
   5. (0.5 points) Using your network, assume we know A = true (you got an A on the exam). Is E conditionally independent of G given A? In a sentence, say why or why not.
2. Consider the Bayes Net pictured below.
   1. (1 point) What is P(E)? Show your work (hint: this should be just 2-3 lines of math).
   2. (1 point) Suppose that the network was computed from maximum likelihood estimates over a data set of ten examples. Suppose we then see two more examples with A=1. What will be the new maximum likelihood estimate for P(A)?
   
3. (1 point) Draw a perceptron (with weights) that has two binary inputs A and B and that computes A NAND B. That is, the perceptron outputs 0 if A=B=1, and 1 otherwise.
4. 1. (0.5 points) Your boss wants you to build a predictor for phenomenon X, using a data set with 10 million examples, and wants the team to meet the following morning to analyze the factors that influence X based your learned model. Given just this information, would you choose neural nets or decision trees? Give a one-sentence justification.
   2. (0.5 points) You have a data set of images, with 256 pixels specified as red, green, and blue values summing to one for each pixel. Your goal is to classify the images in terms of whether they contain a face. Would you use decision trees or neural networks for this task? Give a one-sentence justification.
   3. (1 point) Boosting often improves the accuracy of decision trees. Name two disadvantages to using boosting vs. just a single decision tree.
   4. (1 point) Someone tells you that they have a neat learning algorithm that doesn't always achieve the best accuracy, but is always at least okay -- in fact, for a binary classification problem, given 100 training examples the learning algorithm will always achieve at least 60 percent accuracy over the remaining (unseen) examples. Do you believe this person? In a sentence, why or why not?
   5. (1 point) In a short paragraph (3-4 sentences): describe some new idea related to machine learning that occurred to you while taking the course. It could be a novel application, an idea for a new learning algorithm, a tweak to improve an existing algorithm, a subtle similarity between something we covered and material from another field, etc.