Q1-
Consider the following hypothesis class F = {x 7→ w1 + w2x 2 : [w1 w2] ∈ R 2}, where x ∈ R is the input datum.
1. Provide a hypothesis class that is less expressive than F.
2. Provide a hypothesis class that is more expressive than F.
3. Provide a hypothesis class that is disjoint from F.
Q2-
Consider the squared loss Loss(x, y, w) = (y − max{w · φ(x), 0}) 2 1. Draw the computational graph of the function Loss(x, y, w).
2. Number the internal nodes, 1, 2, . . ., and next to every node i in the graph, indicate the forward value with fi , and indicate the backword value with gi . Also, on the edges of the graph indicate the corresponding derivatives, and use the forward values as appropriate to do so. Provide the expression of the gi ’s as function of other backword and edge values, as appropriate.
3. Assume that w = [1 2], φ = [−1 1], and y = 3. Compute all the forward values, effectively performing a forward pass.
4. Using the forward values computed previously, compute all the backward values, effectively performing a backword pass. In particular, compute also the quantity ∂Loss ∂w .