Probability Theory And Examples Homework Chart

COURSE COVERAGE

DateCoverage (with homework)
Jan. 6
  • Quick review of probability.
  • References: "A first course in rigorous probability theory" by Jeff Rosenthal
  • Homework: [PDF]
Jan. 8
  • More on weak convergence; definition of Brownian motion and Brownian Bridge; nonparametric estimation of a distribution and the empirical CDF.
  • References: "A first course in rigorous probability theory" by Jeff Rosenthal (for BM def'n and weak convergence)
    "Probability: Theory and Examples" by Rick Durrett (for solid CLT proof)
    "All of Nonparametric Statistics" by Larry Wasserman (for empirical cdf stuff - though not complete...)
  • Homework: [PDF]
Jan. 13
  • Theoretical results for the empirical CDF: Glivenko-Cantelli, Donsker (that's what the CLT version is called), and DKW inequality. Applying results to form confidence bands for true CDF.
  • References: "All of Nonparametric Statistics" by Wasserman
  • Homework: [PDF][R script for conf. bands]
Jan. 15
  • Types of confidence sets; Kolmogorov-Smirnov Test; Introduction to kernel density estimation and histograms.
  • References: "All of Nonparametric Statistics" by Wasserman
  • Homework: [PDF]
Jan. 20
  • Histograms con't: asymptotic IMSE and choosing the optimal bandwith.
  • References: "All of Nonparametric Statistics" by Wasserman
  • Homework: [PDF]
Jan. 22
  • Various definitions of continuity. Proper proof of asymptotic IMSE formulas for histogram.
  • References: "All of Nonparametric Statistics" by Wasserman
    Analysis review (warning: not official, not mine) [PDF]
    My written version of that proof: [PDF]
  • Homework: [PDF]
Jan. 27
  • Kernel density estimation - main ideas.
  • References: "All of Nonparametric Statistics" by Wasserman
  • Homework: [PDF]
Jan. 29
  • Kernel density estimation con't: proper proofs and what happens in higher dimensions.
  • References: "All of Nonparametric Statistics" by Wasserman
    "Introduction to Nonparametric Estimation" by Tsybakov (available through library)
  • Homework: [PDF]
Feb. 3
  • Introduction to the LASSO.
  • References: "Statistics for High-Dimensional Data" by Buehlmann and van de Geer (available through library) - so far we have only looked at small parts of chapter 2
    Original LASSO paper: [PDF]
    Retrospective on the LASSO: [PDF]
    "Shrink It" by the Fifth Moment (words are in the comments): [youtube]
  • Homework: [PDF]
Feb. 5 TEST ONE
Feb. 10
  • Understanding the role of the penalty function: Hodges' example, LASSO solution if X has orthonormal columns
  • References: R script to plot Hodges example risk (and resulting pictures PDF)
    Wasserman's blog entry on super-efficiency (a must-read!)
    Fan and Li's SCAD paper
  • Homework: [PDF]
Feb. 12
  • Unbiasedness, sparsity and continuity via the penalty function, SCAD, non-rigorous review of MLE asymptotics
  • References: SCAD paper (see above for link)
    MLE asymptotics: Chapter 6 of "Introduction to Mathematical Statistics" by Hogg, McKean, and Craig (in particular, look over Theorems 6.1.1, 6.1.3, 6.2.2 for the univariate case)
  • Homework: [PDF]
Feb. 16-20 READING WEEK
Feb.24
  • SCAD theory: theorem 1 and proof, lemma 1
  • References: SCAD paper
Feb.26
  • SCAD theory: proof of lemma 1 and theorem 2; when do these results apply?
  • References: SCAD paper
  • Homework: [PDF]
March 16
  • Quick review of SCAD results. Some consistency results for M-estimators.
  • References: SCAD paper, Asymptotic Statistics by van der Vaart
  • Homework: [PDF]
March 18
  • Lasso asymptotics (Theorem 1 and proof, Theorem 2 of Knigth&Fu), finite p.
  • References: [paper] (but fixed up proofs are given in lectures)
  • Homework: [PDF]
March 24
  • Proof of Theorem 2 of Knight&Fu, discussed Theorem 3 (with warnings)
  • Homework: [PDF]
March 26
  • An overiew of results for the Lasso; Theorems from Zou (2006, JASA).
  • References: [paper]
March 31
  • Continued with Theorems from Zou (JASA, 2006).
April 2TEST 2
April 7
  • Finished all proofs from Zou. Started on Chapter 6 of Statistics for High-Dimensional Data: theory for the LASSO when p>n.
  • References: Statistics for High-Dimensional Data by Buehlmann and van de Geer (electronic version available in library).
  • Homework: [Zou]
April 9
  • Finished Section 6.2.2 from Chapter 6 in Statistics for High-Dimensional Data. End of new material.
  • Homework: [SHDD]
April 14Review of theoretical results for the LASSO: SCAD paper, Knight&Fu, Zou (all three p


  1. Thursday 1/16 Introduction: Bertrands paradox asks for a solid foundation of probability theory. The Petersburg Casino will be treated later. We will flip coins more often in this course.
  2. Tuesday 1/21 Kolmogorov's Axioms of Probability theory . Basic properties. Examples: 2 dices, Boys/Girl problem Bertrand solved. Homework 1 (PS file)
  3. Thursday 1/23 Examples: radioactive decay, Monty Hall problem Continuity property of P, Conditional probability: boy or girl problem revisited. An urn problem.
  4. Tuesday 1/28 The urn problem, Bayes rule, the drawer problem, independence of events. Pairwise independence is not enough. Homework 2 (PS file)
  5. Thursday 1/30 More examples on independence and conditional probability and Bayes rule. Theory: Bernoulli formula and Sylvester's "switch-on switch off" formula for the probability of a union of events. Reminder: even without probability some questions can be difficult: Life also can be complicated
  6. Tuesday 2/3 Review of combinatorics Permutations, Sampling and Combinations. Examples, the Bursday paradox. Distribution of Homework 3 (PS file)
  7. Thursday 2/5 More examples in combinatorics. Especially the odds in Arizona lotto and some combinatorial questions in music .
  8. Tuesday 2/11 Discrete random variables, densities, distributions, examples of distributions. How to get onto mars using probability. Homework 4 (PS file)
  9. Thursday 2/12 More examples of distributions : A sailor, Euler and a devilish random variable . Independent random variables.
  10. Tuesday 2/18 On distributions: what happens when adding independent random variables. From Bernoulli to Poisson. Applications. Checklist for Midterm. distributions II : Homework 5 (PS file)
  11. Thursday 2/20 Midterm topics: Material until Tuesday 2/18.
  12. Midterm 1 (PS file)
  13. Tuesday 2/25 Review over Midterm. Expectation of random discrete variables: Definition, computing examples. Expectation overview . Homework 6 (PS file)
  14. Thursday 2/27 Properties of expectation, more examples. Definition of variance. Probability generating functions and its use for computing expectation and variance.
  15. Tuesday 3/4 Covariance, correlation, correlation coefficient, regression line. Independent random variables are uncorrelated. Schwartz inequality. Homework 7 (PS file)
  16. Thursday 3/6 Excess and other higher moments, sum of independent random variables. Estimating probabilities with the Chebychev inequality. The weak law of large numbers for IID random variables.
  17. Tuesday 3/11 An application of the weak law of large numbers in analysis: the Weierstrass theorem We reviewed some material in form a Selftest (PS file) which is corrected by yourself and will not be part of the grade. The test should be useful for spotting eventual white spots in the first half of the course. Homework 8 (PS file)
  18. Thursday 3/13 Preparation for continuous random variables. Countable versus uncountable. Why did we introduce sigma algebras? The Banach-Tarski paradox. What is integration ? Hacking the code at the foundations of probability theory with the theorem of Caratheodory. The paradox of Schwartz on triangulations of surfaces.
  19. SPRING BREAK.

0 thoughts on “Probability Theory And Examples Homework Chart”

    -->

Leave a Comment

Your email address will not be published. Required fields are marked *