Math 5040: Stochastic Processes and Simulation I, Fall 2014

Schedule: 11:50--12:40 MWF, 8/25 to 12/12 except for 9/1, 10/13,15,17 and 11/28, in LCB 215. Notice: PhD students may want to register under the number 6810.

Instructor: S. Ethier (Prof.), 581-6148, ethier@math.utah.edu.

Office hours: 3:45--5:00 MW, JWB 119. Other times are available by appointment.

Text: Introduction to Stochastic Processes, Second Edition by Gregory F. Lawler (Chapman & Hall/CRC, 2006).

Prerequisite: Math 5010. This is absolutely essential. Also, some knowledge of computer programming will be helpful.

Topics covered: We will cover the whole book over the course of two semesters. This will probably require supplementing the material in the book. But we will begin with some lectures on simulation, which the book does not cover. For that material a good reference is Sheldon Ross's Introduction to Probability Models, Chapter 11. I will post lecture notes on this web page, at least for the simulation material.

Assignments: There will be a weekly assignment, mostly textbook problems but also simulation problems. For the simulation problems, you can use whatever programming language you prefer (C++, Mathematica, Maple, MatLab, Java, Python, R, SAS, etc.). You will turn in your program (with documentation, i.e., comments explaining what your instructions mean) AND output on paper. I do not want an electronic file. The purpose of the assignments is to learn the material. Therefore, working in groups is not allowed because the result is usually that one person does the work and the others copy it. Assignments will be posted on this page on Fridays and due the following Friday.

Grades: There will be a midterm exam shortly after Fall Break. There will be a final exam on Friday, December 19 at 10:30 a.m. Note that early exams cannot be given under any circumstances. Grades will be based on homework (30%), midterm exam (30%), and final exam (40%).

Expected learning outcomes: The student who successfully completes Math 5040 will have a working knowledge of computer simulation and will be familiar with the basics of Markov chain theory, including Markov chain Monte Carlo (MCMC). The student who also successfully completes Math 5050 will have a good overview of stochastic processes and in particular will be familiar with martingales, renewal processes, reversible Markov chains, Brownian motion, and stochastic integration. Such a student will have sufficient background to apply stochastic processes in his or her own field of interest.

Lectures 1--5 on simulation.

Week 1 (Aug. 25, 27, 29). We covered the simulation notes through the discrete inverse transformation method. We did not get to the rejection method. Assignment 1. Suggested solution of Problem 2.
Week 2 (Sep. 3, 5). We finished the simulation notes. Assignment 2. Suggested solutions.
Week 3 (Sep. 8, 10, 12). We started Markov chains, covering the definitions and three examples, the Markov chain for Parrondo's game, the Ehrenfest chain, and the random walk on the vertices of an N-dimensional hypercube. Assignment 3.
Week 4 (Sep. 15, 17, 19). We established the limit theorem for finite, irreducible, aperiodic Markov chains. Assignment 4.
Week 5 (Sep. 22, 24, 26). Periodic example. We finished Chapter 1 on finite Markov chains. Assignment 5: Chapter 1, Exercises 5, 8, 10, 14.
Week 6 (Sep. 29, Oct. 1, 3). We started Chapter 2, getting a necessary and sufficient condition for recurrence. Assignment 6. The hint given for Problem 3 is misleading. Instead, use the Fact at the top of page 50. Solutions on Canvas.
Week 7 (Oct. 6, 8, 10). We nearly finished Section 2.3. Assignment 7: Chapter 2, Exercises 1, 2, 3, 7.
Week 8 (Oct. 20, 22, 24). We didn't quite finish branching processes and had our midterm exam. No new assignment for next week. Assignment 7 can be turned in at class on Monday, Oct. 27.
Week 9 (Oct. 27, 29, 31). We started Chapter 3, covering the Poisson process and starting continuous time Markov chains. Assignment 8: Lawler, Exercises 2.8, 2.10, 3.1, 3.2.
Week 10 (Nov. 3, 5, 7). We continued Chapter 3 through Section 3.2. Assignment 9: Lawler, Exercises 3.4, 3.6, 3.8, 3.10.
Week 11 (Nov. 10, 12, 14). We finished Chapter 3. Assignment 10. Clarification of Problem 3. Potential customers arrive at the rate of 3 per hour. If the shop is full (with two customers, one being served, one waiting), a newly arriving potential customer is lost forever.
Week 12 (Nov. 17, 19, 21). We nearly finished Section 4.1. Example 1. Since next Friday is not a class day, there is no new assignment this week.
Week 13 (Nov. 24, 26). We finished Chapter 4. Example 2. Assignment 11: Lawler, Exercises 4.1, 4.3, 4.6, 4.8.
Week 14 (Dec. 1, 3, 5). We continued with MCMC. MCMC Lectures 1--5. Assignment 12. (Monday, 4 pm. Typo corrected in Assignment 12.)
Week 15 (Dec. 8, 10, 12). We finished MCMC on Wednesday. Review Friday.

Final exam is scheduled for Friday, Dec. 19, 10:30--12:30, usual room. You may bring one sheet of notes and a calculator. The final exam from 2012 is posted on Canvas, but written solutions have been lost.