Simple Discrete Models

For those of you interested in learning a bit more about discrete models, below are some fantastic resources to help you on your journey:

  • Book: Murphy — Chapter 2 — Probability
  • Book: Murphy — Chapter 3 — Generative Models for Discrete Data
  • Book: Bishop — Chapter 2, Sections 2.1-2.2 — Probability Distributions
  • Book: MacKay — Chapter 2 — Probability, Entropy, and Inference
  • Book: MacKay — Chapter 3 — More About Inference
  • Book: Mackay — Chapter 23 — Useful Probability Distributions


Book reference for above:

Machine Learning: A Probabilistic Perspective Kevin P. Murphy, MIT Press, 2012.

Pattern Recognition and Machine Learning Christopher M. Bishop, Springer, 2006. An excellent and affordable book on machine learning, with a Bayesian focus. It covers fewer topics than the Murphy book, but goes into more depth on the topics it covers.

David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press.



  1. I am glad to see you referencing MacKay’s book. If people haven’t read that and are interested in machine learning/data science that book should blow the minds.

    I know it did for me.


  2. Please consider writing a few posts around the topics in the chapters you have written about. I would love to gain a greater understanding around the subject. Most of the information seems to be over my head, but I might have a better shot of learning the material with your perspective.

  3. I got this website from a friend who told me about discrete models and now as I am browsing this website and reading your resources I see that they are very informative.

Leave a Comment

Your email address will not be published. Required fields are marked *