Math for Machine Learning

Stochastic Processes (Random Planar Maps)

Once you decide to get started with Data Science (in a serious way), the first few months (if not year) can seem pretty difficult. At times, maybe even hopeless, especially if you do not have the necessary academic background or it has been a while since you have been in an academic setting. In my judgment, you should not let that stop you from your pursuit. There are so many resources online/offline to help you start to fill in your gaps.

Below are some of the key areas that you should have mastery over for you to go further with data science/machine learning:

Calculus

  • Functions
  • Continuity
  • Differentiability
  • Integration (single and multi-variables)
  • Optimization
  • Convexity/Concavity

Linear Algebra

  • Vectors
  • Matrices
  • Eigenvalue
  • Vector
  • Singular Value Decomposition
  • Least Squares Estimation and Matrix Algebra

Statistics/Probability

  • Basic probability
  • Sample spaces
  • Conditional probabilities and independence
  • Random variables
  • Moments
  • Distributions
  • Chi-Squared
  • F-Test
  • T-Test
  • Bayes’ Theorem
  • Marginalization
  • Bayesian Inference
  • Likelihood
  • Estimation
  • Regression
  • Analysis of Variance

Stochastic Processes and Dynamical Systems

  • Dirichlet Processes
  • Gaussian Processes for Machine Learning

What else do you think is necessary?

Simple Discrete Models

For those of you interested in learning a bit more about discrete models, below are some fantastic resources to help you on your journey:

  • Book: Murphy — Chapter 2 — Probability
  • Book: Murphy — Chapter 3 — Generative Models for Discrete Data
  • Book: Bishop — Chapter 2, Sections 2.1-2.2 — Probability Distributions
  • Book: MacKay — Chapter 2 — Probability, Entropy, and Inference
  • Book: MacKay — Chapter 3 — More About Inference
  • Book: Mackay — Chapter 23 — Useful Probability Distributions

 

Book reference for above:

Machine Learning: A Probabilistic Perspective Kevin P. Murphy, MIT Press, 2012.

Pattern Recognition and Machine Learning Christopher M. Bishop, Springer, 2006. An excellent and affordable book on machine learning, with a Bayesian focus. It covers fewer topics than the Murphy book, but goes into more depth on the topics it covers.

David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press.