CS782 - Pattern Recognition and Applications - Spring'2006

• Study of recent advances in development of statistical pattern recognition algorithms, approximation, and estimation techniques. Topics include statistical estimation theory (decision rules and Bayes error), classifier design, parameter estimation, feature extraction (for representation and classification), clustering, statistical learning theory, support vector machines and other kernel methods, and applications in biometrics, such as face recognition, iris recognition, and fingerprint recognition.  Additional topics include nonparametric density estimation, nonparametric classifier design, machine learning for pattern recognition, and evolutionary computation for pattern recognition.

• Prerequisites: CS 610 – Data Structures and Algorithms

• K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd edition, Morgan Kaufmann, 1990.
• R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, 2001.
• V. N. Vapnik, The Nature of Statistical Learning Theory, 2nd edition, Springer, 2000.
  
• T. M. Mitchell, Machine Learning, WCB/McGraw-Hill, 1997.
• S. Haykin, Neural Networks - A Comprehensive Foundation, 2nd edition, Prentice-Hall, 1999.
  
• Selected papers.

1. Introduction
• Pattern Recognition Fundamentals
• Formulation of Pattern Recognition Problems
• Major Components of a Pattern Recognition System
  
• Related Fields: AI, ML, NN, EC, SLT/SVM
2. Bayes Decision Theory - Decision Rules
• The Bayes Decision Rule for Minimum Error
• The Bayes Decision Rule for Minimum Cost
• The Neyman-Pearson Decision Rule
• The Minimax Decision Rule
  
3. Bayes Decision Theory - Bayes Error
   
• Error Probability and the Bayes Error
  
• Upper Bounds on the Bayes Error
• Chernoff Distance and Bhattacharyya Distance
  
4. Parametric Classifier Design
• The Bayes Classifier
• Quadratic Discriminant Analysis (QDA) and Linear Discriminant Analysis (LDA)
• Linear Classifier Design and Examples
  
• Quadratic Classifier Design
• An example: A Bayesian Discriminating Features Method for Face Detection (PAMI, 2003)
• Piecewise Classifier Design
  
5. Parameter Estimation
• Maximum-Likelihood Estimation
• Bayesian Estimation
6. Feature Extraction and Mapping for Representation
• Optimal Feature Representation Methods
•  Principal Component Analysis (PCA)
  
• Real Data Case Study I - MEF Classifier Design
• Most Expressive Feature Extraction
• MEF-based Linear Classifier Design
7. Feature Extraction and Mapping for Classification
• Optimal Feature Classification Methods
• Linear Discriminant Analysis (LDA)
  
• Real Data Case Study II - MDF Classifier Design
• Most Discriminating Feature Extraction
• MDF-based Linear Classifier Design
8. Statistical Learning Theory (SLT)
   
• Structural Risk Minimization (SRM)
  
• Support Vector Machines (SVM)
• More Kernel Methods - Kernel PCA, Kernel Fisher Analysis (KFA)
  
9. Applications: Biometrics
• Face Recognition
  
• Iris Recognition
  
• Fingerprint Recognition
10. Clustering
    
• Parametric Clustering
• Nonparametric Clustering
11. Nonparametric Density Estimation
• Parzen Density Estimation
• KNN Density Estimation
• Expansion by Basis Functions
  
12. Nonparametric Classifier Design
• Parzen Approach and its Error Estimation
• KNN Approach and its Error Estimation
  
13. Machine Learning  for  Pattern Recognition
• Bayesian Learning
  
• Neural networks
  
• Decision Trees
14. Evolutionary Computation for  Pattern Recognition
• Genetic Algorithms (GA)
• Evolutionary Strategy (ES)
• Evolutionary Programming (EP)

Projects (topics are related to our course Contents)

Presentation (~10 mins.)

Paper (~10 pages)

Class attendance

NJIT Honor Code will be upheld, and any violations will be brought to the immediate attention of the Dean of Students

Students will be consulted with by the instructor and must agree to any modifications or deviations from the syllabus throughout the course of the semester.

• The Face Recognition Home Page
• Face Detection Home Page
• MATLAB
• Matlab Primer (Third Edition, By Kermit Sigmon - pdf file)
• Matlab Getting Started (copyright Mathworks - pdf file)
• Matlab Image Processing Toolbox User's Guide (copyright Mathworks - pdf file)