background

Recently, I find a good cources about multimodal machine learning. In this blog, I will study it and note my understanding.
Here is orgin URL : ppt

O

master bae knowledge about Unimodal and classic work

KR

1. Unimodal basic representations
   ▪ Visual, language and acoustic modalities
2. Data-driven machine learning
   ▪ Training, validation and testing
   ▪ Example: K-nearest neighbor
3. Linear Classification
   ▪ Score function
   ▪ Two loss functions (cross-entropy and hinge loss)

1. Unimodal basic representations(visual, language,speech )

visual

visual multi-class :

visual multi-class and multi task :

2. language

language word level class :

language document-level class :

language utterance-level class :

3. audio

audio class:

2. Data-Driven Machine Learning

1. K-Nearest Neighbor

3. Train,validation and test。
   

3. Linear Classification

1. Interpreting Multiple Linear Classifiers：

Here is multiple linear calssifiter base saple linear classifiter.

2. Loss function

loss funcation often made up of three parts, data term is used to train data lable.

regularzation is make our model not voerfiting.

constraints is make our model has sepecial function in some area.