1. 264 is a method and format for video compression, the precise of converting video into a format that takes up less capacity, and it is an industry standard. The presentation well describes all the components in this figure. Before talking about the whole precise of encoding in this figure, let me give the definition of some concepts. So first, what is I-frame, P-frame, and B-frame? I-frame does not depend on any frames, it uses intra-prediction to encode the frame. P-frame and B-frame can use the previous encoded frame as reference, so it can use inter-prediction to encode. These two figures are generated by stream I, I also use this bitstream analyzer to analyze. Red column represents I-frame, blue column represents B-frame. So we can find that P-frame uses the last frame. We can see that the previous frame used P-frame and B-frame, both are these frames. In the previous slide, I have shown you how to use these three frames to do inter-prediction. I-frame does not depend on any frame. I have just mentioned two terms, inter-prediction and intra-prediction, what do they mean? Intra-prediction form the prediction from the coded frame. It is used by P-frame and B-frame. Intra-prediction form the prediction from the previously coded parts from the current frame. Let me give the brief introduction. introduction of the precise self-encoding. This is the current frame which is waiting to be encoded. First it needs to decide the frame type. If it is I-frame, it only uses intra-prediction. If it is P or B-frame, it uses intra-prediction or inter-prediction for each macro block, depending on the cost of intra-prediction and intra-prediction. Manager original data minus prediction is the residual data. It does DCT and quantization for the residual data. As for the coding part, it does zigzag and also uses the prediction to reduce the size of some header information. For the result of DCT and quantization, it also do the reverse calculation for it. After that, it adds the prediction and the reverse calculation result together. This precise self-encoding. Intra-prediction uses the restricted data for the next intra-prediction. Before adding to the reference list and being used by inter-prediction, it uses loop filter to reduce blocking distortion. I will talk about each component in detail. First I want to talk about how that they decide the frame type. I use stream I to help me to analyze the bitstream. This is the B-frame. Red color represents I-frame, blue color represents B-frame, P-frame, green color represents B-frame. So, how does it determine this? I-frame is determined by two factors. When there is no significant change of thing, it is determined by precise parameter GOP size. If GOP size is 100, every 100 of them will generate an I-frame. When a significant change of thing is detected, it inserts an I-frame instantly, regardless of the precise GOP size. Let's see the video again. When the thing changes, it inserts an I-frame. I think now you have a brief understanding how to decide the frame type. GOP is a precise parameter. But how to calculate the significant change of thing? For the current frame, it calculates the cost of inter- and intra-prediction for the whole frame. If the cost of intra-prediction is less, it means that it cannot predict from the previous coded frame well, so it detects the change of thing, so that an I-frame is inserted. Let's see the next slide. You may ask, in the precise of deciding the frame type, it does inter- and intra-prediction. But after that, it also does the intra- and intra-prediction. So, what is the difference? The prediction in this component is only about deciding the frame type, it is not related to the final bitstream, so it needs to make this precise fast enough. The calculation of the prediction of this stage is based on lower-resolution frames. For example, the original width is 1920, the width of lower-resolution frame is 960. Okay. So, apart from that, I think it goes too far, there is no selective partition mode. Originally, there are many partition modes, but both of them make the decision fast. Now let's talk about the intra-prediction. It forms the prediction from the previously coded part from the current frame. Totally, it has nine prediction modes. For the first three modes. For the first three modes. For the first three modes. For the vertical, horizontal, and DC. I draw this table to help you understand this most. For the vertical mode, it uses the above values. For the horizontal mode, it uses the live values. The 16 x 16 micro block can be divided into 16 4x4 block. It calculates the STTD for each partition mode and choose the mode with the smallest STTD. of each partition block. It calculates STD for all the prediction modes and choose the prediction mode with the smallest STD. For example, in order to calculate STD for 4x4 partition mode, it needs to use 9 prediction modes for each 4x4 block and choose the prediction mode with the smallest STD. Totally, it needs to calculate 16 times. The sum of them is STD of the 4x4 partition mode. Let's see an example in reality. This is a frame of the video. I also use this stream eye to help me analyze. Let's have a look at these two parts. The partition mode of the skies is 16 by 16 and 8 by 8. For the detailed part, 16 and 8 by 8 partition mode cannot work well, so it uses 4x4 partition mode instead. You may wonder why does not all the partition modes use 4x4 mode? Because it needs more calculation resource. It needs to use 4x4 mode only when it really needs it. In order to avoid using the small partition mode, when it calculates STD for 4x4 mode, it adds a small punishment. When it calculates STD for 4x4 mode, it adds a larger punishment. I have finished talking about the intra-prediction for I-frame. Let's come to the prediction for p-frame. It is more complicated because for each macroblock, it needs to decide to use inter-prediction or intra-prediction. So it needs to compare the STD for inter and intra-prediction and choose the one with the smallest STD. In the previous slides, I have talked about the intra-prediction. The important thing for intra-prediction is about motion estimation. This is about selecting a prediction region and generating a prediction block. The offset between the position of the current partition and the prediction region is the motion vector. In order to make good motion estimation, the estimation is based on quarter pixel. The quarter pixel is generated by interpolation between adjacent symbols. It means that there are more pixels and there are more partition modes. It needs to calculate the STD for each partition mode and choose a mode with the smallest STD. Let's see the selection of partition modes in the real video frame. At the right hand side of the frame, it is sky. There is no sun. Let's see the selection of partition modes in the real video frame. At the right hand So there is not so much motion. Most of the partition modes are 16 by 16 partition mode. In the left-hand side of the frame, there is a person riding bicycle. In order to make the good prediction of the motion, most of the partition modes are 8 by 8 mode. Now we know how to do prediction. We use original frame minus prediction, then we get the residual data. What does residual data look like? Let's visualize it. From the video, we can find that the prediction is not accurate in where there is much motion. So the residual data in that part is larger. So let's see it again. When there is much motion, the prediction is not good. It leads to the larger residual data. The last video is residual data, and this part of video is the original video. Now let's come to the translation. For each 16 by 16 macroblock, it does 16 times 4 by 4 DCT transform for it. From the figure, we can find that there is the other transform. The second transform is applied to the lowest of DC frequency coefficients of the first transform. These DC values tend to be highly correlated, and the second transform improves the coding performance. The DC coefficients are the same. The DC coefficient block is further transformed using a 4 by 4 hard-marked transform. Then it does quantization for 4 by 4 DC block and 16 by 16 AC block. This is the precise of forward transform quantization. And this is the precise of inverse transform and inverse quantization. It is very similar to the above precise. After transform and quantization, it needs to do coding. First, it needs to encode the code. It uses the frame's information like partition mode and quantization value, but it does not directly encode them. In order to improve the coding performance, it uses prediction to reduce the bits. For I Frame, it has three partition modes. It uses minimal value of the upper and left blocks partition mode to predict. If it is the same, it does not need to encode the mode. For P-Frame, it needs to predict the number of blocks. For P-Frame, it needs to predict the number of blocks. For P-Frame, it needs to predict the number of blocks. The motion vector, the prediction vector is the median vector of the upper and left blocks vector. It only needs to encode the difference between the actual vector and prediction vector. As for encoding the quantization values, it only encodes the difference between the actual quantization and last block quantization. After finishing the precise of transform and quantization, it scans the coefficient blocks in a zigzag order, then encodes. We have finished this part, let's talk about the loop filter. Before adding the constructed frame into the reference list, this needs to use loop filter. The filter smooths block edges, improving the appearance of the decoded frame. This generally improves compression performance because the filtered image is more fit for reproduction of the original frame than a blocky and filtered image. From these two figures, we can find that this figure is more fit for than this figure. So that's all for the whole process of H. 264 encoding, thanks for listening.