ÁùºÏ²ÊÖ±²¥¿ª½±

The reason behind the low frame rate of the heatmap design can be found in Figure 2. The final 68 facial landmarks as shown in the bright pixels of the 112x112 resolution image are generated by 68 heatmaps shown in Figure 2(b). That is, each of the facial landmarks can be derived by a 112x112 heatmap, leading to computational complexity that is much higher than the 136 output architecture. 

Figure 3. Two neural network architecture advanced designs (offset and residual) of last layer of inference part

Mapping the Architecture to a Simpler Application ¨C Lane Departure Warning

In an ideal world, system designers can develop a single solution framework that applies to all kinds of applications. To test this, we tried mapping the facial landmark tracking neural network architecture to a lane departure warning (lane departure warning) application. In Figure 4, we illustrate the idea of converting the lane departure warning application to the same problem domain of facial landmark as shown in Figure 4(a) by predicting three points (yellow and red circles) from an input image as shown in Figure 4(b). 

Figure 4. The concept behind converting a facial landmark application to lane departure warning is just mapping points

Since our neural network architecture as shown in Figure 3(b) works well for facial landmark applications, we simply modified the inference layer of Figure 3(b) such that the last inference layer contains three 28x28 heatmap and six 28x28 offsets in the x and y coordinates as shown in Figure 5(a). However, the result of Figure 5(a) was not good, so we further modified the network by make the network deeper as shown in Figure 5(b). Unfortunately, that attempt¡¯s results were not accurate enough either.

Figure 5. Two neural network architecture designs that did not provide accurate lane departure warning results

To design the proper neural network architecture for lane departure warning, we thought about the property of neural network as shown in Figure 6. Figure 6(a) shows the two major parts: the backbone (feature extraction) and inference (fully connected) layers, of the deep convolutional neural network architecture. Figure 6(b) shows an example from the low-level to high-level features according to the layers in the feature extraction part of the network architecture. It is obvious that the lanes in the lane departure warning application are either straight lines or curves and thus we needed to increase the number of low-level features instead of increasing the number of network layers. 

Figure 6. The low- and high-level features according to the layers of neural network

Figures 7 and 8 show the final network architecture for the lane departure warning application and the experimental results, respectively. Note that the red, blue and green lines in Figure 8 indicate the left, right, and ground truth, lane lines, respectively.   

Figure 7. The final neural network architecture design for lane departure warning

Figure 8. The experimental results of the lane departure warning system

Conclusion

Although deep learning is a powerful tool in machine learning, its performance is still dependent on the neural network architecture. Therefore, we need to understand the fundamentals of the problem to design the proper neural network architecture. In this article, we showed one way to design neural network architectures for different applications. The network architecture for facial landmark detection was initially designed using MobileNET as the backbone network and 136 outputs for x and y coordinates. It was further modified to be a small resolution heatmap and used additional offsets to achieve the requirement of real-time processing. The exact same structure could not be used for lane departure warning since lane departure warning requires more low-level features instead of deeper layers. In this way, the design of network architectures can build from more complex to simpler, and the proper neural network architecture can be designed accordingly.