The demand for artificial intelligence applications is rising in every field. And with this demand, machine learning algorithms and techniques are becoming more complicated and compute-intensive. For embedded devices, these compute-intensive and high memory-constrained algorithms are becoming a challenge. Dedicated hardware accelerators are generally the leading solution for embedded systems. This is because of their high performance, low area, and power footprint. Although hardware accelerators are a great solution for machine learning, they lack flexibility and programmability. Another alternative is General-Purpose Processors (GPPs), which provide better flexibility and programmability as compared with hardware accelerators, however, they lack performance, area, and power. Between these two plausible solutions, there is a big gap in terms of different metrics, which can be bridged by Application-Specific Instruction-set Processors (ASIPs). ASIPs are specialized processor systems with an architecture that is tailor-made for a certain application. ASIPs provide better performance as compared with general-purpose processors and have more flexibility and programmability as compared with hardware accelerators. The main goal of an ASIP design is to maximize the performance-to-power ratio for a specific application. This work provides an in-depth survey of different ASIP designs with a focus on Deep Neural Networks (DNNs). For a generic comparison, the proposed related works of ASIP design are classified based on their microarchitecture approach and optimization adopted. The strong and weak points are pointed out in different optimization and microarchitectures, and future trends for ASIP design for fast machine learning are identified.