With the rising variation and complexity of embedded work-loads, FPGA-based systems are being increasingly used for many applications. The reconfigurability and high parallelism offered by FPGAs are used to enhance the overall performance of these applications. However, the resource constraints of embedded platforms can limit the performance in multiple ways. In recent years, Approximate Computing has emerged as a viable tool for improving the performance by utilizing reduced precision data structures and resource-optimized high-performance arithmetic operators. However, most of the related state-of-the-art research has mainly focused on utilizing approximate computing principles individually on different layers of the computing stack. Nonetheless, approximations across different layers of computing stack can substantially enhance the system's performance. To this end, we present a framework to enable the intelligent exploration and highly accurate identification of the feasible design points in the large design space enabled by cross-layer approximations. Our framework proposes a novel polynomial regression-based method to model approximate arithmetic operators. The proposed method enables machine learning models to better correlate approximate operators with their impact on an application's output quality. We use a 2D convolution operator as a test case and present the results for FPGA-based approximate hardware accelerators.