The field of computer vision is steadily growing in its complexity and application areas. FPGAs have shown that they can meet the growing demands for performance and energy efficiency. However, their programmability is a major challenge for software programmers. With OpenVX a standard for cross platform acceleration of computer vision applications exists. Existing OpenVX FPGA frameworks often contain non-standard constructs and consider either fixed processor architectures or specialized non-adaptive accelerators. Therefore, we propose ArcvaVX, a framework that generates a runtime-adaptive vision architecture from OpenVX applications, which is performant and flexible. It (1) verifies the user implemented OpenVX applications, partitions them into task graphs and creates their meta-data (2) maps these tasks to physical nodes, creates a schedule, and clusters and places the nodes within a partition-based topology (3) creates the hardware architecture, including additional components required to generate a runtime-adaptive system. These components contain runtime configurable network adapters that can prevent deadlocks, a controller for direct memory access, and a manager that configures both and maintains the schedule. The architecture is designed for applications with high data rates and low synchronization overhead. The evaluation shows a low latency overhead of 0.006% added by the architecture, while resource consumption is more than halved compared to a design consisting only of accelerators.