This chapter deals with computational techniques developed for the most common vision-based interventional imaging modalities using visible spectrum energy, white light endoscopy and microscopy. While interventional imaging is progressing rapidly to include different energy spectra and exogenous signals (i.e., tracer-based), the development of appropriate computational methods is lagging behind. This is mainly due to the lower adoption of such imaging technologies. Vision-based techniques offer a number of possibilities that rely on this sensory input. Such techniques are employed to investigate, replicate, and potentially standardize human actions; such as the recognition of pathological tissue and critical structures. Additionally, a notable usage is to extend current intraoperative capabilities by linking systems and pipelines to robotic instruments and/or advanced navigation systems. In this chapter, we discuss the capabilities of such techniques to acquire various metrics (e.g., geometry) from the surgical site and to automatically understand image data through classification and detection using vision-based methods. In turn, enabling computer-assisted interventions (CAI).