The present paper proposes a Bayesian framework for inverse problems that seamlessly integrates optimization and inversion to enable rapid surrogate modeling, accurate parameter inference, and rigorous uncertainty quantification. Bayesian optimization is employed to adaptively construct accurate Gaussian process surrogate models using a minimal number of high-fidelity model evaluations, strategically focusing sampling in regions of high predictive uncertainty. The trained surrogate model is then leveraged within a Bayesian inversion scheme to infer optimal parameter values by combining prior knowledge with observed quantities of interest, resulting in posterior distributions that rigorously characterize epistemic uncertainty. The framework is theoretically grounded, computationally efficient, and particularly suited for engineering applications in which high-fidelity models -- whether arising from numerical simulations or physical experiments -- are computationally expensive, analytically intractable, or difficult to replicate, and data availability is limited. Furthermore, the combined use of Bayesian optimization and inversion outperforms their separate application, highlighting the synergistic benefits of unifying the two approaches. The performance of the proposed Bayesian framework is demonstrated on a suite of one- and two-dimensional analytical benchmarks, including the Mixed Gaussian-Periodic, Lévy, Griewank, Forrester, and Rosenbrock functions, which provide a controlled setting to assess surrogate modeling accuracy, parameter inference robustness, and uncertainty quantification. The results demonstrate the framework's effectiveness in efficiently solving inverse problems while providing informative uncertainty quantification and supporting reliable engineering decision-making at reduced computational cost.