The family of human APOBEC3 (A3) restriction factors is formed by seven different proteins, A3A–D and A3F–H. Among these A3s, A3B harbors strong restriction activity against several retroviruses, such as SIV, and MLV. How lentiviruses and other retroviruses, prevalent in many primate species, counteract A3B is poorly understood. In this study, we found that A3B strongly inhibited SIVmac and HIV-2 infectivity, which was antagonized by their Vif proteins. Both SIVmac and HIV-2 Vifs diminished the protein level of A3B in viral producer cells, and hindered A3B incorporation into viral particles. We observed that HIV-2 Vif binds A3B and induces its degradation by assembly of an A3-Vif-CUL5-ElonginB/C E3-ligase complex. A3B and HIV-2 Vif localize and interact in the nucleus. In addition, we also found that the accessory protein Bet of prototype foamy virus (PFV) significantly antagonized the anti-SIVmac activity of A3B. Like Vif, Bet prevented the incorporation of A3B into viral particles. However, in contrast to Vif Bet did not induce the degradation of A3B. Rather, Bet binds A3B to block formation of high molecular weight A3B complexes and induces A3B cytoplasmic trapping. In summary, these findings indicate that A3B is recognized by diverse retroviruses and counteracted by virus-specific pathways that could be targeted to inhibit A3B mutating activity in cancers.