In this work, we investigate the ligand exchange of cetyltrimethylammonium bromide (CTAB) with bovine serum albumin for gold nanorods. We demonstrate by surface-enhanced Raman scattering measurements that CTAB, which is used as a shape-directing agent in the particle synthesis, is completely removed from solution and particle surface. Thus, the protein-coated nanorods are suitable for bioapplications, where cationic surfactants must be avoided. At the same time, the colloidal stability of the system is significantly increased, as evidenced by spectroscopic investigation of the particle longitudinal surface plasmon resonance, which is sensitive to aggregation. Particles are stable at very high concentrations (c_Au 20 mg/mL) in biological media such as phosphate buffer saline or Dulbecco's Modified Eagle's Medium and over a large pH range (2-12). Particles can even be freeze-dried (lyophilized) and redispersed. The protocol was applied to gold nanoparticles with a large range of aspect ratios and sizes with main absorption frequencies covering the visible and the near-IR spectral range from 600 to 1100 nm. Thus, these colloidally stable and surfactant-free protein-coated nanoparticles are of great interest for various plasmonic and biomedical applications. (Figure Presented).