Purpose Since handheld ultrasound devices are becoming increasingly ubiquitous, objective criteria to determine image quality are needed. We therefore conducted a comparison of objective quality measures and clinical performance. Material and Methods A comparison of handheld devices (Butterfly IQ+, Clarius HD, Clarius HD3, Philips Lumify, GE VScan Air) and workstations (GE Logiq E10, Toshiba Aplio 500) was performed using a phantom. As a comparison, clinical investigations were performed by two experienced ultrasonographers by measuring the resolution of anatomical structures in the liver, pancreas, and intestine in ten subjects. Results Axial full width at half maximum resolution (FWHM) of 100 µm phantom pins at depths between one and twelve cm ranged from 0.6–1.9 mm without correlation to pin depth. Lateral FWHM resolution ranged from 1.3–8.7 mm and was positively correlated with depth (r = 0.6). Axial and lateral resolution differed between devices (p < 0.001) with the lowest median lateral resolution observed in the E10 (5.4 mm) and the lowest axial resolution (1.6 mm) for the IQ + device. Although devices showed no significant differences in most clinical applications, ultrasonographers were able to differentiate a median of two additional layers in the wall of the sigmoid colon and one additional structure in segmental portal fields (p < 0.05) using cartwheel devices. Conclusion While handheld devices showed superior or similar performance in the phantom and routine measurements, workstations still provided superior clinical imaging and resolution of anatomical substructures, indicating a lack of objective measurements to evaluate clinical ultrasound devices.