Dietary fiber may contribute to increasing the nutritional value of “unhealthy food”—for instance, spreads with high fat and sugar content. The high amount of fiber and the presence of phenolic compounds, organic fruit acids, and aroma compounds make currant pomace a promising ingredient to be used in a wide range of foods. However, the particle size of this by-product is a key factor influencing texture, rheology, physical stability, and sensory properties of the final commodities. Wet planetary ball milling of seedless currant pomace suspended in oil resulted in particles <30 µm, which is required for a creamy texture. Spread stiffness and viscosity were adapted by lowering the solid fat content in a way that the fruity spreads with 16 g/100 g pomace resembled a sweet commercial nut spread. The pomace showed stabilizing effects, as oil separation was reduced and a viscosity increase during storage was prevented. Principal component analysis after sensory flash profiling of five formulations highlighted differences in fruitiness, sweetness, greasiness, and viscosity. Hence, depending on the pomace level and fat composition in the formulation, the properties of sweet spreads can be specifically designed to fulfill the respective requirements. Additionally, sweet and savory wafer fillings provide great potential to be enriched with fruit pomace.