Microtomography methods allow the representation and analysis of the inner constituents of material minimised structures. In the scope of CRC/TRR280, an image-based framework to investigate carbon-fibre reinforced concrete structures has been developed to determine the position of the carbon rovings as well as their geometry. In order to allow the use of the exact geometry information, the scaled boundary isogeometric analysis is used. Segmentation methods applied to the voxel data facilitate the identification of the components, carbon rovings and concrete. Since the density of carbon fibre and some concrete constituents is rather similar, a segmentation using simple thresholds is ineffective. Instead we apply a deep convolutional neural network for segmentation. In this contribution, a 3D U-Net [1] is used to examine volumetric reconstructions of carbon-fibre reinforced concrete. In a next step, surface extraction methods are applied to the resulting binary voxel space to estimate the shape of the carbon rovings. The obtained geometric description can be directly used for investigations using the scaled boundary isogeometric analysis [2]. This method represents solids by their boundaries, which are described by NURBS functions. Hence, the surface extraction from the previous step is incorporated in the analysis process. For the analysis star shaped domains are presumed, which is ensured by subdivision. The geometric description of the material is embedded in a homogenisation approach which yields the stress resultants for the analysis of shell structures. The homogenisation utilises a representative volume element (RVE), for which the extracted geometry data is employed. The numerical homogenisation determines the equivalent homogeneous material parameter, which will be used on the macroscopic scale. The macroscopic structure is modelled using shell elements. The presented method yields a general framework for image-based modelling and analysis. By the enrichment of the model with detailed geometry data, the accuracy of the analysis can be improved.