The thermoelectric properties of multi-wall carbon nanotube (MWCNT) -grafted glass fiber yarns (GF-CNT) and their epoxy model composites, as well as of polymer nanocomposites consisting of a polycarbonate (PC) matrix filled with differently functionalized MWCNTs have been examined. The GF-CNT hierarchical multi-scale structures were prepared by dip coating glass fiber yarns in a solution of carbonyl chloride modified MWCNTs; MWCNT-COCl (at a concentration of 0.5 mg/ml) under Ar atmosphere. The resulting GF-CNT exhibited high electrical conductivity (σ?=?2.1×103 S/m) due to the dense MWCNT deposited networks. The fiber surface morphology was investigated by scanning electron microscopy (SEM). The GF-CNT showed Seebeck coefficient (S); S = 16.8 μV/K, and power factor (P.F); P.F = 0.59 μW/mK-2. The high electrical conductivity of the GF-CNT is a key parameter for an optimum thermoelectric performance, since it can facilitate the flow of the thermally induced charge carriers upon being exposed to a temperature gradient. Polycarbonate/MWCNT nanocomposites were prepared by small-scale melt-mixing process using a microcompounder. Unfunctionalized, carboxyl (-COOH) and hydroxyl (-OH) modified MWCNTs were incorporated in PC at a constant amount of 2.5 wt.%, concentration above the electrical percolation threshold. The amount of MWCNTs was kept low to understand the fundamental aspects of their physical properties and their correlation to the composite morphology, as revealed by transmission electron microscopy (TEM). It was found that different functional groups can affect the thermoelectric performance and the conductivity of the nanocomposites. Namely, the highest Seebeck coefficient (S) was found for the composite containing carboxyl functionalized MWCNTs (11.3 μV/K), due to the highest oxygen content of MWCNTs proven by X-Ray Photoelectron spectroscopy (XPS). It is believed that MWCNT-grafted glass fibers as reinforcements in composite structural materials and PC/MWCNT nanocomposites are ideal candidates for large-scale thermal energy harvesting. However, the thermoelectric values are still too low for commercial applications and in the future could be enhanced as will be discussed in this work.