Transition metal oxides with a perovskite-type structure constitute a large group of compounds with interesting properties. Among them are materials such as the prototypical ferroelectric system BaTiO₃, colossal magnetoresistance manganites and the high-T_c superconductors. Hundreds of these compounds are magnetic', and hundreds of others are ferroelectric², but these properties very seldom coexist. Compounds with an interdependence of magnetism and ferroelectricity could be very useful: they would open up a plethora of new applications, such as switching of magnetic memory elements by electric fields. Here, we report on a possible way to avoid this incompatibility, and show that in charge-ordered and orbitally ordered perovskites it is possible to make use of the coupling between magnetic and charge ordering to obtain ferroelectric magnets. In particular, in manganites that are less than half doped there is a type of charge ordering that is intermediate between site-centred and bond-centred. Such a state breaks inversion symmetry and is predicted to be magnetic and ferroelectric.