Rallis was born in 1794 in Chios. He settled in Constantinople but fled after the local pogrom against the Greeks after the commencement of the Greek War of Independence. He subsequently moved to Syros and, in 1834, to Piraeus.
Rallis became involved in commerce and banking, acquiring a fortune and making a name for himself in local circles.
He served as mayor of Piraeus from 1856 until 1865. His administration was marked by public works among which were the building of public schools and the central marketplace. During his term in office, the foundations of the church of St Spyridon were laid in the centre of the city.