Karoulias was born on May 25, 1953 in Samos. He started his career at AEK in the 1974 season, participating in 3 league games. In the Summer of that year, he was sent to Apollon Athens as part of the deal that sent Christos Ardizoglou the other way.
Karoulias broke into the first team of Apollon, playing on the left side of defence. In 1980, he was acquired by Panathinaikos. With the greens, he won the league title twice (1984, 1986) and the Cup four times (1982, 1984, 1986, 1988). He ended his career in 1988, having appeared in 259 Alpha Ethniki matches with 10 goals to his credit.