Goumas was born on May 24, 1975 in the village of Ampelonas, near Larisa. He was one of the many players to emerge from Panathinaikos FC's youth academy and was in their first team from 1994 until 2009 when he retired. In total, he appeared in 277 league games for the greens scoring 27 goals.
He was known for his tough tackling, excellent aerial ability and his habit of scoring goals off set pieces.
Goumas debuted with the National Football Team of Greece on February 3, 1999 against Finland and was capped a total of 45 times. He was part of the Greek squad that won the Euro 2004 Championship but did not participate in any games.