Panagoulias played his football for Aris Thessaloniki, in the 1950s, part of that time as captain of his team. After his University studies, he moved to the United States in the early 1960s, where he attended more University in New York and coached the New York Greek Americans to three consecutive US Open Cup titles in 1967, 1968 and 1969.
He returned to Greece as the assistant coach of the Greek National Football Team, under the famous Northern Ireland coach Billy Bingham, in 1972. The following year, he became head coach of the Greek team, leading Greece to the Euro 1980 finals, in Italy.
Panagoulias returned to Greece to once more coach Olympiakos, earning the championship title in 1986 and 1987. Afterwards, he coached Aris from 1987 - 1990, returning to the Greek National Football Team in 1992.
Panagoulias led the team to its first appearance in a World Cup final in 1994, however, he soon afterwards was forced to resign after a despirited, dismal performance by the national team in the US, amid much criticism from his players - especially Stelios Manolas - for his handling of the entire situation.