Initially an inside forward, and later in his career a wing half, Motherwell born Joe Devine played junior football for Motherwell Watsonians in 1922 and Cleland Juniors in 1923 before joining Scottish League club Bathgate in 1924, from where he joined First Division Burnley a year later, making his Football League debut against Leicester City in September 1925. He became a Burnley regular from the end of October 1926 and was an ever present in 1928-29, scoring 12 goals, as Burnley narrowly avoided relegation for the second season running. However before that relegation came later in 1930, he had moved to Newcastle United in January 1930 after 29 goals in 121 appearances, scoring 8 goals for The Toon to help them preserve their own top flight status as they finished 19th including a goal in a 3-0 win at Burnley. However the next season he found himself out of favour at Newcastle and in January 1931 he joined rivals Sunderland, playing in their FA Cup semi final defeat to Birmingham that March and spending 2 and a half years on Wearside, scoring 7 times in 77 appearances. In May 1933 he dropped down 2 leagues to join Queen’s Park Rangers, scoring 10 goals in 65 appearances, but in January 1935 he returned to the First Division with Birmingham, for whom he scored twice in 56 games over the next 2 and a half seasons. In May 1937 he played a final season at Chesterfield before retiring, having made 27 appearances for The Spireites.
After his playing career, he became manager of Valur and Iceland both sides of the Second World War.