Stafford born inside forward Charlie Baker began his football career with hometown club Stafford Rangers before joining Stoke in April 1889, making his Football League debut at Accrington the same month. The following season he appeared fairly regularly, scoring twice in 16 appearances as Stoke finished at the foot of the Football League table and failed to be re-elected for the following season. As a consequence, Stoke played 1890-91 in the Football Alliance, finishing the season as Champions.
In August 1891, he moved to Wolverhampton Wanderers where he played alongside Will Devey and England international forwards Harry Wood and Robert Topham. In 1891-92 he was a virtual ever-present, making 24 appearances in the League scoring 5 goals, as well as scoring 4 goals in 4 FA Cup appearances. The following season, he lost his place to Joe Butcher and returned to Stoke in January 1893 after 10 goals in 41 appearances.
Although he made four appearances for Stoke at the end of the 1892-93 season, he made no appearances at all in the first team in the following season, presumably as a result of injury.
On 26th April 1893, he was part of a Stoke side who were invited to play a friendly match against Southampton St Mary’s, played at the County Cricket ground in Northlands Road, Southampton. The Saints, who included the 18-year-old Charles Miller (considered to be the father of football in Brazil) in their line-up, were “outplayed fairly and squarely on every point”, although the spectators “thoroughly enjoyed the exhibition” and looked forward to witnessing “more matches of a similar character” in future. Also playing for Stoke were Alf Littlehales, Willie Naughton and Lachie Thomson, all of whom were to move to Southampton within two years.
In the summer of 1894, along with fellow Stoke players Lachie Thomson and Alf Littlehales, he was persuaded to move to the south coast, where Southampton were about to embark on their first season in the new Southern League. He was appointed the first team captain and “his surges down the right wing made him a favourite with the Southampton faithful”. He scored a hat-trick in an FA Cup qualifying match at the Antelope Ground against Reading on 3rd November 1894, as well as scoring in the next two matches as the Saints progressed to the first round proper where they lost 4-1 to Nottingham Forest. Baker went on to score a total of twelve (six league and six FA Cup) goals that season, making him the club’s top scorer.
According to Holley & Chalk, he was “a neat dribbler, (who) had a firm command over the ball and could centre with precision” In 1895-96 he was an ever-present as Southampton finished the season in third place. His Southampton career covered 42 first team appearances, in which he scored 17 goals. At the end of the season he announced his retirement from professional football; the club presented him with a gold watch, before Baker returned to play for Stafford Rangers before his eventual retirement.