On September 8, 1916, the New York Yankees came to Philadelphia to play the hapless Athletics. The fans sitting in Shibe Park for the rain-delayed game were outnumbered by the men in uniform. In fact, when the Yankees were at bat, they had more players on the bench than fans in the seats. After an 8-2 win over the Yankees, for the only time in major league history, a team was two games back for every fan in attendance. 46 games back of the first place Red Sox and an announced attendance of 23.
The modern record unofficially was set on August 24, 2011. The head count, done by a tweeter, was 347 at a Reds-Marlins game at Sun Life Stadium. Hurricane Irene caused a large evacuation of the already-miniscule Marlins fanbase. The official attendance is far higher, as it is based on tickets sold, which are much more than 347.
Officially, the “modern” record also involves the Athletics. On April 17, 1979 only 653 people watched the hapless A’s (they would lose 108 games in 1979) play the Seattle Mariners (who would lose 95 games that year). The A’s won that game, too: 6-5.