Only problem I see with manipulandum is no one would know what you mean. Counter would work for, maybe, something like a backgammon piece or a Monopoly piece, something that marks your progress in a game where you proceed around a game board.

Perhaps you could select from this list of more specific terms that would be in the group you are trying to find a word for:

baseball
basketball
hockey puck
tennis ball
chess piece
poker chip
backgammon piece
mahjongg tile
checker (what do you call these when the game is called draughts?)