Because bigger guys are often more generally tense. Being tense means that both agonistic and antagonistic muslces are at least partially contracting. As we all know, movement relies on one of the pairs being more relaxed than the other. Speed of movement dictates that the greater the difference in relaxation/flexion, the greater the speed. If both sides are partially tensed, the difference between them is reduced and therefore, so is the speed.