Someone calling you a pencil dick doesn't cut you. Its just words, not grounded in anything the speaker knows about you. What's scary about rape threats is that it points out an uncomfortable truth: the 95th percentile women is about as strong in the upper body as the 5th percentile man. The guy making the rape threat knows that and so does the target. Its backed by reality in a way most comments directed at men aren't.
It's also backed up by a culture that tells women they're responsible for their own assault/rape/harassment, so defying these threats means you're also defying the people who say it's your own fault, go somewhere else, you're asking for it, etc.