Rejoicing with Trembling

This morning as I read Psalm 2 for my devotions, I came across verse 11, which reads:

Serve Jehovah with fear, and rejoice with trembling.

I took great pleasure in a command with so much apparent diversity of motives and expressions. However, the notion of rejoicing with trembling might be a bit puzzling. I looked to Calvin and wanted to share what he had to say about this verse.

There is a great difference between the pleasant and cheerful state of a peaceful conscience, which the faithful enjoy in having the favor of God, whom they fear, and the unbridled insolence to which the wicked are carried, by contempt and forgetfulness of God. The language of the prophet, therefore, implies, that so long as the proud profligately rejoice in the gratification of the lusts of the flesh, they sport with their own destruction, while, on the contrary, the only true and salutary joy is that which arises from resting in the fear and reverence of God.

[Edit: John Piper did a way better job of discussing this exact same verse; but I didn't know that when I originally posted this.]