SB 6.1.44

sambhavanti hi bhadrāṇi
 viparītāni cānaghāḥ
kāriṇāṁ guṇa-saṅgo ’sti
 dehavān na hy akarma-kṛt
Word for word: 
sambhavanti — there are; hi — indeed; bhadrāṇi — auspicious, pious activities; viparītāni — just the opposite (inauspicious, sinful activities); ca — also; anaghāḥ — O sinless inhabitants of Vaikuṇṭha; kāriṇām — of the fruitive workers; guṇa-saṅgaḥ — contamination of the three modes of nature; asti — there is; deha-vān — anyone who has accepted this material body; na — not; hi — indeed; akarma-kṛt — without performing action.
Translation: 
O inhabitants of Vaikuṇṭha, you are sinless, but those within this material world are all karmīs, whether acting piously or impiously. Both kinds of action are possible for them because they are contaminated by the three modes of nature and must act accordingly. One who has accepted a material body cannot be inactive, and sinful action is inevitable for one acting under the modes of material nature. Therefore all the living entities within this material world are punishable.
Purport: 

The difference between human beings and nonhuman beings is that a human is supposed to act according to the direction of the Vedas. Unfortunately, men manufacture their own ways of acting, without reference to the Vedas. Therefore all of them commit sinful actions and are punishable.