Magical damage is caused by raw magical energy, typically inflicted by certain arcane spells such as the magic missile line of spells: magic missile, Isaac's lesser missile storm, and Isaac's greater missile storm.
There tend to be few items granting resistance or immunity to magical damage. Magical, divine, and positive energy damage are the three damage types for which there is no protection via spells.