There is no bad weather, only insufficient layers — and a waterproof shell you actually trust.
The three layers
Base layer of merino or synthetic, never cotton. Mid layer of fleece or light down. Outer shell that is genuinely waterproof and windproof, not water-resistant. This is the whole system, summer and winter.
Footwear
Waterproof walking boots with ankle support. Iceland's best short walks are on wet volcanic gravel. Add microspikes between November and March — they cost little and prevent the most common injury we hear about.
Swimwear, always
Geothermal pools are the national pastime and every town has one. Bring a quick-dry towel; many pools charge for rental.
For the northern lights
Aurora viewing means standing still outdoors at -5°C for an hour. Thermal trousers, a hat that covers the ears, and gloves you can operate a camera in. A tripod if you are serious.
What to leave at home
An umbrella — the wind will destroy it within a day. Heavy denim, which stays wet. More than one pair of going-out shoes.