I've fallen into the toilet about three times due to lazy boys who can't put the seat down. When it's 3am, and it's dark, you're not paying attention to that stuff. So I say seat down when not in use.
I keep all the seats down along with the lid. I have a lid cover on all of the toilet seats in the house, so keeping them down makes the bathrooms look neater.