How Many Twitter Lists Is Too Many?

Twitter lists are a very good organizational tool. They let you keep tabs on the people you don't necessarily want to follow, and they can keep you on top of a very specific subject.

I've made lists for things like rappers, people I know from college, and comedians. That way if I don't want these users to dilute my main feed, I still track them. Twitter only lets you make a max of 20 lists, but is that too many in the first place? Or do you need more?