For me its my surroundings, and how I generally feel, sometimes ill use colors I made as I was creating a piece for a palette aswell.
tomfoolery, that's how "Oops! All Yellow" and "Bird Poop Palette" were conceived
e4 e5 Ke2
Instead of focusing on palette inspiration, one should aim for art inspiration. What do I want to create? What do I want to achieve with my creation?
This being said, there are times when an artwork or photo has interesting colors you might want to explore further. This is a great way to look at how colors interact and how to implement them.
Essentially it's the artwork that dictates the palette's quality.
Most of mine are based of of real-world limitations, but with alternative colors. My color counts range from 3 to 20. Common counts would be 6, 8, 12, 15-16.