I did the same thing to my cups of pencils and pens. I can only use one writing implement at a time but I had two coffee cups stuffed with items. I dumped those and kept out only a few pens and pencils (still more than I probably need but about 10% of what I had out) and stored the rest.
My goal was to reduce the noise from anything visible in my workspace and it took most of the day to do this in between work and other things going on. Next, I have to get into the drawers and the files - all those things somewhat out of site but not out of mind. Even if I can't look up from my desk and see inside the storage items that contain clutter, I know it's there and it weighs on me.
As I was considering the plan for what I want to accomplish today, I realized that noise is not the best term to describe what I'm trying to eliminate. A better term is interference because all of these extra unused items sticking out of everything and all of the unneeded words yelling down to me from the bulletin boards interfere with the flow of what I am really wanting to accomplish and the peace of just sitting in silence during those breaks in between trying to accomplish things.
One final thought - it's obvious that interference comes from many different sources. While I am currently focused on my environment, I realize that a major source of interference is social media, which I have talked and written about many times and I've been working on figuring out the best way to make social media constructive interference for a while. I'll write and talk more about that once I have a good system in place for that. It, like most things, is a continuing work in progress.