There is absolutely no need to clean a Pick Bag because the grip and tension will always remain the same but if you want them to continue to look like new you can clean them from time to time.
The best way I have found is to use a fluff less cloth with a brake cleaner. A brake cleaner works perfectly with natural organic rubber.
Here you will find a link to the product I use.
It is a German product but I am sure you will find similar products worldwide.
I wish you all the very best with your guitar playing.