If you simply want the experience, whenever.
If you want to be competitive, you should look to build size so that when you are lean, you will be bigger (obviously). Yes, symmetry is important, but symmetry doesn't mean much if you're extremely small and underdeveloped.
Also, in terms of simply achieving your best physique, it might be better to work towards building your body rather than trying to cut fat/diet prior to doing that.
Just my thoughts.