Ideally, higher-glycemic carbs should always be eaten closer to your workout. That being said, quite a few guys on here (myself included) don’t think you need to worry about avoiding things like potatoes and rice too much. It’s things like refined flours and added sugars that you should really worry about.
More to the point though, food does not cause weight gain unless you eat too much of it. Your goal is to lose fat, correct? As long as you’re getting enough protein and some fats, eating carbs (even white rice and potatoes) in moderation, whilst maintaining a caloric deficit, should make you lose fat.