Getting your fruit and veg from a market which sells the excess from supermarkets is a great option.
It is less than half the price and you will start doing something about all the food wastage western countries have an issue with.
The trade off is it's not as perfectly shaped, may have marks, less shiny, less well presented. Tastes the same.
If you have good freezer space, a trip down to the butcher and ask them to give you some huge chunks of an animal deboned (think 20kg) which you then slice into steaks will get you faaaar cheaper price - again upto half price.
A 10kg sack of rice from an asian grocer os stupid cheap per kg. While youre there, miso, noodles, seaweed, curry pastes, that crap is tasty and cheap as anything.
Oily fish like sardines are awesome and cheap as anything. Even the canned stuff.
Pickled anything is usually a nice combination of tasty, cheap and pretty good for you.
If you're really manly, chicken hearts are basically free. Some skewers, capsicum, onion, zucchini, olives, a grill, some wicked hot sauce and you will be in heaven.
Just stay away from anywhere jerks who wear skinny jeans, big sunglasses, crappy beards go for authentic ingredients (especially butchers who sell gourmet sausages - seems to attract the arseholes). They usually get fucked over by shop owners, driving the price up for everyone.