You will re-inhale a little bit of CO2, but only the CO2 that is in the device itself. So it's a very small amount. A similar amount that a scuba diver would get from the dead air space of their regulator, but less than a snorkel.
When you breath out, the very end of your exhalation will still be in the device. So when you start to inhale the next breath of air, you first get the air that's in the device before fresh air enters. So whatever CO2 is in the airspace of the device will be inhaled. It's such a small amount that it's not a concern what-so-ever.