But hormones, not just activity, control what you expend and what and how you eat effects your hormones. If you cut the calories of a hibernating type of squirrel, even by as much as half, going into winter they will still gain just as much weight and fat as squirrels eating 2 to 3 times as much. With certain hormone amounts, obese rats can starve to death while remaining obese.
That "expend" term in your equation has dozens of variables that effect and control it. And it is at least as important as calories eaten. And really and truly, on a long term scale, the hormones are more important because they control your appetite mechanisms that will eventually override even the best self control.