The presence of a fever is not a sensitive marker for an abscess. A better sign is the presence of pain, usually described as a dull ache. Also, check to see if the lump is hotter and redder than adjacent skin.
Even if it doesn't have these signs, chances are its still an infection. You need to have it cut open, the pus drained and take some flucloxacillin.
You documented your positive symptoms and relevant negatives well - kudos. Certainly injecting regularly makes one suspicious of an abscess first thing, but the distance from the injection site combined with the proximity to the muscle insertion makes one think it could be some sort of musculoskeletal flare up, especially if the pain became evident a few hours after working out.
The lack of an imprint in the skin when pushing on it does not indicate its not an abscess. An abscess, if anything, will rebound the skin faster, somewhat similar to a water balloon. You could give it more time, or you could go into the urgent care clinic. It all depends on how sick you feel you are. Caution is always the best route for someone who has not had or seen many abscessed before, they can become quite serious. If you feel sick, or like the pain is more profound than a simple injury, go in to casualty and don't let any internet doctor dickhead tell you otherwise.
Depending how deep the abscess is, you can try and transilluminate it. This involves getting a small torch and shining it at point blank range against the skin, on an angle. If it is an abscess, and it is superficial enough, the fluid inside the cavity will glow red. Google search the word to get an idea of what it looks like. If this test is negative, it should not serve to discourage you from attending casualty, it is not to be used to rule out an abscess. If it is positive, it should most certainly encourage you to go, as it quite a specific test.