Sounds to me like you lower back is getting worked fine. Are you squatting and deadlifting? That's probably enough. I'd basically not worry about it, as a beginner, unless you're having trouble lifting (form issues, pain). If you're having trouble lifting it could be because your lower back is weak, or it's overworked. What does your current program look like?
Compound lifts such as squats and deadlifts should be sufficient in working your lower back. I highly recommend hip mobility exercises as well. They are crucial in maintaining flexibility and overall back health.