Yes, that's correct. And yes that is common parlance both in epidemiology and statistics.
In regression modeling, you try to explain the variability in the outcome Y with the variability on your predictors X. You therefore regress Y on X.
Cheers,
Francois