Frequently we do not leave the past behind. We clasp on to it. We dissect it, and let fears for the future, tempered by the past, unconsciously prevent us from taking up the task eternal.
If the heart wanders or is distracted, bring it back to the point quite gently -- and even if you do nothing during the whole of your prayer but bring your heart back, though it went away every time you brought it back, your time would be very well employed.