They constantly trained; much like our working out today. Practical matters had them building and re-enforcing castles. Knights where obligated to give their overlords 40 days of service each year, and if they had their own families, dwellings they would return and take care of them. Think of it as being in the military reserves. If they were needed in battle - they were all called up. They all lived relatively close by - within a day or two ride away. They were fit warriors. If they did not have places of their own, they stayed on and the castle became part of the regular guard of the family.