Speaking at the premiere of his new film Hancock, Smith explained that he had started fencing with the pair.
He told The Mirror: "Tom has a room for training. We don't get enough time to hang out, just us three guys, so this is his way of getting together and bonding. David and I go to his home and just do fencing. It's a lot of fun.
"We wanted an activity that was strenuous, but we're getting older - we have to think about slowing down."
Smith joked: "We've got to watch our joints. Especially my knee."