"That's quite alright." If he had at all expected to be invited in he shows no trace of disappointment. Really, he had expected no such thing. If an invitation had come, he would have likely declined.

"Though if you wanted to go flying some day, care to give me a day when you're free and have the time?"