This week a candidate asked for my feedback on how to decline an offer while preserving the relationship. I conferred with other experts at BCG, and I think the most important theme that came out of our discussions is to show appreciation for the offer and sincere regret for being unable to accept it.
A carefully-worded thank you note to the section head that expresses your sincere thanks for the opportunity is in order. If there were other individuals at the firm that you connected with, a hand-written or e-mailed note is a nice courtesy. It is important to convey that the decision you made was a difficult one.
Furthermore, I would mention that you hope to run into them professionally in the future. OR tell them that you enjoyed meeting them so much that you would like to keep in contact- then DO! Think of this an opportunity to expand your network of contacts.