A.: Business is business. Speak to your friend directly about your decision not to work on the project with him. Be clear that you do not want to work on this project and out of respect for him you want to let him know right away. Thank him for extending the invitation to you.
You can never control how people will respond to you. You can improve your chances of having a positive outcome, though, by being forthright and kind as you communicate with him. Tell him you value your relationship too much to be indecisive, and this particular project is not right for you at this time. If he has a problem with that or tries to persuade you further, stick to your decision. If you can think of someone to refer him to who may be interested, do that. But do not give in. Trust your instincts. Time will tell whether he gets over it or not.