There are many aspects of a loan agreement besides just the rate. Covenants, guarantees, pledged assets, remedies, etc. All of these issues are subject to negotiation.
The most important factor in getting what you want is… time. If the bank knows your back is up against the wall time-wise, they will be less flexible because you don’t have time to explore alternatives. If you have the ability to say, “no” and walk away, the balance of power shifts to you. Like all negotiations, the person who cares the most about reaching an agreement is the one who usually doesn’t get the deal they want.
In other words, you have to care…but not that much. Time is what allows you to have that attitude. This is why you must negotiate with the bank early when taking on debt. Don’t wait until you find the actual building, acquisition, equipment or whatever else you are financing with debt. Waiting until you are ready to buy before you approach the bank guarantees that you will not get the deal you want.
That being said, all the time in the world won’t help you strike a great deal if the bank views you as a marginal credit.