Yes, a castle was built to give time to call in reinforcements.
But with the option of the attacker to just attack, what benefit is there to the attacker to siege? Why not just attack right now?
Historically, an attacker would besiege a castle in the hopes that they would just give up without a fight. Many times, a deal was struck between attacker and defender that if relief did not come within a certain amount of time, the defender would surrender. This way, both attacker and defender would come out (relatively) unharmed.
In KoH, though, there is absolutely no benefit to the attacking side to siege.
M2:TW as this same problem. I would siege a castle just long enough to build siege equipment and then attack. Waiting was useless as the defender would always sally forth just before the timer expired.
There needs to be a benefit to the attacker for besieging. Such as the defender should losing troops as the siege goes on.