I think banks came from a system were there were no checks at all. Upon withdrawal you actually received a loan from the bank where the withdrawal took place. Only later this loan was cancelled against a credit that was on your account already.
Credit card transactions with a credit card that is coupled with a bank account still work this way. Sometimes up to a month the money is not withdrawn from your bank-account.
It makes sense that a loan without collateral is more expensive than a loan with collateral, hence the overdraft penalty.