What you're looking for is an iframe, you'll find tons of tutorials about that, but here's a quick example:
In your page e.g. index.html:
<html>
<head>
</head>
<body>
<div id="sidebar">
<iframe width="100" height="500" src="sidebar.html"></iframe>
</div>
<div>
Your main page content
</div>
</body>
</html>
In your sidebar.html:
<html>
<head>
</head>
<body>
<div>
Sidebar content goes here...
</div>
</body>
</html>
Now anything you update on the sidebar.html will be updated on index.html.
Add this iframe to every page that you want to contain this sidebar.
Hope this helps...