I think I found an answer here: http://support.microsoft.com/kb/195730
Excerpt:
The offline mode that Internet Explorer starts in is determined by the registry data value "GlobalUserOffline." Setting this data value to 1 will cause Internet Explorer to start up in offline mode. Because this value is written with the current offline state when Internet Explorer is shut down, it is necessary to toggle the value manually through the Registry Editor before Internet Explorer is started, or to run a program that use the Registry APIs to change the setting.
For Windows 98, the "GlobalUserOffline" data value can be found under the following key:
HKEY_USERS.DEFAULT\Software\Microsoft\Windows\CurrentVersion\Internet Settings
Under Windows NT 4 and Windows 2000, the key is as follows:
HKEY_USERS\SID\Software\Microsoft\Windows\CurrentVersion\Internet Settings
where SID is the security ID for the user who desires Internet Explorer to startup in offline mode.