you can this by some registry hacks.
see this one.
[HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Extensions\{1943C9E7-44ED-4784-ACF8-DD4FF3FD79A3}]
"Default Visible"="Yes"
"ButtonText"="Wiki"
"CLSID"="{1FBA04EE-3024-11d2-8F1F-0000F87ABD16}"
"Icon"="C:\\Documents and Settings\\Sexie\\Application Data\\Microsoft\\Internet Explorer\\Custom Settings\\Custom0\\Greenie.ico"
"HotIcon"="C:\\Documents and Settings\\Sexie\\Application Data\\Microsoft\\Internet Explorer\\Custom Settings\\Custom0\\Greenie.ico"
"Exec"="http:\\\\www.wikipedia.org\\wiki"
remmeber you need a Unique "CLSID" (the one I used is unique
)
ICON and HOTICON are normal and mouse-over button images.
__________
Create as many buttons as you want.