You are able to launch Firefox so where is the problem ...?
Though I am not sure but my understanding is that Firefox keeps itself loaded in memory at all time so it opens faster than IE but utilizes more resources. Thats probably why it's always up there.
In case you are facing issue's with killing the Firefox Process then try this :
Create a Kill Firefox.bat file.
1. Open up notepad.exe
2. Paste in taskkill /F /IM firefox.exe
3. Save file as Kill Firefox.bat
4. Create a shortcut to the file or drop it in your quick launch bar
Now, surf the net, when Firefox hangs just click your new icon or press your shortcut and Firefox is gone. Reopen Firefox and restore your sessions or start a new session. If it doesn’t open Kill Firefox.bat one more time and all is good.
Source: *2paperclips.com/2006/11/20/firefox-hanging-end-the-firefoxexe-process-in-one-step-kill-firefoxbat/