If you want to learn Web Design / Web Development, HTML, CSS, Javascript, jQuery are the basics.
For Web Development, you can learn PHP (Open Source), ASP.NET (Microsoft), JSP (Need to know Java). But web development is not limited to this, you can also use Python, Ruby, Perl etc.
Generally Windows Hosting are priced higher than PHP (Linux) Hosting. [NOTE : Linux Hosting doesn't means you can't access the site from Windows PC. Hosting has nothing to do with who / from what OS the site can be accessed]
So if you love open source more go for PHP, else you can try ASP.NET too. Generally max sites you see on internet are on PHP. It's not like PHP is tougher than ASP.NET. It's just personal preference.
So I'd say start with PHP, learn the basics of web development and then you can always learn ASP.NET, JSP, Python, Ruby on Rails to expand your knowledge. Doesn't matter if you have prior programming experience or not, you can start with PHP.
In that case right now bookmark W3Schools Online Web Tutorials. Here you'll get all the resources to learn HTML, CSS, Javascript, jQuery, PHP etc. and much more. Be sure to learn them before moving on to PHP.
For Web Hosting, you just need to buy a domain and hosting space and using FTP Programs (E.g. Filezilla) you just need to transfer the files to the server. Generally the domain is pointed to your hosting space by DNS Mapping. Though there's nothing rocket science there, but s long as you are buying the domain and hosting from same provider you don't need to understand core level right now. But ofcourse for your knowledge you can learn the theory to get the idea of HOW WEB WORKS..!! For more take a look at Web Hosting Introduction
Hosting and Domain comes for free too, but they are not at all reliable. So at the beginning of your experiement you can use Hostso / Hostdime, they are real cheap but reliable. And once you develop some professional site and looking for professional hosting you can look at Hostgator.
Hope that helps.