sam9s VMWare project! powered by ESXi Hypervisor

sam9s

The High 5 Flyer
welcome to sam9s ESXi VMWare Project! to run all your servers/workstation from one machine

sam9s is back with another weekend project, which basically involves creating an ESXi server and running all your servers or workstations from it.

What is ESXi …

ESXi is an enterprise-class, type-1 hypervisor developed by VMware for deploying and serving virtual computers. Virtual computers are nothing but virtual machines running an OS (windows, Linux, OSx, Android ...etc) just like you will on a physical machine/hardware. Difference is you are running it on a virtualized environment. Advantages are many, but mainly, to emphasize, if you have multiple computers/servers running at your home/office for different purpose, you can run all machines on one single hypervisor (ESXi in this case), and manage all machines from one location. One single dashboard. This saves you, your physical hardware plus makes it easy for you to test/experiment on any given OS for your needs.

What could be the need of Virtual Machines in a Home Entertainment scenario …..

Well, lets see, in the past couple of years since I have been a member of this forum, I have come across numerous occasions where we discuss the need to have a NAS, a media Server, simple file server… etc etc. Particularly Media server, be it for Music or Movies or both. One of the advantages of running a virtual world is you can have multiple machines running all under one roof with only one physical machine running. So you can have one server for Movies and Music (like Plex or Emby) and one as a NAS (like OMV, Unraid or NAS4Free). Or dedicated one to one server for Movies and TV series (like Plex again) and another one for Music (Like Logitech Media Server or Daphile or Subsonic) and a third one acting as your NAS (Like Amahi 8). Possibilities are endless. You can have one Windows machine running for all your experiments or temporary tasks, like you want to test some application and you do not want to install it on your prime desktop/gaming machine. You can install it on your windows virtual machine and test it.

Today I will show you my Home Virtual Lab I created using ESXi and how am I running all the servers I need, in all its Glory.

Before we move ahead, a pic of my ESXi dashboard showing all the servers I am currently running.

6Zu37Wq.jpg


Ignore the 3 servers (domain, application and database) those are for my office purpose. Rest what I am currently using are ….

Plex Server
Home Server
Radio-PC
Fedora Amahi Server.
VPN server


Plex Server/Radio-PC

For plex and Radio I was running a separate physical machine on Gigabyte E350N hardware, the problem with E350N is that the processor horsepower is not up to the mark for plex to do transcoding, and down conversion, hence the experience was not as smooth as I would have hoped for, especially while viewing media remotely via my tab/mobile. And one of the main reason of using plex is transcoding so I can view my media while I am on the go. Same reason I use Logitech media server for my audio need while I am on 3G. My LMS was running on a Rasp Pi, now its on ESXi

So what I did was to uninstall my plex from E350N machine and created a new virtual Ubuntu Server on ESXi (which has a much much better hardware). This Made Plex much more powerful and transcoding ran with its full glory.

Next what I did was to convert my physical E350N computer to virtual ESXi compatible machine and transferred it to my ESXi server (yes you can do that, you can turn any running physical machine to virtual to make it run via VMWare.) This not only freed my one physical machine, but brought it under one roof to be able to manage more effectively. This took care of my Radio PC part. If you remember I run my own Radio Station (sam9sradio). This is on a windows server.

Home Server

Home server is nothing but a generic Ubuntu server 64 bit that I am running. At present it is hosting the Logitech Media Server,(which earlier was running on RPi) but I am planning to run a Mail Server and a Linux based OpenVPN server on it in future. Plus I might also use this linux machine to learn few CLI commands to make my life more linux friendly ..:)

VPN Server

Running a VPN server is very important if you plan to access your home network and resources from internet. Not only it makes it more secure, it also brings everything accessible from within your home network, no need for any dependency on any DDNS service. I initially decided (and I might go with it in future) to have an OpenVPN server running on Ubuntu, but that is far too complex and difficult to configure. So for the sake of simplicity and since I was already running a Domain server I decided to make my already running windows server 2012 machine into a VPN server as well. Another advantage of this approach is, since I am running ADDS on server 2012. Control and authentication of who and how to access my VPN became more controlled and secure.

Fedora Amahi 8

Well this is just for testing, I just wanted to see what Amahi Home Server is, and the best way to do this is via VM. What I think of Amahi is for another discussion or thread, but people interested can visit Amahi Home Server - Making Home Networking Simple

Getting the ESXi running ……

Below is the hardware I am running my ESXi on.

Core i7 2600K@4Ghz
16GB 1600 Mhz DDR3 GSkill
1 TB HDD for VMs
32GB SSD OR
8GB Pendrive for the OS (ESXi)


Before you venture into the world of VMWare or ESXi, you need to understand that more beefy your system is, better and more VMs you can run effectively. A Quad Core 2.0+ Ghz cpu is bare minimum and the more RAM is at your disposal, better it is for your VMs. Again 8 GB is bare bare min, but 16 GB is a good start. 32GB will the sweet spot if you ask me.

Getting the software ESXi package ….

Head over to ….

Login - My VMware

and download two files ..

1. ESXi ISO image (Includes VMware Tools)
2. VMware vSphere Client


You need to be registered and activated in order to get ESXi downloads and free license. Please do it.

Checkout the below snap.

oWkZWrK.png


Apart from this you would need two more apps to start the ESXi implementation.

Rufus
Putty

If you are using windows 7 then you also need to install some kind of mounting application to mount the ISO file. I use virtual clone drive, you can use WinISO or DAEMON lite, or any ISO mounting app.
If you have windows 8 + then you can mount the ISO natively.


The Procedure, Deploying ESXi ….

Assuming you have the two files that we downloaded in the steps above …

Insert the pen drive on a separate desktop/laptol available. Mount the ISO
VMware-VMvisor-Installer-6.0.0.update01-3073146.x86_64.iso Fire up Rufus. Make sure you have Pen drive selected under Device Select the ISO from under format. Rest of the setting should be exactly like the image below …

Click on start and create the Boot Disk

rTmkk0P.jpg


Now we are ready to insert the pen drive in to ESXi server and make the machine boot from it. Before we do that just couple of things you need to take care off. Go into the BIOS of your machine and make sure the below two things are enabled.

Intel Virtualization Technology
No execute memory protection
and of course Boot from USB should be enabled also.


Once this is all done power on the machine and let it boot from Pendrive. There are couple of entries you might want to do. Change the IP to static and also provide a hostname to the server along with a Fully Qualified name.

Go to configure management network.
ipv4 configuration
set static ipv4address
Choose as address that belongs to your home network ip pool,and is NOT taken by any other device. (192.168.x.y)
Then move to DNS configuration and change the hostname to esxi
Then choose a custom suffix DNS, like I have lab.local for my ESXi Host.


For some supporting images of the installation you can go to …

ESXi Deployment - Album on Imgur

Thats it the hypervisor is ready. Next step is to install, the VMWare desktop client to manage the hypervisor.

Double click VMware-viclient-all-6.0.0-3016447.exe
Let the setup run by default “next next” selection
Complete the setup and open the vsphere client.


When you open vsphere client you need to put the credentials and ip you provided while configuring ESXi hypervisor. Once done you will be presented with the console.

js2A6k7.jpg


V3joLNK.jpg


You can start creating VM and start managing data stores, but this is not what I have done VMWare has introduced a web client (still in beta stage) to manage your VM and everything. That is what I have been using and that is what I will suggest you do as well. The reason being the vsphere desktop client will be phased out pretty soon and only web client will be supported.Plus web client is super fast and responsive, unlike desktop client

Before we move ahead to install web client, I would suggest to create atleast one data store using the existing vsphere desktop client ….

Creating a datastore :: A datastore is a place on your HDD where all your VMs and ISO files are stored (ISO are the setup ISO created by your OS setup disks (windows,linux etc)

The procedure is nicely explained in this video ...

https://www.youtube.com/watch?v=8srGqgIcsqs

Now we have the data store installed let's install the web client.

To install web client we need a program called putty (that we downloaded earlier). Using putty we will log in to the CLI of ESXi. Fire up putty and put the ip of your ESXi server under ip and simply click connect. You will be asked to accept the SSL which once done you will be presented with the login prompt. Put the credentials you created while installing ESXi (same with which you logged in to vcenter client). You will be presented with the CLI command prompt.

IAouCpM.jpg


Past the below command and press enter to install the Web GUI for vcenter client

esxcli software vib install -v http://download3.vmware.com/software/vmw-tools/esxui/esxui_signed.vib

Once the install is complete simple open your browser and enter the below url …

https://youresxihostip/ui (example … https://192.168.x.y/ui)

You will be presented with the login screen, put the credentials (same while creating ESXi and same used to log in to vcenter desktop client)

0BmkkLV.jpg


Creating VMs …..

Before we start creating VM we have to do couple of basic activities …

Upload ISO to a datastore :: A datastore is a place on your HDD where all your VMs and ISO files are stored (ISO are the setup ISO created by your OS setup disks (windows,linux etc)
Click on storage
Click on datastore you created.
Click on datastore browser.


Here you find all activities you can do like upload files and create folders etc. I would suggest to create an ISO folder and upload all OS ISO to it. Check out my folder structure.

wnwa7n7.jpg


Creating your first VM ….

Well instead of sharing snaps, created a small video for the same.

https://vimeo.com/154181581

This way you can create as many VM as you want, depending on the resources you have at your disposal your ESXi can manage unlimited number of VMs ….

For any queries and suggestions, start filling the thread …… :)

Regards
Sammy
 

Hrishi

******************
Sam, I am planning to use hypervisor type 1 on my laptop with configuration i7 3610qm hyperthreading enabled and 8 gigs of ram.

I haven't used type 1 hypervisor on a personal laptop before so what concerns is that will the entire disk be wiped for eEsxi install ?
And the guests on this host will be able to use the 2gb vram of dGPU ?

I will be accessing them from my Desktop.

Sent from my LG-E970 using Tapatalk
 
OP
sam9s

sam9s

The High 5 Flyer
Sam, I am planning to use hypervisor type 1 on my laptop with configuration i7 3610qm hyperthreading enabled and 8 gigs of ram.

I haven't used type 1 hypervisor on a personal laptop before so what concerns is that will the entire disk be wiped for eEsxi install ?
And the guests on this host will be able to use the 2gb vram of dGPU ?

I will be accessing them from my Desktop.

Sent from my LG-E970 using Tapatalk

Hi ...Hrishi,

You do realise that with type 1 hypervisor you would not be able to use your machine for anything, except for running ESXi, The entire disk does not get wiped out unless you plan to install ESXi on to the internal HDD, mine is currently running off a USB Stick. However when you will create a Datastore (you will have to, if at all you wanna use ESXi), the OS will wipe it out and clean the HDD to get it ready for storage. Having said that even if you run the hypervisor via a pen drive, as I mentioned your system will be locked at a headless state, so you would not be able to use the internal HDD, except for using it as a datastore for ESXi.

Coming to GPU, for that to work you have to passthrough the video card to the VM created via ESXi console. If you do not know how to do that I can provide pointers, you can google it as well. Also v.imp, you CPU should support VT-D for the passthrough to work.
If you ask me, You would not want to go with all that pain to get the GPU working unless you plannning do some GPU intensive work like Photoshop, 3D max, etc which involves rendering and stuff like that. No high end Gaming plz.. VMWare is not meant for high end gaming, even though I might work but the experience will not be smooth enough.

How many VMs you planning to run? If its more than 3, I would recomend to upgrade to atleast 12GB ram...8+4

Regards
Sammy
 

Hrishi

******************
Hi ...Hrishi,

You do realise that with type 1 hypervisor you would not be able to use your machine for anything, except for running ESXi, The entire disk does not get wiped out unless you plan to install ESXi on to the internal HDD, mine is currently running off a USB Stick. However when you will create a Datastore (you will have to, if at all you wanna use ESXi), the OS will wipe it out and clean the HDD to get it ready for storage. Having said that even if you run the hypervisor via a pen drive, as I mentioned your system will be locked at a headless state, so you would not be able to use the internal HDD, except for using it as a datastore for ESXi.

Coming to GPU, for that to work you have to passthrough the video card to the VM created via ESXi console. If you do not know how to do that I can provide pointers, you can google it as well. Also v.imp, you CPU should support VT-D for the passthrough to work.
If you ask me, You would not want to go with all that pain to get the GPU working unless you plannning do some GPU intensive work like Photoshop, 3D max, etc which involves rendering and stuff like that. No high end Gaming plz.. VMWare is not meant for high end gaming, even though I might work but the experience will not be smooth enough.

How many VMs you planning to run? If its more than 3, I would recomend to upgrade to atleast 12GB ram...8+4

Regards
Sammy
Thanks. I will scavenge a 500gig hdd for it. I have swapped the dvd drive with hdd caddy so It alows two drives for storage.

I have plans to run Kodi's standalone instance, Windows Server 12 and An instance of Ubuntu.
I think these shouldn't consume more than 8 gigs and 8 cpus.
My main purpose for using the dgpu was for running 1080p br videos using kodi.

Sent from my LG-E970 using Tapatalk
 
OP
sam9s

sam9s

The High 5 Flyer
Thanks. I will scavenge a 500gig hdd for it. I have swapped the dvd drive with hdd caddy so It alows two drives for storage.

I have plans to run Kodi's standalone instance, Windows Server 12 and An instance of Ubuntu.
I think these shouldn't consume more than 8 gigs and 8 cpus.
My main purpose for using the dgpu was for running 1080p br videos using kodi.

Sent from my LG-E970 using Tapatalk

ok ... kodi will work, but its a little bit complex to get that working, google for more details, how to passthrough GPU, plus as I said you CPU should support VT-D, including your mobo and BIOS as well. I checked your CPU i7 3610qm, it does not support VT-D, (Intel Core™ i7-3610QM Processor (6M Cache, up to 3.30 GHz) Specification) you woud not be able to run GPU intensive applications ....
 
OP
sam9s

sam9s

The High 5 Flyer
evening friends, appears not many were interested to run multiple servers on Home Network. Anyway since I was already running ESXi and have the option to test run any server I want without the headache of assembling the hardware. I wanted to give a shot to some on my earlier server projects that I installed used for a day or two and for some reason rejected them. Two the projects came in to my mind and I decided to try them out again .....

1. DAPHILE
2. SUBSONIC.


DAPHILE is a fork of LMS, but with added advantages like support for DSD. DOP, etc. But for me the major advantage was that DAPHILE installs like a stand alone OS via and ISO file and does not need an underlying OS running for the same, unlike LMS.

I installed DAPHILE, on on my own VMs and started using it, unfortunately, with time I realised, DAPHILE is very unstable, probably on VM I am not sure, but the OS either kept crashing or got stuck in loop. Had to remove it after a use of couple of days.

SUBSONIC. :: yesterday I tried subsonic, which is also kinda like LMS in its approach, but very different in its UI and working. Plus SUBSONIC also supports video, unlike LMS.

I just loved the UI, and the themes you can opt. The Main reason I love LMS is coz of its unprecedented transcoding and downsampling support. LMS has been my one choice for music listening while on 3G or even on low bandwidth options like TataPhoton.

SUBSONIC really performed well on the transcoding part. Its not superbly good as LMS, coz I did get few (mind you only few) hiccups on music streaming while transcoding over 3g, but over all it was a good experience. (LMS is king though only when it comes purely to transcoding). I tested only on the AUDIO part. Since I am also already using PLEX (which works wonders when it comes to Transcoding video) I did not try that on SUBSONIC. (If anyone is interested I can give it a shot. ). But where SUBSONIC scores over LMS is the UI and features, like Media Sharing on the fly, podcasting, bandwidth control for downloads, upload music directly from UI. Multi user support (I loved this), LDAP auth, Themes, ability to use built in DDNS (like mine is (sam9s Subsonic)... ...etc etc and lots more, makes subsonic a really good music server.

Where is lacks is limited, infact only couple linux based headless clients for the same. LMS has many, like my fav PiCorePlayer. Subsonic though has truck loads of clients for windows/mac, chrome plugins and Android/iOS. It seriously lacks the same for Linux and headless Linux ......

Anyway people interested can give it a shot and try. Few images before I close out this post.

Remember this is SUBSONIC running on my HomeServer headless linux machine, also running LMS ...

1. Subsonic running on browser chrome

r50eeBy.jpg


2. Subsonic running as chrome HTML5 plugin

BIlV6jJ.jpg


3. Subsonic running as windows desktop application

42McXUk.jpg


Regards
Sammy
 

Faun

Wahahaha~!
Staff member
That's pretty detailed. I am not at my home. Can't really try these. Surely be handy when I plan to go this way.

Thanks [MENTION=6966]sam9s[/MENTION] . You always share amazing stuff !
 

whitestar_999

Super Moderator
Staff member
Blame it on "fast culture/no time" for lack of responses here or anywhere in general regarding such threads.Only enthusiasts with time & patience appreciate work like this.Also considering your home theater setup this is totally worth it :)

P.S.Do you also spend time on achieving best possible picture quality like using colorimeter like Spyder & tinkering with madVR renderer?
 
OP
sam9s

sam9s

The High 5 Flyer
opps I guess I missed the post whitestar, I apologise, .... I did use madVR, and experimented with it, yep the PQ was tad better but not something that I might say did wonders ..... personal preference I suppose ... :)
 

whitestar_999

Super Moderator
Staff member
Indeed picture & sound quality is always subjective,though as per technical specifications madvr is the best renderer out there.It has to be,considering its highest settings can bring even 980Ti almost to its knees.:)
 
Top Bottom