I finally managed to install windows, but not from the pen drive, directly from the ISO.
So here are the problems I faced and the solutions, to help someone who stumbles across this thread with a similar problem:
Problem: Tried installing a caddy, but my DVD drive won't come out.
Solution: It mostly gets stuck in the front of the frame, i.e., the side of the laptop. Just push the edges of the laptop frame around the DVD drive outward, it should come out. Be gentle in this and don't break the frame. If this doesn't work, the DVD drive might be stuck somewhere else.
Problem: Installed an SSD on my laptop and cloned my C drive to the SSD. But there was no option to switch to the SSD as boot drive.
Solution: HP Laptop BIOS does not allow manually changing between two hard drives/SSDs. If your SSD comes with a cloning software, that may be able to help. Mine didn't, this step was a dead end for me.
Problem: Tried doing a clean install of Windows, but unable to boot from USB.
Solution: Some times you might need to disable secure boot and enable legacy boot options in your BIOS. This is required if you create a bootable device with anything other than Microsoft Media Creation tool. If the secure boot option is disabled, you might need to reset the security keys to enable it. If it still doesn't work, check if your USB drive is faulty by using another pen drive (this was the problem I was facing).
Problem: Unable to boot from USB, so tried installing directly by mounting the ISO. But can't change drive letters in this case.
Solution: For a long time now, if you've had a working PC, you don't need to create boot media at all. You can mount windows iso image and run the setup from it. However, this will not allow you to format drives or change drive letters. To get around this problem, create a clean partition, install Windows on that. After you boot into the new Windows installation, format the old drive and delete the boot record from msconfig.
Problem: Installed Windows on A:. Some programs don't work properly
Solution: Apparently there are some programs which still honour legacy rules and don't work properly if you try to install them on A: or B:. So even if you are not able to install on C:, use any drive letter from D-Z for your Windows installation, but don't use A and B.
Problem: Windows installation gets stuck.
Solution: In my case, there were two reasons for this which I managed to solve. First was Windows update. To solve this, don't connect to the internet during the setup process, just select "I don't have internet connection right now" option. Second was that my laptop was overheating and shutting down in the middle. To solve this, I placed my laptop in an AC room under the fan, and whenever I heard the laptop fan working too hard, I used additional fan to blow air on it. Sounds ridiculous, but my laptop overheats only during Windows installation and gaming, not during regular usage. I got it serviced and cleaned recently. So I don't know why it happens.