Do you get this message while the OS is just loading up, or when it tries to initialise the display (on running startx)? It sounds like your monitor is not able to handle the resolution/refresh rate. Could you post the file /etc/X11/xorg.conf here? Also, what did you enter in the horizontal and vertical refresh rates for your monitor?