When your time is set to manual under Server > System Clock and it continues to drift from the actual time it is likely that your BIOS time is off.
In order to correct this you will need to log into the BIOS of the server by restarting and hitting the corresponding key which will be illustrated on the screen.
Once inside, adjust the time and save your settings. Finally restart the switchvox.
Also a bad CMOS battery can cause the time to drift. If the time in the BIOS is correct, you may need to replace the CMOS battery.