DELETE FILES WITHOUT MOVING FILE TO RECYCLE BIN

You can also delete files from hard disk without placing it in hard disk by selecting file and pressing Shift + Delete keys at a time.

FILES COMPLETELY DELETE VIA ONLY DELETE KEY ,FILE NOT IN RECYCLEBIN
Alternate:
Whenever a file is deleted , it moves into recycle bin.But not from the hard disk.So to remove any file from hard disk directly without moving it to Recycle Bin just follow these steps:

GPEDIT.MSC

1. Go to Start>Run , type gpedit.msc to open group policy editor
   
2. In group policy editor , expand User Configuration>Administrative
3. Templates>Windows Components and select Windows Explorer
4. On right window pane you can see Setting : Do not move deleted files to recycle bin
5. Double Click it and select Enabled and click Ok
6. That's it now you can delete files in your hard disk without moving it in Recycle Bin

DO NOT MOVE FILES TO RECYCLE BIN