How can I delete files in Windows XP/2003 without having them go to the Recycle Bin?

You can delete files immediately without having them move to the Recycle Bin first.

There are 2 ways to do so:

The first and easiest (also works for other versions of Windows) is to delete the file, files or directories while pressing SHIFT.

The second method is to go to do it via Group Policy:

1.       Go to the Start menu, select Run and type gpedit.msc

2.       In Group Policy select User Configuration, Administrative Templates, Windows Components, Windows Explorer and find the "Do not move deleted files to the Recycle Bin" setting.

3.       Set it to be enabled.