The system works exactly the same for both Prospect Stage and Prospect Source. Let's explain how to delete a Prospect Source, but it is exactly the same way to delete a Prospect Stage.
Go to Admin - Settings
Go to the Prospect tab, and press on the "Config Prospect Source" button
Then edit the source you want to delete, click on the "Edit" button
Click on the "Delete" button
You will have a confirmation message, press on "Yes"
You will a successful message, the source has been deleted