To remove a product image:
Admin -> Products -> pick product -> Edit -> Manage Product Image Gallery
From next screen, select image to remove, then look at the Attachment Details. Find the ‘Delete Permanently’ link and click it
Close the pop up window and the image still shows up on the product page. Click the Update button. Exit page and come back, and the image is gone.