Caution: This post discusses deleting files, although it does not involve changing your registry. :-) Hey, I had to add some kind of disclaimer.
Something you may want to be aware of when using Windows Live Writer. If you move over and click to delete an old post, if you have the password saved and/or input the password when prompted after hitting delete, it will not only delete the post from the history of Windows Live Writer, it will delete the post from your blog as well. If you have not saved the password and cancel when prompted, then it will simply delete it the local copy and your post on your blog should stay. Another way to handle it, if you want to clean up your old posts from the PC is to go to your My Documents\My Weblog Posts\Recent Posts folder and delete the files with the .wpost extension.
Deleting Drafts is not such a big concern as they aren't sent to your blog to begin with, so deleting them via the interface is fine, but you can delete the .wpost files in My Documents\My Weblog Posts\Drafts if you prefer.