If you end up with just some stuff tagged/categorized and the rest not, you can always of course change your templates to not display the categories (it's pretty easy) and have them remain there safely in the database for the rainy weekend you change your mind and decide to go on another category frenzy...