Make some chests and store stuff in there. Serves a few purposes - increases your inventory and saves things from loss when you die. Then since you generally need to be near the workbench or forge to make things you have to return to that area anyway so you have room for new stuff to collect. Of course, keep the dirt and stones and stuff for ladders and torches, etc that you need along the way.
The two JJ wikis should help a lot if you're looking for a recipe that isn't in your craftbook yet. The second is the official wiki, but currently the Wikia one is more complete. http://junkjack.wikia.com/wiki/Junk_Jack_Wiki http://wiki.pixbits.com/index.php/Main_Page
Dropping items on the ground will just leave them to decay eventually. If you're a packrat like me and want to keep every item you find, you should start by making 10-20 chests and placing them in your home. Then figure out a method of organizing what items go in which one so that you can find what you need at anytime.
