I prefer working with natural materials. Cotton, jute, hemp, that stuff will break down over time and not leave a mess. I'll cut up water jugs to use as mini-greenhouses or plant labels, but natural is better when possible.

I also tend to view the entire environment as usable materials. I grab sticks and rocks to use as tools all the time. Because I have stuff growing just everywhere, I can afford to hack off pieces to turn into something else. Permaculture is all about "obtain a yield" but that can be anything you find useful, not just fruit. Craft materials = a yield. \o/