If you're wanting to craft a better way to get water, then you're going to need to locate yourself some Quartzite. This isn't too difficult to find, and you might come across it and not realize it while completing one of the main storyline quests!
To get Quartzite, you will need to go into the Lawn Mite cave that can be found by following the electrical wire of the laser that is having power issues. This is part of fixing the Mysterious Machine, which is the first major quest you will undertake in the game.
If you're having trouble locating it, you can find it more easily by heading to the location marked on this map with a red circle. You can see that is pretty much directly south of the Baseball landmark location, which is very close to where you spawn initially.
Once you have head to this location, you will see the laser projector along with a cord that runs into a cave.
Before you head inside, you might want to get a torch. You can also just wing it and find some of the Slime Mold growing in the cave. You can use this to create a Slime Mold Torch, but it won't last for very long. There's a stick nearby the cave where you can find some sap for a real torch.
Head into the cave and at the very end of it you will find a big red-ish rock that kind of reminds me of salt.
You will need a hammer to break it, so give it some whacks and it will eventually crumble into multiple pieces. You don't need a whole lot of it, but there's quite a few boulders of it in this cave. You can use this stuff for creating a Repair Tool or obtaining the Dew Collector which will make your water supply needs much easier to complete!