If you plan to hike to the Reykjadalur hot springs, you can use the restrooms at the parking lot (so you won’t have to pee in the stream 🙂
The parking lot and starting point for the hot springs are located a few kilometers north of the town of Hveragerði.