Writes still happen, overlayfs redirects them into the upper layer. And the lower layer will need to be updated eventually. So no, just having an overlay won't fix a challenge with durability. You should think about what data needs to be durable, and how to recover from failure.
Power off during write risks data being lost somewhere in the several layers of file system, block, and storage media. Good file systems have a journal for metadata so they will survive, but may lose data. Test this by powering off a device while in use, check if data integrity is maintained.
That raspi-config appears to have upper on tmpfs, so yes avoids wear on storage media, but writes would be lost on reboot! Which might be acceptable in some use cases, reboot and it returns to a known state.
Data that needs to be preserved could be sent elsewhere, for example syslog forwarding to a central logging server on stable power. Or posting data to some remote API. Or saved to a local data partition separate from the volume with the installed software.
Speaking of stable power, an uninterruptable power supply is an excellent investment if power is unreliable. A relatively small battery is enough to gracefully shut down if power is cut.
Regarding the lower layer, upgrading the software is an operation that must be successful for the device to still function. Consider image based updates, such as the SWUpdate embedded device updater (or Ubuntu image upgrades or Fedora IoT, several to choose from). A dual image with recovery is worth studying even if not implemented for this project.
