Sleep mode turns on hours before scheduled time

My sleep mode somehow gets enabled hours before it is scheduled to, even including the wind down time. As a result, I don't get notifications when I want them to still come through.

Where can I find other settings to change the time sleep mode is enabled? None of the settings I can find point towards it turning on far earlier than I have it set to.