While we don't add FDs with an active callback to the watched FDSET, we still
can get notifications for callbacks active due the asynchronous processing
of the same.
To avoid queue multiple callbacks, we check for queued callbacks before
activating new ones.