You can set an alert for an agency when you are editing it. The ‘alert’ field allows you to type in some text, e.g. ‘always follow-up messages’, that displays to all users (including you) whenever this agency is viewed.

A user must dismiss this alert message every time. Alternatively, they can update the text, or remove the alert text if it's no longer current.