Alternative Agent Buttons
Not the most graphically pleasing prototype, but the idea may be worth considering.
The small buttons we need to click when adding/modifying agents are a problem that could be solved by replacing them with larger, color-coded, more accurately labeled ones. The example above is one idea.