I think they can handle losing him too. They might collectively take a step back at CB for a year but given other players that are bigger priorities and what they can do in the draft, etc. I would let him go (assuming he's still asking for $12M+/year).
The key in the secondary is retaining Joyner so you have the stability at safety for years to come. And I agree I would re-sign Robey-Coleman who for a couple years should not cost a lot. You have decent depth with Hill- who's shown he can spot start, Peterson and Countess as a backup safety. My hope is they could get one of the top two CBs in the draft at #23- a very real possibility- or at minimum get one in the 3rd while adding another mid-level FA CB.
Meantime part of it maintaining and even improving their pass rush. being strong up front rushing the passer can cover up A LOT of deficiencies elsewhere. Quinn and Barwin combined for 13.5 sacks this year and 21 including Longacre and Ebukam. Pretty good. Can they get 20 from the starting OLBs and 25-27 overall from those spots next year?