Every system I've ever worked with allows a little bit of developer-level of prioritization. If the next card has the absolute (top-down driven) priority, then you have to pick that card. Mostly, though, I tend to work in places where "these next 6 cards are up, pick the one you like". This gives the developer a little bit of room for type of work he or she prefers. Plus, it gives the developers a greater sense of ownership since they did get to pick (to some extent) the work they were doing.
Regarding your example, it's a little off base. In an ideal world any developer should be able to pick up any card. In reality, this isn't always true. If I give this project to Jim, it might take 2 days. If I give it to not-Jim, it make take all week. This is a sign! What information sharing is missing? How do you get the other developers to understand the Contracts component as well as Jim?
If the priority is a little bit gray, this stuff tends to work itself out. All the other developers know that Jim can handle the Contracts stuff. However, if Jim has no capacity, then someone else must take up the challenge. Kanban is supposed to alert you to blocked stories.