Well I think an efficcient way to draw your shadow will be to do something like ray-tracing. It means basically that you iter through every pixel in your image and draw in the right color by checking how many "wall" the reversed ray traverses. something like :
For every pixel in the image
compute the reversed lightray (from the pixel to your light direction)
compute the number of walls traversed by your lightray
(= nb of intersections with all the segments/walls)
draw the pixel according to the number of intersections
(the more intersections, the darker)
It should be easy and O(n) in complexity with the number of pixels ^^
By the way I think boost::geometry can manage intersections.