The content of the GroupBox can only be set once, meaning that it can only have a single control inside it. If you want to two radio button, place them in a stack panel or a grid.
<GroupBox Header="Aðgerðir" HorizontalAlignment="Left" VerticalAlignment="Bottom" Margin="270,0,0,178" Height="106" Width="176">
<StackPanel>
<CheckBox Content="CheckBox" HorizontalAlignment="Left" VerticalAlignment="Top"/>
<CheckBox Content="CheckBox" HorizontalAlignment="Left" VerticalAlignment="Bottom"/>
</StackPanel>
</GroupBox>