You can include the delete button in your ItemTemplate
:
<asp:TemplateField>
<ItemTemplate>
<asp:LinkButton ID="lnkbtnedit" Text="Edit" style="color:#003399;" CommandName="Edit" runat="server"></asp:LinkButton>
<asp:LinkButton ID="lnkbtndelete" Text="Delete" style="color:#003399;" CommandName="Delete" runat="server"></asp:LinkButton>
</ItemTemplate>
</asp:TemplateField>
And then just remove the CommandField
.