I believe what you are looking for is found in the Symfony docs at Rendering each field manually. So you want to do something like this for your form then:
<div class="panel-body">
{{ form_start(edit_form }}
<div class="form-group">
{{ form_label(edit_form.categoryName, 'CategoryName', {'label_attr': {'class': 'control-label col-xs-2'}}) }}
<div class="col-xs-6">
{{ form_widget(edit_form.categoryName, {'attr': {'class': 'form-control'}}) }}
</div>
</div>
<div class="form-group">
<div class="col-xs-offset-2 col-xs-10">
<button type="submit" class="btn btn-primary">Edit</button>
</div>
</div>
{{ form_end(form) }}
</div>
Or something along these lines. As you can see, you can add classes to the twig rendered forms. for a complete reference of the twig templating functions, see the docs Twig Reference
If you are rendering the button with twig as well, then you could do this:
<div class="form-group">
<div class="col-xs-offset-2 col-xs-10">
{{ form_widget(form.editButton, {'attr': {'class': 'btn btn-primary'}}) }}
</div>
</div>