Disable “quick edit” only for non admin in functions.php

Question

I have this in my functions.php

function remove_quick_edit( $actions ) {
    unset($actions['inline hide-if-no-js']);
    return $actions;
}
add_filter('post_row_actions','remove_quick_edit',10,1);

to remove the quick edit link in the backend when scrolling the list of published posts.

It works like a charm but it disable it even for the admin role. Is it possible to keep it showing only for the admin while still diabling for the rest? Thanks!

SOLVED thanks to jfacemyer! This is the full code to add in functions.php

function remove_quick_edit( $actions ) {
    unset($actions['inline hide-if-no-js']);
    return $actions;
}
if ( ! current_user_can('manage_options') ) {
    add_filter('post_row_actions','remove_quick_edit',10,1);
}