Use a traffic control matching rule and an ip tables rule to mark the traffic.
tc filter add dev eth0 protocol ip parent 1:0 prio 1 handle 1 fw flowid 1:10
iptables -A OUTPUT -i eth0 -p tcp -j MARK --set-mark 1
Will put all your TCP outbound packets in the 100 megabits per second queue you have.
Since you want to match just one application, let us pretend that your application has a TCP socket listening on port 7654 that all connections are made through. The iptables rule would look more like this
iptables -A OUTPUT -i eth0 -p tcp -m multiport --sports 7654 -j MARK --set-mark 1
Also you should make the default in your tc qdisc
command the 1 Gbps queue, so that normal traffic on the system is not affecting your test in any way.