Generating a reasonable ctags database for Boost

Question

I'm running Ubuntu 8.04 and I ran the command:

$ ctags -R --c++-kinds=+p --fields=+iaS --extra=+q -f ~/.vim/tags/stdlibcpp /usr/include/c++/4.2.4/

to generate a ctags database for the standard C++ library and STL ( libstdc++ ) on my system for use with the OmniCppComplete vim script. This gave me a very reasonable 4MB tags file which seems to work fairly well.

However, when I ran the same command against the installed Boost headers:

$ ctags -R --c++-kinds=+p --fields=+iaS --extra=+q -f ~/.vim/tags/boost /usr/include/boost/

I ended up with a 1.4 GB tags file! I haven't tried it yet, but that seems likes it's going to be too large to be useful. Is there a way to get a slimmer, more usable tags file for my installed Boost headers?

Edit

Just as a note, libstdc++ includes TR1, which has allot of Boost libs in it. So there must be something weird going on for libstdc++ to come out with a 4 MB tags file and Boost to end up with a 1.4 GB tags file.

Just ran across this on the Boost mailing list:

Boost-users Boost and autocompletion

THE ANSWER

Thanks to Neg_EV for figuring out what the problem was, but there's a much better way of solving the problem than what he suggested:

Make sure apt-file is install, and run the following commands

( I keep my library tags in ~/.vim/tags/ ):

$ sudo apt-file update
$ apt-file list boost | grep -E -o '/usr/include/.*\.(h|hpp)' | grep -v '/usr/include/boost/typeof/' > ~/.vim/tags/boost-filelist
$ ctags --sort=foldcase --c++-kinds=+p --fields=+iaS --extra=+q -f ~/.vim/tags/boost -L ~/.vim/tags/boost-filelist

I've upgraded to Ubuntu 10.04 and Boost 1.40 and that's what I tested this solution on, but it should work with any Boost version as far as I can tell.

Answer 1

I know this post is a little old, but I just ran into the same problem. I looked into it a little further and it seems it is one folder in boost that is causing the problem: typeof. I am using boost 1.37 and my tags file was 1.5G, typeof was 1.4G of that. So I just created a tags file without that directory included and the resulting size was 70M. I was even able to sort it without running out of space :) I imagine in newer versions of boost they may be other projects that are too large however the general solution I found is this...

1. Generate a tag file for each boost folder seperately, a simple bash for loop should be able to do this.
2. Look at which ones are too large.
3. Either create a new single tags file excluding those large directories or keep the tag files separated simply deleting the ones that are too large.

This is the script I used (taken from comments):

for i in $(find -maxdepth 1 -type d | grep -v '^\.$' | sed 's/\.\///' ); do
    echo $i;
    ctags -f ~/tmp_tags/$i.tags -R --c++-kinds=+p --fields=+iaS --extra=+q --languages=c++ --sort=foldcase $i;
done

Hope this helps.

Answer 2

use the option

--sort=foldcase

With this the searching of the tags becomes faster.

Quoting from the man page of ctags : "The foldcase value specifies case insensitive (or case-folded) sorting. Fast binary searches of tag files sorted with case-folding will require special support from tools using tag files, such as that found in the ctags readtags library, or Vim version 6.2 or higher (using "set ignorecase"). This option must appear before the first file name"

Answer 3

You SHOULD add this option to your ctags invocation:

    -I "BOOST_SYMBOL_VISIBLE BOOST_SYMBOL_IMPORT BOOST_SYMBOL_EXPORT BOOST_FORCEINLINE BOOST_CONSTEXPR=constexpr BOOST_CONSTEXPR_OR_CONST=constexpr BOOST_STATIC_CONSTEXPR=static\ constexpr BOOST_STD_EXTENSION_NAMESPACE=std BOOST_MOVABLE_BUT_NOT_COPYABLE+ BOOST_COPYABLE_AND_MOVABLE+ BOOST_COPYABLE_AND_MOVABLE_ALT+ BOOST_NOEXCEPT=noexcept BOOST_NOEXCEPT_OR_NOTHROW=noexcept BOOST_NOEXCEPT_IF+ BOOST_NOEXCEPT_EXPR+ BOOST_STATIC_CONSTANT BOOST_DELETED_FUNCTION BOOST_DEFAULTED_FUNCTION BOOST_NESTED_TEMPLATE BOOST_UNREACHABLE_RETURN+ BOOST_DEDUCED_TYPENAME=typename BOOST_CTOR_TYPENAME=typename BOOST_LIKELY+ BOOST_UNLIKELY+ BOOST_ALIGNMENT+ BOOST_FALLTHROUGH"

This is what I use for the ENTIRE /usr/include/boost subdirectory for Boost 1.55. I get a tags file that is ~128MB. The -I seems to be the key here and helps filter out spurious tag generation.

NOTE: I'm using ctags 5.9 on Ubuntu 14.04. I have a special -I for generating ctags for C++ standard headers. This took a while for me to figure out why some header files generated almost no tags while others generated enormous amounts of tags.