Consider using System.Collections.BitArray, which is an arbitrary-length array of bits. You can do intersections (AND), unions (OR), and complements (NOT).
You can now haver more than 32 categories keyed by integers:
public static class Categories
{
public const int Category1 = 1;
public const int Category2 = 2;
//...
public const int Category3123 = 3123;
public const int Max = 5000;
}
BitArray myBitset = new BitArray((int)Categories.Max);
myBitSet.Set(Categories.Category1);
myBitSet.Set(Categories.Category4);
I used ints rather than enums to avoid all the casts to int that are necessary when using BitArray, but obviously you could encapsulate these in a class that deals with categories.