How to combine two querysets when defining choices in a ModelMultipleChoiceField?

Question

I am using a piece of code in two separate places in order to dynamically generate some form fields. In both cases, dynamic_fields is a dictionary where the keys are objects and the values are lists of objects (in the event of an empty list, the value is False instead):

class ExampleForm(forms.ModelForm):
    def __init__(self, *args, **kwargs):
        dynamic_fields = kwargs.pop('dynamic_fields')
        super(ExampleForm, self).__init__(*args, **kwargs)

        for key in dynamic_fields:
            if dynamic_fields[key]:
                self.fields[key.description] = forms.ModelMultipleChoiceField(widget=forms.CheckboxSelectMultiple, queryset=dynamic_fields[key], required=False)


    class Meta:
        model = Foo
        fields = ()

In one view, for any key the value is a list of objects returned with a single DB query - a single, normal queryset. This view works just fine.

In the other view, it takes multiple queries to get everything I need to construct a given value. I am first instantiating the dictionary with the values set equal to blank lists, then adding the querysets I get from these multiple queries to the appropriate lists one at a time with basic list comprehension (dict[key] += queryset). This makes each value a 2-D list, which I then flatten (and remove duplicates) by doing:

for key in dict:
    dict[key] = list(set(dict[key]))

I have tried this several different ways - directly appending the queries in each queryset to the values/lists, leaving it as a list of lists, using append instead of += - but I get the same error every time: 'list' object has no attribute 'none'.

Looking through the traceback, the error is coming up in the form's clean method. This is the relevant section from the code in django.forms.models:

def clean(self, value):
    if self.required and not value:
        raise ValidationError(self.error_messages['required'], code='required')
    elif not self.required and not value:
        return self.queryset.none() # the offending line

My thought process so far: in my first view, I'm generating the list that serves as the value for each key via a single query, but I'm combining multiple queries into a list in my second view. That list doesn't have a none method like I would normally have with a single queryset.

How do I combine multiple querysets without losing access to this method?

I found this post, but I'm still running into the same issue using itertools.chain as suggested there. The only thing I've been able to accomplish with that is changing the error to say 'chain' or 'set' object has no attribute 'none'.

---

Edit: here's some additional information about how the querysets are generated. I have the following models (only relevant fields are shown):

class Profile(models.Model):
    user = models.OneToOneField(User)
    preferred_genres = models.ManyToManyField(Genre, blank=True)

class Genre(models.Model):
    description = models.CharField(max_length=200, unique=True)
    parent = models.ForeignKey("Genre", null=True, blank=True)

class Trope(models.Model):
    description = models.CharField(max_length=200, unique=True)
    genre_relation = models.ManyToManyField(Genre)

In (the working) view #1, the dictionary I use to generate my fields has keys equal to a certain Genre, and values equal to a list of Genres for whom the key is a parent. In other words, for every key, the queryset is Genre.objects.filter(parent=key, **kwargs).

In the non-functional view #2, we need to start with the profile's preferred_genres field. For every preferred_genre I need to pull the associated Tropes and combine them into a single queryset. Right now, I am looping through preferred_genres and doing something like this:

for g in preferred_genres:
    tropeset = g.trope_set.all()

This gets me a bunch of individual querysets containing the information I need, but I can't find a way to combine the multiple tropesets into one big queryset (as opposed to a list without the none attribute). (As an aside, this also hammers my database with a bunch of queries. I am also trying to wrap my head around how I can maybe use prefetch_related to reduce the number of queries, but one thing at a time.)

If I can't combine these querysets into one but CAN somehow accomplish these lookups with a single query, I am all ears! I am now reading the documentation regarding complex queries with the Q object. It is tantalizing - I can conceptualize how this would accomplish what I'm looking for, but only if I can call all of the queries at one time. Since I have to call them iteratively one at a time, I am not sure how to use the Q object to | or & them together.

Answer 1

You can combine querysets by using the | and & operators.

from functools import reduce
from operator import and_, or_

querysets = [q1, q2, q3, ...]  # List of querysets you want to combine.

# Objects that are present in *at least one* of the queries
combined_or_querysets = reduce(or_, querysets[1:], querysets[0])

# Objects that are present in *all* of the queries
combined_and_querysets = reduce(and_, querysets[1:], querysets[0])

From Django 1.11+ you can also use the union and intersection methods.

Answer 2

I've found a "solution" to this problem. If I structure the query like so, I can get everything I need in one swoop without having to combine querysets after the fact:

desired_value = Trope.objects.filter(genre_relation__in=preferred_genres).distinct()

I still do not know how to combine multiple querysets into one without losing the inherent "queryset-ness" that seems to be necessary for the form to render properly. However, for my specific use case, restructuring the query as noted renders the issue moot.