Performing a simple search in MySQL db with variable amount of input

Question

I have a many-to-many db with these three tables, Films, Ambiences, Films_Ambience:

CREATE TABLE Films (  
id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY(id),    
Title VARCHAR(255));

CREATE TABLE Ambiences (  
id INT NOT NULL AUTO_INCREMENT,   
PRIMARY KEY(id),  
ambienceName VARCHAR(255));

CREATE TABLE Films_Ambiences (
film_id INT NOT NULL,  
ambience_id INT NOT NULL,  
PRIMARY KEY (film_id, ambience_id),  
FOREIGN KEY (film_id) REFERENCES Films(id) ON UPDATE CASCADE,  
FOREIGN KEY (ambience_id) REFERENCES Ambiences(id) ON UPDATE CASCADE);

I am using information from a form to search for specific films (e.g. film that is funny and scary at the same time). The form are simply 'ticks' next to given names. The information is sent by $_POST.

The problem is that I don't know how many requirements will there be. I know the maximum number that a user can choose but I cannot tell how many or which they will actually pick (I could do that by checking isset($_POST['somethin']) but it would be very.. monotonous if I had, for example, 20 different options. So I cannot do anything like:

$ambience1 = $_POST["a1"];
$ambience2 = $_POST["a2"];
$ambience3 = $_POST["a2"];
...
...
...

and:

SELECT *,GROUP_CONCAT(ambienceName SEPARATOR ' ') AS ambiences
FROM Films AS f 
INNER JOIN Films_Ambiences as fa ON f.id = fa.film_id           
INNER JOIN Ambiences AS a ON a.id = fa.ambience_id
GROUP BY Title
HAVING (ambiences LIKE '%$ambience1%' AND ambiences LIKE '%$ambience2%' AND ...

I'm not even sure where to start. Can I do it with SQL or rather PHP?

Here's an SQLFiddle if U like.

Answer 1

Using keyword search predicates LIKE '%pattern%' is a sure way to cause poor performance, because it forces a table-scan.

The best way to do a relational division query, that is to match only movies where all three criteria are matched, is to find individual rows for each of the criteria, and then JOIN them together.

SELECT f.*, CONCAT_WS(' ', a1.ambienceName, a2.ambienceName, a3.ambienceName) AS ambiences
FROM Films AS f 
INNER JOIN Films_Ambiences as fa1 ON f.id = fa1.film_id           
INNER JOIN Ambiences AS a1 ON a1.id = fa1.ambience_id
INNER JOIN Films_Ambiences as fa2 ON f.id = fa2.film_id           
INNER JOIN Ambiences AS a2 ON a2.id = fa2.ambience_id
INNER JOIN Films_Ambiences as fa3 ON f.id = fa3.film_id           
INNER JOIN Ambiences AS a3 ON a3.id = fa3.ambience_id
WHERE (a1.ambienceName, a2.ambienceName, a3.ambienceName) = (?, ?, ?);

You'll need an additional JOIN to Films_Ambiences and Ambiences for each search term.

You should have an index on ambienceName, and then all three lookups will be more efficient.

ALTER TABLE Ambiences ADD KEY (ambienceName);

I compared different solutions for relational division in a recent presentation:

• Slides: http://www.slideshare.net/billkarwin/sql-query-patterns-optimized
• Webinar recording: http://www.percona.com/webinars/mysql-query-patterns-optimized

---

Re your comment:

Is there a way to alter this query so that it also displays the rest of the ambiences after the criteria are found?

Yes, but you have to join one more time to get the full set of ambiences for the film:

SELECT f.*, GROUP_CONCAT(a_all.ambienceName) AS ambiences
FROM Films AS f 
INNER JOIN Films_Ambiences as fa1 ON f.id = fa1.film_id           
INNER JOIN Ambiences AS a1 ON a1.id = fa1.ambience_id
INNER JOIN Films_Ambiences as fa2 ON f.id = fa2.film_id           
INNER JOIN Ambiences AS a2 ON a2.id = fa2.ambience_id
INNER JOIN Films_Ambiences as fa3 ON f.id = fa3.film_id           
INNER JOIN Ambiences AS a3 ON a3.id = fa3.ambience_id
INNER JOIN Films_Ambiences AS fa_all ON f.id = fa_all.film_id
INNER JOIN Ambiences AS a_all ON a_all.id = fa_all.ambience_id
WHERE (a1.ambienceName, a2.ambienceName, a3.ambienceName) = (?, ?, ?)
GROUP BY f.id;

is there a way to alter this query so that the result are only films that have the ambiences required but no more?

The query above should do that.

---

What the query does, I think, is to look for films that include the given ambiences (so it also find films that have more ambiences).

Right, the query does not match a film unless it matches all three ambiences in the search criteria. But the film may have other ambiences beyond those in the search criteria, and all of the film's ambiences (those in the search criteria plus others) are collected as GROUP_CONCAT(a_all.ambienceName).

I tested this example:

mysql> INSERT INTO Ambiences (ambienceName) 
 VALUES ('funny'), ('scary'), ('1950s'), ('London'), ('bank'), ('crime'), ('stupid');
mysql> INSERT INTO Films (title) 
 VALUES ('Mary Poppins'), ('Heist'), ('Scary Movie'), ('Godzilla'), ('Signs');
mysql> INSERT INTO Films_Ambiences 
 VALUES (1,1),(1,2),(1,4),(1,5), (2,1),(2,2),(2,5),(2,6), (3,1),(3,2),(3,7), (4,2),(4,3), (5,2),(5,7);

mysql> SELECT f.*, GROUP_CONCAT(a_all.ambienceName) AS ambiences 
 FROM Films AS f  
 INNER JOIN Films_Ambiences as fa1 ON f.id = fa1.film_id            
 INNER JOIN Ambiences AS a1 ON a1.id = fa1.ambience_id 
 INNER JOIN Films_Ambiences as fa2 ON f.id = fa2.film_id            
 INNER JOIN Ambiences AS a2 ON a2.id = fa2.ambience_id 
 INNER JOIN Films_Ambiences as fa3 ON f.id = fa3.film_id            
 INNER JOIN Ambiences AS a3 ON a3.id = fa3.ambience_id 
 INNER JOIN Films_Ambiences AS fa_all ON f.id = fa_all.film_id 
 INNER JOIN Ambiences AS a_all ON a_all.id = fa_all.ambience_id 
 WHERE (a1.ambienceName, a2.ambienceName, a3.ambienceName) = ('funny','scary','bank') 
 GROUP BY f.id;
+----+--------------+-------------------------+
| id | Title        | ambiences               |
+----+--------------+-------------------------+
|  1 | Mary Poppins | funny,scary,London,bank |
|  2 | Heist        | funny,scary,bank,crime  |
+----+--------------+-------------------------+

By the way, here's the EXPLAIN showing usage of indexes:

+----+-------------+--------+--------+----------------------+--------------+---------+-----------------------------+------+-----------------------------------------------------------+
| id | select_type | table  | type   | possible_keys        | key          | key_len | ref                         | rows | Extra                                                     |
+----+-------------+--------+--------+----------------------+--------------+---------+-----------------------------+------+-----------------------------------------------------------+
|  1 | SIMPLE      | a1     | ref    | PRIMARY,ambienceName | ambienceName | 258     | const                       |    1 | Using where; Using index; Using temporary; Using filesort |
|  1 | SIMPLE      | a2     | ref    | PRIMARY,ambienceName | ambienceName | 258     | const                       |    1 | Using where; Using index                                  |
|  1 | SIMPLE      | a3     | ref    | PRIMARY,ambienceName | ambienceName | 258     | const                       |    1 | Using where; Using index                                  |
|  1 | SIMPLE      | fa1    | ref    | PRIMARY,ambience_id  | ambience_id  | 4       | test.a1.id                  |    1 | Using index                                               |
|  1 | SIMPLE      | f      | eq_ref | PRIMARY              | PRIMARY      | 4       | test.fa1.film_id            |    1 | NULL                                                      |
|  1 | SIMPLE      | fa2    | eq_ref | PRIMARY,ambience_id  | PRIMARY      | 8       | test.fa1.film_id,test.a2.id |    1 | Using index                                               |
|  1 | SIMPLE      | fa3    | eq_ref | PRIMARY,ambience_id  | PRIMARY      | 8       | test.fa1.film_id,test.a3.id |    1 | Using index                                               |
|  1 | SIMPLE      | fa_all | ref    | PRIMARY,ambience_id  | PRIMARY      | 4       | test.fa1.film_id            |    1 | Using index                                               |
|  1 | SIMPLE      | a_all  | eq_ref | PRIMARY              | PRIMARY      | 4       | test.fa_all.ambience_id     |    1 | NULL                                                      |
+----+-------------+--------+--------+----------------------+--------------+---------+-----------------------------+------+-----------------------------------------------------------+

---

I have a film1 which is scary, funny, stupid. When I search for a film which is only scary, stupid I will get film1 anyway. What if I dont want that?

Oh, okay, I totally didn't understand that was what you meant, and it's an unusual requirement in these types of problems.

Here's a solution:

mysql> SELECT f.*, GROUP_CONCAT(a_all.ambienceName) AS ambiences
 FROM Films AS f
 INNER JOIN Films_Ambiences as fa1 ON f.id = fa1.film_id
 INNER JOIN Ambiences AS a1 ON a1.id = fa1.ambience_id
 INNER JOIN Films_Ambiences as fa2 ON f.id = fa2.film_id
 INNER JOIN Ambiences AS a2 ON a2.id = fa2.ambience_id
 INNER JOIN Films_Ambiences AS fa_all ON f.id = fa_all.film_id
 WHERE (a1.ambienceName, a2.ambienceName) = ('scary','stupid')
 GROUP BY f.id
 HAVING COUNT(*) = 2
+----+-------+--------------+
| id | Title | ambiences    |
+----+-------+--------------+
|  5 | Signs | scary,stupid |
+----+-------+--------------+

There's no need to join to a_all in this case, because we don't need the list of ambiences names, we only need the count of ambiences, which we can get just by joining to fa_all.

Answer 2

I think that you can use PHP to build your SQL, something like this:

$ambienceWhere = '1=1 ';
for ($i=0;$i<NUMBER_OF_POSSIBLE_AMBIENCES;$i++) {
  if (isset($_POST['a' . $i])) { // or another criteria to avoid processing this one       
    $ambienceWhere .= ' AND '; 
    $ambienceWhere .= ' a.ambiences LIKE \'%' . $_POST['a'. $i] '%\' ';
  }
}
$query = 'SELECT ....... WHERE ('. $ambienceWhere.') .....'

Answer 3

Your form should have the ambiences in an array of elements, perhaps a multiple select. This will then turn into an array in PHP, $_POST['ambience'][]. Then you can write:

$ambience_query = implode(' AND ', array_map(function($a) use($mysqli) {
    return "'%" . mysqli_real_escape_string($mysqli, $a) . "'";
}, $_POST['ambience']));

$query = "SELECT *, GROUP_CONCAT(ambienceName SEPARATOR ' ') AS ambiences
FROM FROM Films AS f 
INNER JOIN Films_Ambiences as fa ON f.id = fa.film_id           
INNER JOIN Ambiences AS a ON a.id = fa.ambience_id
GROUP BY Title
HAVING $ambience_query";

This is a very expensive query. It will have to calculate the GROUP_CONCAT(ambienceName) of every film in the database before filtering down to the ones you want. it would be better to structure the query like:

SELECT f.*
FROM Films f
INNER JOIN Films_Ambiences fa1 ON f.id = fa1.film_id
INNER JOIN Ambiences a1 ON a1.id = f1.ambience_id
INNER JOIN Films_Ambiences fa2 ON f.id = fa2.film_id
INNER JOIN Ambiences a2 ON a2.id = f2.ambience_id
INNER JOIN Films_Ambiences fa3 ON f.id = fa3.film_id
INNER JOIN Ambiences a3 ON a3.id = f3.ambience_id
...
WHERE a1.ambienceName = '$_POST[ambience][1]'
  AND a2.ambienceName = '$_POST[ambience][2]'
  AND a3.ambienceName = '$_POST[ambience][3]'
  ...

You can use loops similar to the above or Josejulio's answer to construct this query.