Why does whitespace affect MySQL Left Join and return NULL values?

https://stackoverflow.com/questions/8666655

08-04-2021
|

Question

I am having an odd issue and need some help determining the cause. So far, I have only been able to observe the symptoms. What it "seems" is happening is that MySQL is executing the query different based on whether or not I have a newline character before my first "LEFT JOIN".

Here is my setup:

I am running queries on a MySQL Database instance provided by www.xeround.com. I am using the Kohana framework, version 3.0. I am running PHP 5.3.3 FastCGI on Lighttpd. I compiled PHP with the MySQL Native Driver:

--with-mysql=mysqlnd --with-mysqli=mysqlnd --with-pdo-mysql=mysqlnd

I am using the default Kohana 3 MySQL Database Driver and QueryBuilder.

I am going to skip the PHP code and share the SQL since that seems to be the issue. The QueryBuilder ends up building this query:

SELECT `fieldreps`.`user_id` AS `fieldreps.user_id`, `fieldreps`.`availability_id` AS `fieldreps.availability_id`, `fieldreps`.`applicant_type` AS `fieldreps.applicant_type`, `fieldreps`.`license_number` AS `fieldreps.license_number`, `fieldreps`.`license_exp` AS `fieldreps.license_exp`, `fieldreps`.`license_state` AS `fieldreps.license_state`, `fieldreps`.`car` AS `fieldreps.car`, `fieldreps`.`authorized_worker` AS `fieldreps.authorized_worker`, `fieldreps`.`restrictions` AS `fieldreps.restrictions`, `fieldreps`.`night` AS `fieldreps.night`, `fieldreps`.`day` AS `fieldreps.day`, `fieldreps`.`longer` AS `fieldreps.longer`, `fieldreps`.`commitment` AS `fieldreps.commitment`, `fieldreps`.`travel_metro` AS `fieldreps.travel_metro`, `fieldreps`.`travel_states` AS `fieldreps.travel_states`, `fieldreps`.`big_employee` AS `fieldreps.big_employee`, `fieldreps`.`employed` AS `fieldreps.employed`, `fieldreps`.`retail` AS `fieldreps.retail`, `fieldreps`.`status` AS `fieldreps.status`, `fieldreps`.`start` AS `fieldreps.start`, `fieldreps`.`sales` AS `fieldreps.sales`, `fieldreps`.`study` AS `fieldreps.study`, `fieldreps`.`relevant` AS `fieldreps.relevant`, `fieldreps`.`experience` AS `fieldreps.experience`, `fieldreps`.`claims` AS `fieldreps.claims`, `fieldreps`.`education` AS `fieldreps.education`, `fieldreps`.`degree_details` AS `fieldreps.degree_details`, `fieldreps`.`degree_institution` AS `fieldreps.degree_institution`, `fieldreps`.`other_training` AS `fieldreps.other_training`, `fieldreps`.`jobs` AS `fieldreps.jobs`, `fieldreps`.`current_training` AS `fieldreps.current_training`, `fieldreps`.`interested` AS `fieldreps.interested`, `fieldreps`.`achievements` AS `fieldreps.achievements`, `fieldreps`.`passions` AS `fieldreps.passions`, `fieldreps`.`ambitions` AS `fieldreps.ambitions`, `fieldreps`.`max_travel_time` AS `fieldreps.max_travel_time`, `fieldreps`.`creation_time` AS `fieldreps.creation_time`, `fieldreps`.`resume` AS `fieldreps.resume`, `users`.`user_id` AS `users.user_id`, `users`.`email` AS `users.email`, `users`.`name` AS `users.name`, `users`.`password` AS `users.password`, `users`.`given_name` AS `users.given_name`, `users`.`title` AS `users.title`, `users`.`nationality` AS `users.nationality`, `availabilities`.`availability_id` AS `availabilities.availability_id`, `availabilities`.`fieldrep_id` AS `availabilities.fieldrep_id`, `availabilities`.`mon_start` AS `availabilities.mon_start`, `availabilities`.`mon_end` AS `availabilities.mon_end`, `availabilities`.`tue_start` AS `availabilities.tue_start`, `availabilities`.`tue_end` AS `availabilities.tue_end`, `availabilities`.`wed_start` AS `availabilities.wed_start`, `availabilities`.`wed_end` AS `availabilities.wed_end`, `availabilities`.`thur_start` AS `availabilities.thur_start`, `availabilities`.`thur_end` AS `availabilities.thur_end`, `availabilities`.`fri_start` AS `availabilities.fri_start`, `availabilities`.`fri_end` AS `availabilities.fri_end`, `availabilities`.`sat_start` AS `availabilities.sat_start`, `availabilities`.`sat_end` AS `availabilities.sat_end`, `availabilities`.`sun_start` AS `availabilities.sun_start`, `availabilities`.`sun_end` AS `availabilities.sun_end` FROM `fieldreps` AS `fieldreps` LEFT JOIN `users` ON (`fieldreps`.`user_id` = `users`.`user_id`) LEFT JOIN `availabilities` ON (`fieldreps`.`availability_id` = `availabilities`.`availability_id`)

Which returns a result like this:(I removed most of the fields returned from the Fieldreps table to keep it short and less personal.)

However, when I insert a newline character immediately before the first LEFT JOIN, like this:

SELECT `fieldreps`.`user_id` AS `fieldreps.user_id`, `fieldreps`.`availability_id` AS `fieldreps.availability_id`, `fieldreps`.`applicant_type` AS `fieldreps.applicant_type`, `fieldreps`.`license_number` AS `fieldreps.license_number`, `fieldreps`.`license_exp` AS `fieldreps.license_exp`, `fieldreps`.`license_state` AS `fieldreps.license_state`, `fieldreps`.`car` AS `fieldreps.car`, `fieldreps`.`authorized_worker` AS `fieldreps.authorized_worker`, `fieldreps`.`restrictions` AS `fieldreps.restrictions`, `fieldreps`.`night` AS `fieldreps.night`, `fieldreps`.`day` AS `fieldreps.day`, `fieldreps`.`longer` AS `fieldreps.longer`, `fieldreps`.`commitment` AS `fieldreps.commitment`, `fieldreps`.`travel_metro` AS `fieldreps.travel_metro`, `fieldreps`.`travel_states` AS `fieldreps.travel_states`, `fieldreps`.`big_employee` AS `fieldreps.big_employee`, `fieldreps`.`employed` AS `fieldreps.employed`, `fieldreps`.`retail` AS `fieldreps.retail`, `fieldreps`.`status` AS `fieldreps.status`, `fieldreps`.`start` AS `fieldreps.start`, `fieldreps`.`sales` AS `fieldreps.sales`, `fieldreps`.`study` AS `fieldreps.study`, `fieldreps`.`relevant` AS `fieldreps.relevant`, `fieldreps`.`experience` AS `fieldreps.experience`, `fieldreps`.`claims` AS `fieldreps.claims`, `fieldreps`.`education` AS `fieldreps.education`, `fieldreps`.`degree_details` AS `fieldreps.degree_details`, `fieldreps`.`degree_institution` AS `fieldreps.degree_institution`, `fieldreps`.`other_training` AS `fieldreps.other_training`, `fieldreps`.`jobs` AS `fieldreps.jobs`, `fieldreps`.`current_training` AS `fieldreps.current_training`, `fieldreps`.`interested` AS `fieldreps.interested`, `fieldreps`.`achievements` AS `fieldreps.achievements`, `fieldreps`.`passions` AS `fieldreps.passions`, `fieldreps`.`ambitions` AS `fieldreps.ambitions`, `fieldreps`.`max_travel_time` AS `fieldreps.max_travel_time`, `fieldreps`.`creation_time` AS `fieldreps.creation_time`, `fieldreps`.`resume` AS `fieldreps.resume`, `users`.`user_id` AS `users.user_id`, `users`.`email` AS `users.email`, `users`.`name` AS `users.name`, `users`.`password` AS `users.password`, `users`.`given_name` AS `users.given_name`, `users`.`title` AS `users.title`, `users`.`nationality` AS `users.nationality`, `availabilities`.`availability_id` AS `availabilities.availability_id`, `availabilities`.`fieldrep_id` AS `availabilities.fieldrep_id`, `availabilities`.`mon_start` AS `availabilities.mon_start`, `availabilities`.`mon_end` AS `availabilities.mon_end`, `availabilities`.`tue_start` AS `availabilities.tue_start`, `availabilities`.`tue_end` AS `availabilities.tue_end`, `availabilities`.`wed_start` AS `availabilities.wed_start`, `availabilities`.`wed_end` AS `availabilities.wed_end`, `availabilities`.`thur_start` AS `availabilities.thur_start`, `availabilities`.`thur_end` AS `availabilities.thur_end`, `availabilities`.`fri_start` AS `availabilities.fri_start`, `availabilities`.`fri_end` AS `availabilities.fri_end`, `availabilities`.`sat_start` AS `availabilities.sat_start`, `availabilities`.`sat_end` AS `availabilities.sat_end`, `availabilities`.`sun_start` AS `availabilities.sun_start`, `availabilities`.`sun_end` AS `availabilities.sun_end` FROM `fieldreps` AS `fieldreps` 
LEFT JOIN `users` ON (`fieldreps`.`user_id` = `users`.`user_id`) LEFT JOIN `availabilities` ON (`fieldreps`.`availability_id` = `availabilities`.`availability_id`)

I now get the result I want:

Notice that the LEFT JOIN on the Users table works fine in both scenarios. I also tried adding the newline before the second LEFT JOIN, instead of the first. Same unwanted result. The only thing that I can find to fix it is placing a newline in front of the first LEFT JOIN. The queries that are built by QueryBuilder and sent to MySQL are always long, single-line, minimum whitespace strings...no newlines! This is the first time I have had this issue. I just can't believe there would be a requirement to place a newline before the first LEFT JOIN.

My question boils down to these points:

Is this supposed be happening this way? (Whitespace effecting the query result)
What is the best way to go about getting the result I want.

I am not an SQL expert, otherwise I would probably not need to ask this question. I am sure there is something I am overlooking.

Thank you,

Jonathan

Solution

My issue was right in front of me like I thought:

Indices

I had inadvertently created two indices on availabilities.fieldrep_id, and the indices on the rest of the schema were, shoddy-at best.

I used EXPLAIN which indicated it was not using an index for 2 of the three joins. I cleaned up my indices and now it works great, everytime.

Also, I think I figured out why the newline would change the query. If I ran the query over and over I would see it switch between the examples given, regardless of adding the newline or not. I am assuming that MySQL has some caching for repeat queries. I bet that the newline was enough to trigger it to ignore the cached version, hence it would sometimes give me the desired result.

Regardless, the issue was with indices. Everything else was just a symptom.

Maybe this will help someone save a bit of time.

Licensed under: CC-BY-SA with attribution

Not affiliated with StackOverflow