Split data over multiple columns into rows (in SQL or SAS)

Question

I have data in the following form:

Country    City1    City2    City3       AverageTemperature
UK         London   Glasgow  Manchester  15
Italy      Rome     Naples   Venice      25
Germany    Munich   Berlin               20
USA        New York                      25

With either SQL or a SAS data step, I would like to get the data in this form:

Country    City        AverageTemperature
UK         London      15
UK         Glasgow     15
UK         Manchester  15
Italy      Rome        25
Italy      Naples      25
Italy      Venice      25
Germany    Munich      20
Germany    Berlin      20
USA        New York    25

So that I have the data across individual rows. I have thought about doing this by looping over the three city columns where the city is not blank, but I'm not sure how to confidently do this - is it easily done with either SQL or SAS? Just a pointer would be greatly appreciated.

Answer 1

SELECT COUNTRY, City1,  AverageTemperature FROM Table_Name
UNION ALL 
SELECT COUNTRY, City2,  AverageTemperature FROM Table_Name
UNION ALL 
SELECT COUNTRY, City3,  AverageTemperature FROM Table_Name

To get rows where City column is not null you can do something like this

SELECT COUNTRY, City1,  AverageTemperature FROM Table_Name 
WHERE City1 IS NOT NULL
UNION ALL 
SELECT COUNTRY, City2,  AverageTemperature FROM Table_Name 
WHERE City2 IS NOT NULL
UNION ALL 
SELECT COUNTRY, City3,  AverageTemperature FROM Table_Name 
WHERE City3 IS NOT NULL

Answer 2

Simple in a SAS Data Step

data out;
set in;
array cities[3] city1-city3;
format city $12.;
do i=1 to 3;
  if compress(cities[i]) ^= "" then do;
     city = cities[i];
     output;
  end;
end;
keep country city  AverageTemperature;
run;

Answer 3

You can unpivot this using "relatively" standard SQL. Here is an approach that only requires scanning the data once:

select country, city, averagetemperatur
from (select t.country,
             (case when n = 1 then t.city1
                   when n = 2 then t.city2
                   when n = 3 then t.city3
             end) as city,
            t.averagetemperature
     from t cross join
          (select 1 as n union all select 2 union all select 3) n
    ) t
 where city is not null;

The exact syntax for creating the table with three rows (n) can vary depending on the database.

Answer 4

A macro loop should do the job:

%MACRO Cities ;
  %DO N=1 %TO 3 ;
    proc sql ;
      create table Cities_&N as
      select Country, City&N as City, AverageTemperature
      from your_table_name_here
      where City&N is not null ;
    quit ;
  %END ;

  data Cities ;
    set Cities_: ;
  run ;
%MEND ;

%Cities ;