Selecting some part of a string from a column in hive and getting a count

Question

I have a column 'Platform' with rows as below.

name3:"string1/string2/string3/s:1.2.1/ABCD/XYZ".

And i have another column name with name 'name2'. My table looks like this

' id       |    name2     |   name3
-----------+--------------+---------------------
 1         |      x1      | string1/string2/string3/s:1.2.1/ABCD/XYZ
 2         |      x1      | string1/string2/string3/S:2.2.1/ABCD/XYZ
 3         |      x2      | string5/string4/string3/s:1.1/ABCD/XYZ
 4         |      x3      | string1/string6/string7/m:0.2.2/ABCD/XYZ
 5         |      x2      | string1/string2/string3/S:2.2.0/ABCD/XYZ'

I want to get counts of event based on substrings of platform. like

'name3     | X1    |   X2 |    X3    |

string4        |       |   1  |          |
string6        |       |      |   1      |'

or if i want to get the count based on just 'android' or 'iOS', how can i do that?

'name3     | X1    |   X2 |    X3    |

 string4          |       |   1  |          |
 string1      |   2   |   1  |   1      |'

The query i am using for counts is below. It is working fine for getting the counts of events but unable to figure out how the get the counts based on substrings.

'select name2,
    count(1) AS total
from table1 where name2='x1' OR name2='x2' OR name2='x3'
group by name2;'

Any suggestions?

Answer 1

First of all, I would split that string up into a view with actual columns. Something like:

create view my_view as select
id,
event,
regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 1) as os,
regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 2) as brand,
regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 3) as model,
regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 4) as lte,
regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 5) as abcd,
regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 6) as user,
regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 7) as xyz
from my_table;

Then querying that view is easier. You could also just have this as an inner-query. This sort of "count-where" query where you want different counts in different columns is a pretty common use. The best way I know of to do it is with the pattern:

sum(if( [condition] , 1, 0))

So for your examples, it would be:

select os,
sum(if(event = 'x1', 1, 0)) as x1,
sum(if(event = 'x2', 1, 0)) as x2,
sum(if(event = 'x3', 1, 0)) as x3
from my_view
group by os;

Or:

select brand,
sum(if(event = 'x1', 1, 0)) as x1,
sum(if(event = 'x2', 1, 0)) as x2,
sum(if(event = 'x3', 1, 0)) as x3
from my_view
group by brand;

And here's the above query, but using that view as an inner query instead of an actual view:

select brand,
sum(if(event = 'x1', 1, 0)) as x1,
sum(if(event = 'x2', 1, 0)) as x2,
sum(if(event = 'x3', 1, 0)) as x3
from (
  select
  id,
  event,
  regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 1) as os,
  regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 2) as brand,
  regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 3) as model,
  regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 4) as lte,
  regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 5) as abcd,
  regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 6) as user,
  regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$', 7) as xyz
  from my_table
) t
group by brand;

Answer 2

Hope this helps...

Query:

 select a.platforms, a.event, count(1) as count from 
 (select regexp_extract(platform, '^(.*)/(.*)/(.*)/(.*)/(.*)/(.*)/(.*)$',1) 
 as platforms, event from table1) a group by a.platforms, a.event;

Output:

platforms       event   count
android         x1      2
android         x2      1    
android         x3      1
ios             x2      1