Python pandas extracting hyphenated words from cells with phrases

Question

I have a dataframe which contain phrases and I want to extract only compound words separated by a hyphen from the dataframe and place them in another dataframe.

df=pd.DataFrame({'Phrases': ['Trail 1 Yellow-Green','Kim Jong-il was here', 'President Barack Obama', 'methyl-butane', 'Derp da-derp derp', 'Pok-e-mon'],})

So far here is what I got so far:

import pandas as pd

df=pd.DataFrame({'Phrases': ['Trail 1 Yellow-Green','Kim Jong-il was here', 'President Barack Obama', 'methyl-butane', 'Derp da-derp derp', 'Pok-e-mon'],})


new = df['Phrases'].str.extract("(?P<part1>.*?)-(?P<part2>.*)")

results

>>> new
            part1        part2
0  Trail 1 Yellow        Green
1        Kim Jong  il was here
2             NaN          NaN
3          methyl       butane
4         Derp da    derp derp
5             Pok        e-mon

What I want is to have just the word so it would be(note that Pok-e-mon appears as Nan due to 2 hyphens):

>>> new
            part1        part2
0          Yellow        Green
1             Jong          il
2             NaN          NaN
3          methyl       butane
4              da         derp
5             NaN          NaN

Answer 1

You can use this regex:

(?:[^-\w]|^)(?P<part1>[a-zA-Z]+)-(?P<part2>[a-zA-Z]+)(?:[^-\w]|$)

(?:               # non capturing group
    [^-\w]|^        # a non-hyphen or the beginning of the string
)
(?P<part1>
    [a-zA-Z]+     # at least a letter
)-(?P<part2>
    [a-zA-Z]+
)
(?:[^-\w]|$)        # either a non-hyphen character or the end of the string

• Your first problem is that nothing prevents the . from eating up spaces. [a-zA-Z] only select letters so it will avoid "jumping" from one word to another.
• For the pok-e-mon case, you need to check that there isn't a hyphen right before or after your match.

See Demo here

Answer 2

Given the specs I don't see where your first line of Nan, Nan is coming from. Maybe it is typo in your example? Anyways, here is a possible solution.

import re

# returns words with at least one hyphen
def split_phrase(phrase):
    return re.findall('(\w+(?:-\w+)+)', phrase)

# get all words with hyphens
words_with_hyphens = sum(df.Phrases.apply(split_phrase).values)
# split all words into parts
split_words = [word.split('-') for word in words_with_hyphens]
# keep words with two parts only, else return (Nan, Nan)
new_data = [(ws[0], ws[1]) if len(ws) == 2 else (np.nan, np.nan) for ws in split_words]
# create the new DataFrame
pd.DataFrame(new_data, columns=['part1', 'part2'])

#  part1   | part2
#------------------
# 0 Yellow | Green
# 1 Jong   | il
# 2 methyl | butane
# 3 da     | derp
# 4 NaN    | NaN