Numbers with hyphens or strings of numbers with hyphens

Question

I need to make a pandas DataFrame that has a column filled with hyphenated numbers. The only way I could think of to do this was to use strings. This all worked fine, until I needed to sort them to get them back into order after a regrouping. The problem is that strings sort like this:

['100-200','1000-1100','1100-1200','200-300']

This is clearly not how I want it sorted. I want it sorted numberically. How would I get this to work? I am willing to change anything. Keeping the hyphenated string as an integer or float would be the best, but I am unsure how to do that.

Answer 1

You could use sorted to construct a new ordering for the index, and then perform the sort (reordering) using df.take:

import pandas as pd

df = pd.DataFrame({'foo':['100-200','1000-1100','1100-1200','200-300']})
order = sorted(range(len(df)),
               key=lambda idx: map(int, df.ix[idx, 'foo'].split('-')))
df = df.take(order)
print(df)

yields

         foo
0    100-200
3    200-300
1  1000-1100
2  1100-1200

This is similar to @275365's solution, but note that the sorting is done on range(len(df)), not on the strings. The strings are only used in the key parameter to determine the order in which range(len(df)) should be rearranged.

---

Using sorted works fine if the DataFrame is small. You can get better performance when the DataFrame is of moderate size (for example, a few hundred rows on my machine), by using numpy.argsort instead:

import pandas as pd
import numpy as np

df = pd.DataFrame({'foo':['100-200','1000-1100','1100-1200','200-300']*100})

arr = df['foo'].map(lambda item: map(int, item.split('-'))).values
order = np.argsort(arr)
df = df.take(order)

---

Alternatively, you could split your string column into two integer-valued columns, and then use df.sort:

import pandas as pd

df = pd.DataFrame({'foo':['100-200','1000-1100','1100-1200','200-300']})

df[['start', 'end']] = df['foo'].apply(lambda val: pd.Series(map(int, val.split('-'))))
df.sort(['start', 'end'], inplace=True)
print(df)

yields

         foo  start   end
0    100-200    100   200
3    200-300    200   300
1  1000-1100   1000  1100
2  1100-1200   1100  1200

Answer 2

You could try something like this:

>>> t = ['100-200','1000-1100','1100-1200','200-300']
>>> t.sort(key=lambda x: [int(y) for y in x.split('-')])
>>> t
['100-200', '200-300', '1000-1100', '1100-1200']

This would allow you to sort on integers, and if a hyphen exists, it will sort first by the first integer in the key list and then by the second. If no hyphen exists, you will sort just on the integer equivalent of the string:

>>> t = ['100-200','1000-1100','1100-1200','200-300', '100']
>>> t.sort(key=lambda x: [int(y) for y in x.split('-')])
>>> t
['100', '100-200', '200-300', '1000-1100', '1100-1200']

If you have any float equivalents in any strings, simply change int to float like this:

>>> t = ['100-200.3','1000.5-1100','1100.76-1200','200-300.75', '100.35']
>>> t.sort(key=lambda x: [float(y) for y in x.split('-')])
>>> t
['100-200.3', '100.35', '200-300.75', '1000.5-1100', '1100.76-1200']