(An Unofficial) Python FAQ Wiki

putting the community back in "maintained by the community"

How do you remove duplicates from a list?

See the Python Cookbook for a long discussion of many ways to do this:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560

If you don't mind reordering the list, sort it and then scan from the end of the list, deleting duplicates as you go:

if List:
   List.sort()
   last = List[-1]
   for i in range(len(List)-2, -1, -1):
       if last==List[i]: del List[i]
       else: last=List[i]

If all elements of the list may be used as dictionary keys (i.e. they are all hashable) this is often faster

d = {}
for x in List: d[x]=x
List = d.values()

CATEGORY: programming