In this article, in order to remove duplicates from list python, we’ll learn particular ways through one of the operations to separate a unique list from a possible duplicate list. Removing such lists demands a large number of applications hence it makes this skill super useful.
Following are the given methods to follow to perform the operation.
Also Read: Typecasting in Python with Examples
1. Naive Method
While ignoring all the other occurrences of a particular element, in the Naive method, we can simply append the first occurrence after transversing the list to remove duplicates from the list python
# Python 3 code to demonstrate
#removing duplicates from list
#using naive methods
#initializing list
test_list = [1, 3, 5, 6, 3, 5, 6, 1]
print ("The original list is : " + str(test_list))
#using naive method
#to remove duplicated'''
#from list
res = []
for i in test_list:
if i not in res:
res.append(i)
#printing list after removal
print ("The list after removing duplicates : " + str(res))The original list is : [1, 3, 5, 6, 3, 5, 6, 1]
The list after removing duplicates : [1, 3, 5, 6]2. List Comprehension Method
As the List Comprehension Method works the same as the Naive Method, we will use a one-liner tactic to play shorthand remove for the same results.
#Python 3 code to demonstrate
#removing duplicates from list
#using list comprehension
# initializing list
test_list = [1, 3, 5, 6, 3, 5, 6, 1]
print ("The original list is : " + str(test_list))
#using list comprehension
#to remove duplicated
#from list
res = []
[res.append(x) for x in test_list if x not in res]
#printing list after removal
print ("The list after removing duplicates : " + str(res))The original list is : [1, 3, 5, 6, 3, 5, 6, 1]
The list after removing duplicates : [1, 3, 5, 6]3. List Comprehension Method & enumerate()
The List Comprehension Method done with enumerate() will simply skip the recurring elements after scanning them and print the results. The best part is that the ordering of your list will be preserved to remove duplicates from the list python.
#Python 3 code to demonstrate
#removing duplicated from list
#using list comprehension + enumerate()
#initializing list
test_list = [1, 5, 3, 6, 3, 5, 6, 1]
print ("The original list is : " + str(test_list))
#using list comprehension + enumerate()
#to remove duplicated
#from list
res = [i for n, i in enumerate(test_list) if i not in test_list[:n]]
#printing list after removal
print ("The list after removing duplicates : " + str(res))The original list is : [1, 5, 3, 6, 3, 5, 6, 1]
The list after removing duplicates : [1, 5, 3, 6]4. set() Method
Since this is the most used method to remove duplicates from the list, the only downside of using it is, you will lose the ordering of your list while removing the duplicates.
#Python 3 code to demonstrate
#removing duplicates from list
#using set()
#initializing list
test_list = [1, 5, 3, 6, 3, 5, 6, 1]
print ("The original list is : " + str(test_list))
using set()
#to remove duplicated
#from list
test_list = list(set(test_list))
#printing list after removal
distorted ordering
print ("The list after removing duplicates : " + str(test_list))The original list is : [1, 5, 3, 6, 3, 5, 6, 1]
The list after removing duplicates : [1, 3, 5, 6]5. collection.OrderDict.fromkeys() Method
This method is used to first remove the duplicates and then return a converted list of a dictionary which can also work with strings and is considered the fastest method among all.
#Python 3 code to demonstrate
#removing duplicates from list
#using collections.OrderedDict.fromkeys()
#from collections import OrderedDict
#initializing list
test_list = [1, 5, 3, 6, 3, 5, 6, 1]
print ("The original list is : " + str(test_list))
#using collections.OrderedDict.fromkeys()
#to remove duplicated
#from list
res = list(OrderedDict.fromkeys(test_list))
#printing list after removal
print ("The list after removing duplicates : " + str(res))The original list is : [1, 5, 3, 6, 3, 5, 6, 1]
The list after removing duplicates : [1, 5, 3, 6]The above methods are the basic category of list concepts we’ve covered in python. Stay updated on all the python concepts like Lists, Strings &, Arrays.
