Python is known for its powerful\u00a0general purpose built-in data types like list<\/em>, dict<\/em>, tuple<\/em> and set<\/em>. \u00a0But Python also has collection objects like Java and C++. \u00a0These objects are developed on top of the general built-in containers with addtional functionalities which can be used in special scenarios.<\/p>\n The objective of this article is to introduce\u00a0python collection objects and explain them with apropriate\u00a0code snippets. \u00a0The collections library contains the collections objects, they are namedtuples (v2.6), deque (v2.4), ChainMap(v3.3), Counter(v2.7 ), OrderedDict(v2.7), defaultdict(v2.5) . \u00a0Python 3.x also has userDict, userList, userString to create own custom container\u00a0types (not in the scope of this article), which deserves a separate article.<\/p>\n NOTE: Python 2.x user might aware of various releases they are objects got introduced. \u00a0All these objects are available in Python 3.x from 3.1 onwards, except that ChainMap which \u00a0got introduced in v3.3. \u00a0All the code snippets in the articles are executed in Python 3.5 environment.<\/p>\n As the name suggests, namedtuple is a tuple with name. \u00a0In standard tuple, we access the elements using the index, whereas namedtuple allows user to define name for elements. \u00a0This is very handy especially processing csv (comma separated value) files and working with complex and large dataset, where the code becomes messy with the use of indices (not so pythonic).<\/p>\n Namedtuples are available in collections library in python. We have to import collections library before using any of container object from this library.<\/p>\n In the above code snippet, in the first line we import namedtuple from the collections library. In the second line we create a namedtuple called \u201csaleRecord\u201d, which has\u00a0shopId, saleDate, salesAmount and totalCustomers as fields. Note\u00a0that namedtuple() takes two string arguments, first argument is the name of tuple and second argument is the list of fields names seperated by space<\/strong> or comma<\/strong>. In the above example\u00a0space is used as delimeter. The above code is pretty much clear that tuple is accessed using the names. It is also possible to access them using indexes of the tuples which is the usual way.<\/p>\n The _make method is used to convert the given iteratable item (list, tuple,dictionary) into a named tuple.<\/p>\n The _fields is a tuple, which contains the names of the tuple.<\/p>\n As we discussed namedtuple will be very handy while processing a csv data file, where we can access the data using names instead of indexes, which make the code more meaningful and efficient.<\/p>\n In the above code snippet, we have the files salesRecord.csv which contains sales records of shops of a particular retain chain. It contains the values for the fields shopId,saleDate,totalSales,totalCustomers. The fields are delimited by comma and the records are delimited by new line. Counter is used for rapid tallies. \u00a0It is a dictionary, where the elements are stored as keys and their counts are stored as values.<\/p>\n The Counter() class takes an iteratable object as an argument and computes the count for each element in the object and present as a key value pair.<\/p>\n In the above code snippet, listOfInts<\/em> is a list which contains numbers. It is passed to Counter()<\/em> and we got cnt, which is a container object. The cnt<\/em> is a dictionary, which contains the unique numbers present in the given list as keys, and their respect counts as the value.<\/p>\n Counter is a subclass of dictionary. \u00a0So it can be accessed the same as dictionary. \u00a0 The \u201ccnt\u201d can be handled as a regular dictionary object.<\/p>\n The most_common(n)<\/em> of Counter class, provides most commonly occured keys. The n<\/em> is used as a rank, for example, n = 2<\/em> will provide top two keys.<\/p>\n In the above code, we could see that the string is parsed as independent characters as keys and their respective count is\u00a0stored as values. So, the letterCnt.most_common(1) provides the top letter which has highest occurances.<\/p>\n The Counter() subclass is also called as Multiset. It supports addition, substraction, unition and intersection operations on the Counter class.<\/p>\n The defaultdict() is available part of collections library. It allows the user to specify a function to be called when key is not present in the dictionary.<\/p>\n In a standard dictionary, accesing an element where the key is not present, will raise \u201cKey Error\u201d. So, this is a problem when working working with collections (list, set, etc), especially while creating them.<\/p>\n So, when a dictionary is queried for a key, which is not exists, the function passed as an argument to the named argument \u201cdefault_dictionary\u201d of default_dict() will called to set a value for given \u201ckey\u201d into dictionary.<\/p>\n The defaultdict() is available part of collections library. \u00a0The default dict takes a function without argument which returns value as an argument.<\/p>\n In the above example, the booksIndex<\/em> is a defaultdict, where it set \u2018Not Available<\/em>\u201d as a value if any non-existant key is accessed. We have added values for keys a, b & c into the defaultdict<\/em>. The print(booksIndex)<\/em> shows that the defaultdict<\/em> contains values only for these keys. While trying to access the value for key \u2018z\u2019, which we have not set, it returned value as \u2018Not Available<\/em>\u2018 and updated the dictionary.<\/p>\n In the above example, titleIndex<\/em> contains a list of tuples. We want to aggregate this list of tuples to identify titles for each alphabets. So, we can have a dictionary where key is the alphabet and value is the list of titles. Here we used a defaultdict with \u201clist\u201d as a function to be called for missing elements. So for each new elements list will be called, and it will create an empty list object. The consecutive append()<\/em> methods on the list will add elements to the list.<\/p>\n The ordered dictionary maintains\u00a0the order of elements addition into the dictionary, where the standard dictionary will not maintain the order of inclusion.<\/p>\n Ordered Dictionary is created using OrderedDict()<\/em> from collections library. It an subsclass of regular dictionary, so it inherits all other methods and behaviours of regular dictionary.<\/p>\n OrderedDict can also be created by passing an dictionary or a list of key, value pair tuples.<\/p>\n One of the interesting use case for OrderedDict is Rank problem. For example, consider the problem a dictionary contains students names and their marks, now we have to find out the best student and rank them according to their marks. So, OrderedDict is the right choice here. Since OrderedDict will remember the order or addition and sorted() will sort a dictionary we can combine both to created a rank list based on the student marks. Please check the example below:<\/p>\n In the above example, studentMarks<\/i> is a dictionary contains the student name as a key and their mark as the value. It got sorted using its value and passed to OrderedDict and got stored in rankOrder<\/i>. Now rankOrder<\/i> contains the highest marked student as the first entry, and next highest as the second entry and so on. This ordered is presevered in this dictionary.<\/p>\n Deque means double ended queue and it pronounced as \u201cdeck\u201d. It is an extention to the standard list data structure. The standard list allows the user to append or extend elements only at the end. But deque allows the user to operate on both ends, so that the user can implement both stacks and queues.<\/p>\n The deque() is available in collections library. It takes iteratable entity as an argument and an optional maximum length. If maxlen<\/em> is set, it ensure that deque length does not exceeds the size of the maxlen<\/em>.<\/p>\n In the above example, we have created a deque with maxlen 5, once we appended 6th element on the right, it pushed first element on the left. \u00a0Similarly, it pushes out the last element on the right when we append element on the left.<\/p>\n Operations on the right are common to performing any opertions on the list. \u00a0The methods append(), extend() and pop() are operate on the rightside of the deque().<\/p>\n The special feature of performing operations on the left is supported by set of methods like appendleft(), extendleft(), popleft().<\/p>\n If the maxlen value is not set, the deque does not perform any trimming operations to maintain the size of the deque.<\/p>\n From the above example, the deque aiao<\/em> continues to grow for the append and extend operations performed on it.<\/p>\n ChainMap allows to combine multiple\u00a0dictionaries into a single dictionary, so that operations can be performed on single logical entity. \u00a0The ChainMap() does not create any new dictionary, instead it maintains references to the original dictionaries, all operations are performed only on the referred dictionaries.<\/p>\n We have created ChainMap z from other dictionaries x & y. The ChainMap z is reference to the dictionaries x and y. ChainMap will not maintain duplicate keys, it returns presents value \u2018Cat\u2019 for key \u2018c\u2019. So, basically it skips the second occurance of the same key.<\/p>\n In the above code, we have removed the key \u2018c\u2019 from dict x. Now the ChainMap points the value for key \u2018c\u2019 to \u201cCharlie\u201d, which is present in y<\/em>.<\/p>\n We have seen various python collection data types and understand them with example and use cases. The official python documentation can be referred for further reading.<\/p>\n [1] \u2013 Python Wiki \u2013 https:\/\/docs.python.org\/3.5\/library\/collections.html<\/a><\/p>\n2. Namedtuple<\/h2>\n
2.1\u00a0Example 1<\/h3>\n
>>>from collections import namedtuple\r\n>>>saleRecord = namedtuple('saleRecord','shopId saleDate salesAmout totalCustomers')\r\n>>>\r\n>>>\r\n>>>#Assign values to a named tuple \r\n>>>shop11=saleRecord(11,'2015-01-01',2300,150) \r\n>>>shop12=saleRecord(shopId=22,saleDate=\"2015-01-01\",saleAmout=1512,totalCustomers=125)<\/pre>\n
\nWe have also created two tuples here. They are shop11 and shop12. \u00a0For shop11, the\u00a0values are assigned to fields based on the order of the fields and shop12, the values are\u00a0assigned using the names.<\/p>\n2.2\u00a0Example 2<\/h3>\n
>>>#Reading as a namedtuple\r\n>>>print(\"Shop Id =\",shop12.shopId)\r\n12\r\n>>>print(\"Sale Date=\",shop12.saleDate)\r\n2015-01-01\r\n>>>print(\"Sales Amount =\",shop12.salesAmount)\r\n1512\r\n>>>print(\"Total Customers =\",shop12.totalCustomers)\r\n125<\/pre>\n
2.3 Interesting Methods and Members<\/h3>\n
2.3.1\u00a0_make<\/h4>\n
>>>#Convert a list into a namedtuple\r\n>>>aList = [101,\"2015-01-02\",1250,199]\r\n>>>shop101 = saleRecord._make(aList)\r\n>>>print(shop101)\r\nsaleRecord(shopId=101, saleDate='2015-01-02', salesAmount=1250, totalCustomers=199)\r\n\r\n>>>#Convert a tuple into a namedtuple\r\n>>>aTup =(108,\"2015-02-28\",1990,189)\r\n>>>shop108=saleRecord._make(aTup)\r\n>>>print(shop108)\r\nsaleRecord(shopId=108, saleDate='2015-02-28', salesAmount=1990, totalCustomers=189)\r\n>>><\/pre>\n
2.3.2 _fields<\/h4>\n
>>>print(sho108._fields)\r\n>>>('shopId', 'saleDate', 'salesAmount', 'totalCustomers')<\/pre>\n2.4 CSV File Processing<\/h3>\n
from csv import reader\r\nfrom collections import namedtuple\r\n\r\nsaleRecord = namedtuple('saleRecord','shopId saleDate totalSales totalCustomers')\r\nfileHandle = open(\"salesRecord.csv\",\"r\")\r\ncsvFieldsList=csv.reader(fileHandle)\r\nfor fieldsList in csvFieldsList:\r\n shopRec = saleRecord._make(fieldsList)\r\n overAllSales += shopRec.totalSales;\r\n\r\nprint(\"Total Sales of The Retail Chain =\",overAllSales)<\/pre>\n
\nThe csv.reader() read the file and provides a iterator. The iterator, \u201ccsvFieldsList\u201d provides list of fields for every single row of the csv file. As we know the _make() converts the list into namedtuple and the rest of the code is self explanatory.<\/p>\n3.Counter<\/h2>\n
3.1 Creating Counters<\/h3>\n
>>>from collections import Counter\r\n>>>listOfInts=[1,2,3,4,1,2,3,1,2,1]\r\n>>>cnt=Counter(listOfInts)\r\n>>>print(cnt)\r\nCounter({1: 4, 2: 3, 3: 2, 4: 1})<\/pre>\n3.2 Accessing Counters<\/h3>\n
>>> cnt.items()\r\ndict_items([(1, 4), (2, 3), (3, 2), (4, 1)])\r\n>>> cnt.keys()\r\ndict_keys([1, 2, 3, 4])\r\n>>> cnt.values()\r\ndict_values([4, 3, 2, 1])<\/pre>\n
3.3 Interesting Methods & Usecases<\/h3>\n
3.3.1 most_common<\/h4>\n
>>>name = \"Saravanan Subramanian\"\r\n>>>letterCnt=Counter(name)\r\n>>>letterCnt.most_common(1)\r\n[('a', 7)]\r\n>>>letterCnt.most_common(2)\r\n[('a', 7), ('n', 4)]\r\n>>>letterCnt.most_common(3)\r\n[('a', 7), ('n', 4), ('r', 2)]<\/pre>\n3.3.2 Operations on Counter<\/h4>\n
>>> a = Counter(x=1,y=2,z=3)\r\n>>> b = Counter(x=2,y=3,z=4)\r\n>>> a+b\r\nCounter({'z': 7, 'y': 5, 'x': 3})\r\n>>> a-b #This will result in negative values & will be omitted\r\nCounter() \r\n>>> b-a\r\nCounter({'y': 1, 'x': 1, 'z': 1})\r\n>>> a & b #Chooses the minimum values from their respective pair\r\nCounter({'z': 3, 'y': 2, 'x': 1})\r\n>>> a | b #Chooses the maximum values from their respective pair\r\nCounter({'z': 4, 'y': 3, 'x': 2})<\/pre>\n4. Default Dictionary<\/h2>\n
4.1 Creating Default Dictionary<\/h3>\n
4.1.1 Example 1<\/h4>\n
>>> \r\n>>> booksIndex = defaultdict(lambda:'Not Available')\r\n>>> booksIndex['a']='Arts'\r\n>>> booksIndex['b']='Biography'\r\n>>> booksIndex['c']='Computer'\r\n>>> print(booksIndex)\r\ndefaultdict(<function at 0x030EB3D8>, {'c': 'Computer', 'b': 'Biography', 'a': 'Arts'})\r\n>>> booksIndex['z']\r\n'Not Available'\r\n>>> print(booksIndex)\r\ndefaultdict(<function at 0x030EB3D8>, {'c': 'Computer', 'b': 'Biography', 'z': 'Not Available', 'a': 'Arts'})\r\n>>><\/pre>\n4.1.2 Example 2<\/h4>\n
>>> titleIndex = [('a','Arts'),('b','Biography'),('c','Computer'),('a','Army'),('c','Chemistry'),('d','Dogs')]\r\n>>> rackIndices = defaultdict(list)\r\n>>> for id,title in titleIndex:\r\n\trackIndices[id].append(title)\t\r\n>>> rackIndices.items()\r\ndict_items([('d', ['Dogs']), ('b', ['Biography']), ('a', ['Arts', 'Army']), ('c', ['Computer', 'Chemistry'])])\r\n>>><\/pre>\n5. Ordered Dictionary<\/h2>\n
5.1 Ordered Dictionary Creation<\/h3>\n
>>> from collections import OrderedDict\r\n>>> dOrder=OrderedDict()\r\n>>> dOrder['a']='Alpha'\r\n>>> dOrder['b']='Bravo'\r\n>>> dOrder['c']='Charlie'\r\n>>> dOrder['d']='Delta'\r\n>>> dOrder['e']='Echo'\r\n>>> dOrder\r\n>>> OrderedDict([('a', 'Alpha'), ('b', 'Bravo'), ('c', 'Charlie'), ('d', 'Delta'), ('e', 'Echo')])\r\n>>> >>> dOrder.keys()\r\nodict_keys(['a', 'b', 'c', 'd', 'e'])\r\n>>> dOrder.values()\r\nodict_values(['Alpha', 'Bravo', 'Charlie', 'Delta', 'Echo'])\r\n>>> dOrder.items()\r\nodict_items([('a', 'Alpha'), ('b', 'Bravo'), ('c', 'Charlie'), ('d', 'Delta'), ('e', 'Echo')])\r\n>>><\/pre>\n5.2 Creating from other iteratable items<\/h3>\n
>>> from collections import OrderedDict\r\n>>> listKeyVals = [(1,\"One\"),(2,\"Two\"),(3,\"Three\"),(4,\"Four\"),(5,\"Five\")]\r\n>>> x = OrderedDict(listKeyVals)\r\n>>> x\r\nOrderedDict([(1, 'One'), (2, 'Two'), (3, 'Three'), (4, 'Four'), (5, 'Five')])\r\n>>><\/pre>\n
5.3 Sort and Store<\/h3>\n
>>> studentMarks={}\r\n>>> studentMarks[\"Saravanan\"]=100\r\n>>> studentMarks[\"Subhash\"]=99\r\n>>> studentMarks[\"Raju\"]=78\r\n>>> studentMarks[\"Arun\"]=85\r\n>>> studentMarks[\"Hasan\"]=67\r\n>>> studentMarks\r\n{'Arun': 85, 'Subhash': 99, 'Raju': 78, 'Hasan': 67, 'Saravanan': 100}\r\n>>> sorted(studentMarks.items(),key=lambda t:t[0])\r\n[('Arun', 85), ('Hasan', 67), ('Raju', 78), ('Saravanan', 100), ('Subhash', 99)]\r\n>>> sorted(studentMarks.items(),key=lambda t:t[1])\r\n[('Hasan', 67), ('Raju', 78), ('Arun', 85), ('Subhash', 99), ('Saravanan', 100)]\r\n>>> sorted(studentMarks.items(), key = lambda t:-t[1])\r\n[('Saravanan', 100), ('Subhash', 99), ('Arun', 85), ('Raju', 78), ('Hasan', 67)]\r\n>>> rankOrder = OrderedDict(sorted(studentMarks.items(), key = lambda t:-t[1]))\r\n>>> rankOrder\r\nOrderedDict([('Saravanan', 100), ('Subhash', 99), ('Arun', 85), ('Raju', 78), ('Hasan', 67)])<\/pre>\n6. Deque<\/h2>\n
6.1 Creation & Performing Operations on Deque<\/h3>\n
>>> from collections import deque\r\n>>> aiao = deque([1,2,3,4,5],maxlen=5)\r\naiao = deque([1,2,3,4,5])\r\n>>> aiao.append(6)\r\n>>> aiao\r\ndeque([2, 3, 4, 5, 6], maxlen=5)\r\n>>> aiao.appendleft(1)\r\n>>> aiao\r\ndeque([1, 2, 3, 4, 5], maxlen=5)<\/pre>\n
6.2\u00a0Operations on Right<\/h3>\n
>>> aiao.append(6)\r\n>>> aiao\r\ndeque([2, 3, 4, 5, 6], maxlen=5)\r\n>>> aiao.extend([7,8,9])\r\n>>> aiao\r\ndeque([5, 6, 7, 8, 9], maxlen=5)\r\n>>> aiao.pop()\r\n9<\/pre>\n
6.3 Operation\u00a0on the Left<\/h3>\n
>>> aiao = deque([1,2,3,4,5],maxlen=5)\r\n>>> aiao.appendleft(0)\r\n>>> aiao\r\ndeque([0, 1, 2, 3, 4], maxlen=5)\r\n>>> aiao.extendleft([-1,-2,-3])\r\n>>> aiao\r\ndeque([-3, -2, -1, 0, 1], maxlen=5)\r\n>>> aiao.popleft()\r\n-3<\/pre>\n
6.4\u00a0Example 2 (without maxlen)<\/h2>\n
>>> aiao = deque([1,2,3,4,5])\r\n>>> aiao.appendleft(0)\r\n>>> aiao\r\ndeque([0, 1, 2, 3, 4, 5])\r\n>>> aiao.extendleft([-1,-2,-3])\r\n>>> aiao\r\ndeque([-3, -2, -1, 0, 1, 2, 3, 4, 5])\r\n>>><\/pre>\n
7. ChainMap<\/h2>\n
7.1 Creating ChainMap<\/h3>\n
>>> from collections import ChainMap\r\n>>> x = {'a':'Alpha','b':'Beta','c':'Cat'}\r\n>>> y = { 'c': \"Charlie\", 'd':\"Delta\", 'e':\"Echo\"}\r\n>>> z = ChainMap(x,y)\r\n>>> z\r\nChainMap({'c': 'Cat', 'b': 'Beta', 'a': 'Alpha'}, {'d': 'Delta', 'c': 'Charlie', 'e': 'Echo'})\r\n>>> list(z.keys())\r\n['b', 'd', 'c', 'e', 'a']\r\n>>> list(z.values())\r\n['Beta', 'Delta', 'Cat', 'Echo', 'Alpha']\r\n>>> list(z.items())\r\n[('b', 'Beta'), ('d', 'Delta'), ('c', 'Cat'), ('e', 'Echo'), ('a', 'Alpha')]<\/pre>\n>>> x\r\n{'c': 'Cat', 'b': 'Beta', 'a': 'Alpha'}\r\n>>> y\r\n{'d': 'Delta', 'c': 'Charlie', 'e': 'Echo'}\r\n>>> x.pop('c')\r\n'Cat'\r\n>>> x\r\n{'b': 'Beta', 'a': 'Alpha'}\r\n>>> list(z.keys())\r\n['d', 'c', 'b', 'e', 'a']\r\n>>> list(z.values())\r\n['Delta', 'Charlie', 'Beta', 'Echo', 'Alpha']\r\n>>> list(z.items())\r\n[('d', 'Delta'), ('c', 'Charlie'), ('b', 'Beta'), ('e', 'Echo'), ('a', 'Alpha')]\r\n>>><\/pre>\n8. Summary<\/h2>\n
9. References<\/h2>\n