1.1 Setting up Python<\/h3>\n
If someone needs to go through the Python installation on Windows, please watch this<\/a> link. You can download the Python from this<\/a> link.<\/p>\n Once the python is successfully installed on your system you can install the nltk<\/strong> toolkit using a simple Installation command<\/em><\/span><\/p>\n Before going any deeper in the practical let us download important nltk packages.<\/p>\n The approximate package size would be around ~3GB. To download the other packages we will use a simple python program which upon execution will open a dialogue box and you can click the download button to download and automatically install the packages.<\/p>\n downloadnltkpackages.py<\/em><\/span><\/p>\n If everything goes well the following window would open where you can click the download button to download and install the packages (like collections, corpora, models, or all packages). My recommendation is to choose all<\/em> packages as will require a majority of them in other practical implementations.<\/p>\n Tokenizing<\/strong> is a simple process of breaking up a sentence into a list of words and these words are known as tokens. There are 2 main tokenization techniques i.e.<\/p>\n Let us see a code implementation of how sentence tokenization works in python programming and for this, we will use the tokenizingsentences.py<\/em><\/span><\/p>\n If everything goes well the following output will be shown in the IDE console. Logs<\/em><\/span><\/p>\n Tagging<\/strong> is a simple process of reading a word and assigning it a specific token (popularly known as the part of speech). For tagging to work we use the Let us see a code implementation of word tagging in python programming.<\/p>\n taggingsentences.py<\/em><\/span><\/p>\n If everything goes well the following output will be shown in the IDE console.<\/p>\n Logs<\/em><\/span><\/p>\n As in the previous section, we saw how we can use tagging. Once the tagging of words is done it is important to understand the counting as well. Let us see a code implementation of how one can count tags.<\/p>\n countingtags.py<\/em><\/span><\/p>\n If everything goes well the following output will be shown in the IDE console.<\/p>\n Logs<\/em><\/span><\/p>\n That is all for this tutorial and I hope the article served you with whatever you were looking for. Happy Learning and do not forget to share!<\/p>\n In this tutorial, we learned:<\/p>\n You can download the source code of this tutorial from the Downloads<\/a> section.<\/p>\n This was a tutorial to understand the NLTK (Natural Language Toolkit) in python programming.<\/p>\n1.2 Setting up NLTK in Python<\/h3>\n
pip<\/code> command. You can fire the below command from the command prompt and it will successfully download the module from nltk<\/a> and install it.<\/p>\npip install nltk\n<\/pre>\n
2. NLTK (Natural Language Toolkit) Tutorial in Python<\/h2>\n
2.1 Downloading packages<\/h3>\n
# download nltk packages\n# nltk stands for Natural Language Tool Kit\nimport nltk\n# command will open the download window for nltk\n# click the download button to download the dataset as shown in fig1\nnltk.download()\n<\/pre>\n
![\"nltk](\"https:\/\/examples.javacodegeeks.com\/wp-content\/uploads\/2021\/02\/download-nltk-packages-img1.jpg\")
<\/a>2.2 Tokenizing sentences<\/h3>\n
\n
sent_tokenize(\u2026)<\/code> method from the nltk.tokenize.punkt<\/code> module. This method is well known to mark the beginning and end of sentences at character and punctuation.<\/p>\n# sentence tokenization\n# process of splitting up the sentence into a list of words and this list of words are known as tokens\nfrom nltk.tokenize import sent_tokenize\nrandom_text = \"Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has \" \\\n \"been the industry's standard dummy text ever since the 1500s, when an unknown printer took \" \\\n \"a galley of type and scrambled it to make a type specimen book. It has survived not only five \" \\\n \"centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It \" \\\n \"was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum \" \\\n \"passages, and more recently with desktop publishing software like Aldus PageMaker including \" \\\n \"versions of Lorem Ipsum.\"\n# sent_tokenize(...) uses an instance of PunktSentenceTokenizer\ntokenized_sentences = sent_tokenize(random_text)\n# print(tokenized_sentences)\nfor idx, tokenized_sentence in enumerate(tokenized_sentences):\n print('{} : {}'.format(idx, tokenized_sentence))\n<\/pre>\n0 : Lorem Ipsum is simply dummy text of the printing and typesetting industry.\n1 : Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book.\n2 : It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged.\n3 : It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.\n<\/pre>\n
2.3 Tagging sentences<\/h3>\n
DefaultTagger<\/code> class. Some of them are \u2013<\/p>\n\n\n
\n Abbreviation<\/strong><\/td>\n Description<\/strong><\/td>\n<\/tr>\n \n CC<\/td>\n coordinating conjunction<\/td>\n<\/tr>\n \n DT<\/td>\n determiner<\/td>\n<\/tr>\n \n FW<\/td>\n foreign word<\/td>\n<\/tr>\n \n IN<\/td>\n preposition\/subordinating conjunction<\/td>\n<\/tr>\n \n JJ<\/td>\n adjective (large)<\/td>\n<\/tr>\n \n JJR<\/td>\n adjective,comparative (larger)<\/td>\n<\/tr>\n \n JJS<\/td>\n adjective,superlative (largest)<\/td>\n<\/tr>\n \n NN<\/td>\n noun,singular (cat,tree)<\/td>\n<\/tr>\n \n NNS<\/td>\n noun plural (desks)<\/td>\n<\/tr>\n \n NNP<\/td>\n a proper noun, singular<\/td>\n<\/tr>\n \n NNPS<\/td>\n a proper noun, plural<\/td>\n<\/tr>\n \n RB<\/td>\n adverb (occasionally, swiftly)<\/td>\n<\/tr>\n \n RBR<\/td>\n adverb,comparative (greater)<\/td>\n<\/tr>\n \n TO<\/td>\n infinite marker (to)<\/td>\n<\/tr>\n \n UH<\/td>\n interjection (goodbye)<\/td>\n<\/tr>\n \n VB<\/td>\n verb (ask)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n # tagging sentences\n# tagging is responsible for reading a text in the language and assigning a specific token known to each word\n# pos tagger is used to assign grammatical information to each word in a sentence\n# importing part of speech tagging module\nfrom nltk import pos_tag\nrandom_text = \"Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has \" \\\n \"been the industry's standard dummy text ever since the 1500s, when an unknown printer took \" \\\n \"a galley of type and scrambled it to make a type specimen book. It has survived not only five \" \\\n \"centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It \" \\\n \"was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum \" \\\n \"passages, and more recently with desktop publishing software like Aldus PageMaker including \" \\\n \"versions of Lorem Ipsum.\".split()\n\ntags = pos_tag(random_text)\n# print(tags)\nfor idx, tag in enumerate(tags):\n print('{} : {}'.format(idx, tag))\n<\/pre>\n0 : ('Lorem', 'NNP')\n1 : ('Ipsum', 'NNP')\n2 : ('is', 'VBZ')\n3 : ('simply', 'RB')\n4 : ('dummy', 'JJ')\n5 : ('text', 'NN')\n6 : ('of', 'IN')\n7 : ('the', 'DT')\n8 : ('printing', 'NN')\n9 : ('and', 'CC')\n10 : ('typesetting', 'NN')\n11 : ('industry.', 'NN')\n12 : ('Lorem', 'NNP')\n\n\/\/ other statement omitted for brevity . . .\n<\/pre>\n2.4 Counting tags<\/h3>\n
# tagging sentences and counting them\n# importing the counter module for counting the elements\nfrom collections import Counter\n# importing part of speech tagging module\nfrom nltk import pos_tag\nrandom_text = \"Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has \" \\\n \"been the industry's standard dummy text ever since the 1500s, when an unknown printer took \" \\\n \"a galley of type and scrambled it to make a type specimen book. It has survived not only five \" \\\n \"centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It \" \\\n \"was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum \" \\\n \"passages, and more recently with desktop publishing software like Aldus PageMaker including \" \\\n \"versions of Lorem Ipsum.\".split()\ntags = pos_tag(random_text)\n# print(tags)\n# counting tags are important for the natural language operations\n# counter is a container that keeps the count of each element present in the container\ncounts = Counter(tag for word, tag in tags)\nprint(counts)\n<\/pre>\n
Counter({'NN': 20, 'NNP': 11, 'IN': 10, 'DT': 9, 'RB': 7, 'JJ': 5, 'CC': 4, 'NNS': 4, 'VBZ': 3, 'VBN': 3, 'VBD': 3, 'PRP': 3, 'VBG': 3, 'CD': 2, 'WRB': 1, 'TO': 1, 'VB': 1, 'RBR': 1})\n<\/pre>\n3. Summary<\/h2>\n
\n
<\/a>4. Download the Project<\/h2>\n