{"id":1596,"date":"2020-01-09T11:59:54","date_gmt":"2020-01-09T06:29:54","guid":{"rendered":"https:\/\/www.guru99.com\/seq2seq-model.html"},"modified":"2024-10-03T19:15:36","modified_gmt":"2024-10-03T13:45:36","slug":"seq2seq-model","status":"publish","type":"post","link":"https:\/\/www.guru99.com\/seq2seq-model.html","title":{"rendered":"Seq2seq (Sequence to Sequence) Model with PyTorch","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<h2>What is NLP?<\/h2>\n<p>NLP or Natural Language Processing is one of the popular branches of Artificial Intelligence that helps computers understand, manipulate or respond to a human in their natural language. NLP is the engine behind Google Translate that helps us understand other languages.<\/p>\n<h2>What is Seq2Seq?<\/h2>\n<p><strong>Seq2Seq<\/strong> is a method of encoder-decoder based machine translation and language processing that maps an input of sequence to an output of sequence with a tag and attention value. The idea is to use 2 RNNs that will work together with a special token and try to predict the next state sequence from the previous sequence.<\/p>\n\n<style>.kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-table-of-content-wrap{padding-top:var(--global-kb-spacing-sm, 1.5rem);padding-right:var(--global-kb-spacing-sm, 1.5rem);padding-bottom:var(--global-kb-spacing-sm, 1.5rem);padding-left:var(--global-kb-spacing-sm, 1.5rem);background-color:#edf2f7;border-top-width:1px;border-right-width:1px;border-bottom-width:1px;border-left-width:1px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);max-width:450px;}.kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-table-of-contents-title-wrap{padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-table-of-contents-title{font-weight:regular;font-style:normal;}.kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-table-of-content-wrap .kb-table-of-content-list{font-weight:regular;font-style:normal;margin-top:var(--global-kb-spacing-sm, 1.5rem);margin-right:0px;margin-bottom:0px;margin-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-toggle-icon-style-basiccircle .kb-table-of-contents-icon-trigger:after, .kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-toggle-icon-style-basiccircle .kb-table-of-contents-icon-trigger:before, .kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-toggle-icon-style-arrowcircle .kb-table-of-contents-icon-trigger:after, .kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-toggle-icon-style-arrowcircle .kb-table-of-contents-icon-trigger:before, .kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-toggle-icon-style-xclosecircle .kb-table-of-contents-icon-trigger:after, .kb-table-of-content-nav.kb-table-of-content-id_57115a-80 .kb-toggle-icon-style-xclosecircle .kb-table-of-contents-icon-trigger:before{background-color:#edf2f7;}<\/style>\n\n<h2>How to Predict sequence from the previous sequence<\/h2>\n<p style=\"text-align:center;\"><a href=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ1.png\" data-lasso-id=\"542333\"><img decoding=\"async\" width=\"799\" height=\"292\" src=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ1.png\" alt=\"Predict Sequence from the Previous Sequence\" class=\"\"><\/a><\/p>\n<p>Following are steps for predict sequence from the previous sequence with PyTorch.<\/p>\n<div class='code-block code-block-2' style='margin: 8px 0; clear: both;'>\n<style>\n.guru99_incontent_21 {\n\tmin-height: 280px !important;\n  display: flex;\n  align-items: center;\n  justify-content: center;\n}\n<\/style>\n\n<div align=\"center\" id=\"guru99_mobile_display\" class=\"guru99_incontent_21\">\n    \n  <script>\n    googletag.cmd.push(function() { googletag.display('guru99_mobile_display'); });\n  <\/script>\n<\/div><\/div>\n\n<h3>Step 1) Loading our Data<\/h3>\n<p>For our dataset, you will use a dataset from <a href=\"http:\/\/www.manythings.org\/anki\/\" target=\"_blank\" rel=\"nofollow noopener\" data-lasso-id=\"186096\"> Tab-delimited Bilingual Sentence Pairs<\/a>. Here I will use the English to Indonesian dataset. You can choose anything you like but remember to change the file name and directory in the code.<\/p>\n<pre>from __future__ import unicode_literals, print_function, division\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport torch.nn.functional as F\n\nimport numpy as np\nimport pandas as pd\n\nimport os\nimport re\nimport random\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n<\/pre>\n<h3>Step 2) Data Preparation<\/h3>\n<p>You can&#8217;t use the dataset directly. You need to split the sentences into words and convert it into One-Hot Vector. Every word will be uniquely indexed in the Lang class to make a dictionary. The Lang Class will store every sentence and split it word by word with the addSentence. Then create a dictionary by indexing every unknown word for Sequence to sequence models.<\/p>\n<pre>SOS_token = 0\nEOS_token = 1\nMAX_LENGTH = 20\n\n#initialize Lang Class\nclass Lang:\n   def __init__(self):\n       #initialize containers to hold the words and corresponding index\n       self.word2index = {}\n       self.word2count = {}\n       self.index2word = {0: \"SOS\", 1: \"EOS\"}\n       self.n_words = 2  # Count SOS and EOS\n\n#split a sentence into words and add it to the container\n   def addSentence(self, sentence):\n       for word in sentence.split(' '):\n           self.addWord(word)\n\n#If the word is not in the container, the word will be added to it, \n#else, update the word counter\n   def addWord(self, word):\n       if word not in self.word2index:\n           self.word2index[word] = self.n_words\n           self.word2count[word] = 1\n           self.index2word[self.n_words] = word\n           self.n_words += 1\n       else:\n           self.word2count[word] += 1\n<\/pre>\n<p>The Lang Class is a class that will help us make a dictionary. For each language, every sentence will be split into words and then added to the container. Each container will store the words in the appropriate index, count the word, and add the index of the word so we can use it to find the index of a word or finding a word from its index.<\/p>\n<p>Because our data is separated by TAB, you need to use <a href=\"\/python-pandas-tutorial.html\" data-lasso-id=\"186097\">pandas<\/a> as our data loader. Pandas will read our data as dataFrame and split it into our source and target sentence. For every sentence that you have,<\/p>\n<ul>\n<li>you will normalize it to lower case,<\/li>\n<li>remove all non-character<\/li>\n<li>convert to ASCII from Unicode<\/li>\n<li>split the sentences, so you have each word in it.<\/li>\n<\/ul>\n<pre>#Normalize every sentence\ndef normalize_sentence(df, lang):\n   sentence = df[lang].str.lower()\n   sentence = sentence.str.replace('[^A-Za-z\\s]+', '')\n   sentence = sentence.str.normalize('NFD')\n   sentence = sentence.str.encode('ascii', errors='ignore').str.decode('utf-8')\n   return sentence\n\ndef read_sentence(df, lang1, lang2):\n   sentence1 = normalize_sentence(df, lang1)\n   sentence2 = normalize_sentence(df, lang2)\n   return sentence1, sentence2\n\ndef read_file(loc, lang1, lang2):\n   df = pd.read_csv(loc, delimiter='\\t', header=None, names=[lang1, lang2])\n   return df\n\ndef process_data(lang1,lang2):\n   df = read_file('text\/%s-%s.txt' % (lang1, lang2), lang1, lang2)\n   print(\"Read %s sentence pairs\" % len(df))\n   sentence1, sentence2 = read_sentence(df, lang1, lang2)\n\n   source = Lang()\n   target = Lang()\n   pairs = []\n   for i in range(len(df)):\n       if len(sentence1[i].split(' ')) &lt; MAX_LENGTH and len(sentence2[i].split(' ')) &lt; MAX_LENGTH:\n           full = [sentence1[i], sentence2[i]]\n           source.addSentence(sentence1[i])\n           target.addSentence(sentence2[i])\n           pairs.append(full)\n\n   return source, target, pairs\n<\/pre>\n<p>Another useful function that you will use is the converting pairs into Tensor. This is very important because our network only reads tensor type data. It&#8217;s also important because this is the part that at every end of the sentence there will be a token to tell the network that the input is finished. For every word in the sentence, it will get the index from the appropriate word in the dictionary and add a token at the end of the sentence.<\/p>\n<pre>def indexesFromSentence(lang, sentence):\n   return [lang.word2index[word] for word in sentence.split(' ')]\n\ndef tensorFromSentence(lang, sentence):\n   indexes = indexesFromSentence(lang, sentence)\n   indexes.append(EOS_token)\n   return torch.tensor(indexes, dtype=torch.long, device=device).view(-1, 1)\n\ndef tensorsFromPair(input_lang, output_lang, pair):\n   input_tensor = tensorFromSentence(input_lang, pair[0])\n   target_tensor = tensorFromSentence(output_lang, pair[1])\n   return (input_tensor, target_tensor)\n<\/pre>\n<div class='code-block code-block-3' style='margin: 8px 0; clear: both;'>\n<style>\n.guru99_incontent_21 {\n\tmin-height:280px !important;\n}\n<\/style>\n\n<!-- Tag ID: guru99_static_3 -->\n<div id='guru99_incontent_2' class=\"guru99_incontent_21\">\n  <script>\n    googletag.cmd.push(function() { googletag.display('guru99_incontent_2'); });\n  <\/script>\n<\/div>\n<div class='yarpp yarpp-related yarpp-related-shortcode yarpp-template-yarpp-template-custom'>\n<div class=\"custom-related-posts\">\n    <h3 class=\"related-title\">RELATED ARTICLES<\/h3>\n    <ul>\n            <li>\n            <a href=\"https:\/\/www.guru99.com\/download-install-nltk.html\" rel=\"bookmark\" title=\"How to Download &#038; Install NLTK on Windows\/Mac\">\n                How to Download &#038; Install NLTK on Windows\/Mac            <\/a>\n        <\/li>\n            <li>\n            <a href=\"https:\/\/www.guru99.com\/tokenize-words-sentences-nltk.html\" rel=\"bookmark\" title=\"NLTK Tokenize: Words and Sentences Tokenizer with Example\">\n                NLTK Tokenize: Words and Sentences Tokenizer with Example            <\/a>\n        <\/li>\n            <li>\n            <a href=\"https:\/\/www.guru99.com\/pos-tagging-chunking-nltk.html\" rel=\"bookmark\" title=\"POS Tagging with NLTK and Chunking in NLP [EXAMPLES]\">\n                POS Tagging with NLTK and Chunking in NLP [EXAMPLES]            <\/a>\n        <\/li>\n            <li>\n            <a href=\"https:\/\/www.guru99.com\/stemming-lemmatization-python-nltk.html\" rel=\"bookmark\" title=\"Stemming and Lemmatization in Python NLTK with Examples\">\n                Stemming and Lemmatization in Python NLTK with Examples            <\/a>\n        <\/li>\n        <\/ul>\n<\/div>\n<\/div>\n<\/div>\n\n<h2>Seq2Seq Model<\/h2>\n<figure style=\"text-align:center;\"><a href=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ2.png\" data-lasso-id=\"542334\"><img decoding=\"async\" width=\"800\" height=\"184\" src=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ2.png\" alt=\"Seq2seq Model\" class=\"\"><\/a><figcaption style=\"text-align:center;\">Seq2Seq<\/figcaption><\/figure>\n<p>PyTorch Seq2seq model is a kind of model that use PyTorch encoder decoder on top of the model. The Encoder will encode the sentence word by words into an indexed of vocabulary or known words with index, and the decoder will predict the output of the coded input by decoding the input in sequence and will try to use the last input as the next input if its possible. With this method, it is also possible to predict the next input to create a sentence. Each sentence will be assigned a token to mark the end of the sequence. At the end of prediction, there will also be a token to mark the end of the output.  So, from the encoder, it will pass a state to the decoder to predict the output.<\/p>\n\n\n<figure style=\"text-align:center;\"><a href=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ3.png\" data-lasso-id=\"542335\"><img decoding=\"async\" width=\"686\" height=\"455\" src=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ3.png\" alt=\"Seq2seq Model\" class=\"\"><\/a><figcaption style=\"text-align:center;\">Seq2Seq Model<\/figcaption><\/figure>\n\n\n<p>The Encoder will encode our input sentence word by word in sequence and in the end there will be a token to mark the end of a sentence. The encoder consists of an Embedding layer and a GRU layers. The Embedding layer is a lookup table that stores the embedding of our input into a fixed sized dictionary of words. It will be passed to a GRU layer. GRU layer is a  Gated Recurrent Unit that consists of multiple layer type of <a href=\"\/rnn-tutorial.html\" data-lasso-id=\"186100\">RNN<\/a> that will calculate the sequenced input. This layer will calculate the hidden state from the previous one and update the reset, update, and new gates.<\/p>\n<figure style=\"text-align:center;\"><a href=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ4.png\" data-lasso-id=\"542336\"><img decoding=\"async\" width=\"800\" height=\"407\" src=\"https:\/\/www.guru99.com\/images\/1\/111318_0848_seq2seqSequ4.png\" alt=\"Seq2seq Model\" class=\"\"><\/a><\/p><figcaption style=\"text-align:center;\">Seq2Seq<\/figcaption><\/figure>\n<p>The Decoder will decode the input from the encoder output. It will try to predict the next output and try to use it as the next input if it&#8217;s possible. The Decoder consists of an Embedding layer, GRU layer, and a Linear layer. The embedding layer will make a lookup table for the output and pass it into a GRU layer to calculate the predicted output state. After that, a Linear layer will help to calculate the activation function to determine the true value of the predicted output.<\/p>\n<pre>class Encoder(nn.Module):\n   def __init__(self, input_dim, hidden_dim, embbed_dim, num_layers):\n       super(Encoder, self).__init__()\n      \n       #set the encoder input dimesion , embbed dimesion, hidden dimesion, and number of layers \n       self.input_dim = input_dim\n       self.embbed_dim = embbed_dim\n       self.hidden_dim = hidden_dim\n       self.num_layers = num_layers\n\n       #initialize the embedding layer with input and embbed dimention\n       self.embedding = nn.Embedding(input_dim, self.embbed_dim)\n       #intialize the GRU to take the input dimetion of embbed, and output dimention of hidden and\n       #set the number of gru layers\n       self.gru = nn.GRU(self.embbed_dim, self.hidden_dim, num_layers=self.num_layers)\n              \n   def forward(self, src):\n      \n       embedded = self.embedding(src).view(1,1,-1)\n       outputs, hidden = self.gru(embedded)\n       return outputs, hidden\n\nclass Decoder(nn.Module):\n   def __init__(self, output_dim, hidden_dim, embbed_dim, num_layers):\n       super(Decoder, self).__init__()\n\n#set the encoder output dimension, embed dimension, hidden dimension, and number of layers \n       self.embbed_dim = embbed_dim\n       self.hidden_dim = hidden_dim\n       self.output_dim = output_dim\n       self.num_layers = num_layers\n\n# initialize every layer with the appropriate dimension. For the decoder layer, it will consist of an embedding, GRU, a Linear layer and a Log softmax activation function.\n       self.embedding = nn.Embedding(output_dim, self.embbed_dim)\n       self.gru = nn.GRU(self.embbed_dim, self.hidden_dim, num_layers=self.num_layers)\n       self.out = nn.Linear(self.hidden_dim, output_dim)\n       self.softmax = nn.LogSoftmax(dim=1)\n      \n   def forward(self, input, hidden):\n\n# reshape the input to (1, batch_size)\n       input = input.view(1, -1)\n       embedded = F.relu(self.embedding(input))\n       output, hidden = self.gru(embedded, hidden)       \n       prediction = self.softmax(self.out(output[0]))\n      \n       return prediction, hidden\n\nclass Seq2Seq(nn.Module):\n   def __init__(self, encoder, decoder, device, MAX_LENGTH=MAX_LENGTH):\n       super().__init__()\n      \n#initialize the encoder and decoder\n       self.encoder = encoder\n       self.decoder = decoder\n       self.device = device\n     \n   def forward(self, source, target, teacher_forcing_ratio=0.5):\n\n       input_length = source.size(0) #get the input length (number of words in sentence)\n       batch_size = target.shape[1] \n       target_length = target.shape[0]\n       vocab_size = self.decoder.output_dim\n      \n#initialize a variable to hold the predicted outputs\n       outputs = torch.zeros(target_length, batch_size, vocab_size).to(self.device)\n\n#encode every word in a sentence\n       for i in range(input_length):\n           encoder_output, encoder_hidden = self.encoder(source[i])\n\n#use the encoder\u2019s hidden layer as the decoder hidden\n       decoder_hidden = encoder_hidden.to(device)\n  \n#add a token before the first predicted word\n       decoder_input = torch.tensor([SOS_token], device=device)  # SOS\n\n#topk is used to get the top K value over a list\n#predict the output word from the current target word. If we enable the teaching force,  then the #next decoder input is the next word, else, use the decoder output highest value. \n\n       for t in range(target_length):   \n           decoder_output, decoder_hidden = self.decoder(decoder_input, decoder_hidden)\n           outputs[t] = decoder_output\n           teacher_force = random.random() &lt; teacher_forcing_ratio\n           topv, topi = decoder_output.topk(1)\n           input = (target[t] if teacher_force else topi)\n           if(teacher_force == False and input.item() == EOS_token):\n               break\n\n       return outputs\n<\/pre>\n<h3>Step 3) Training the Model<\/h3>\n<p>The training process in Seq2seq models is starts with converting each pair of sentences into Tensors from their Lang index. Our sequence to sequence model will use SGD as the optimizer and NLLLoss function to calculate the losses. The training process begins with feeding the pair of a sentence to the model to predict the correct output. At each step, the output from the model will be calculated with the true words to find the losses and update the parameters. So because you will use 75000 iterations, our sequence to sequence model will generate random 75000 pairs from our dataset.<\/p>\n<pre>teacher_forcing_ratio = 0.5\n\ndef clacModel(model, input_tensor, target_tensor, model_optimizer, criterion):\n   model_optimizer.zero_grad()\n\n   input_length = input_tensor.size(0)\n   loss = 0\n   epoch_loss = 0\n   # print(input_tensor.shape)\n\n   output = model(input_tensor, target_tensor)\n\n   num_iter = output.size(0)\n   print(num_iter)\n\n#calculate the loss from a predicted sentence with the expected result\n   for ot in range(num_iter):\n       loss += criterion(output[ot], target_tensor[ot])\n\n   loss.backward()\n   model_optimizer.step()\n   epoch_loss = loss.item() \/ num_iter\n\n   return epoch_loss\n\ndef trainModel(model, source, target, pairs, num_iteration=20000):\n   model.train()\n\n   optimizer = optim.SGD(model.parameters(), lr=0.01)\n   criterion = nn.NLLLoss()\n   total_loss_iterations = 0\n\n   training_pairs = [tensorsFromPair(source, target, random.choice(pairs))\n                     for i in range(num_iteration)]\n  \n   for iter in range(1, num_iteration+1):\n       training_pair = training_pairs[iter - 1]\n       input_tensor = training_pair[0]\n       target_tensor = training_pair[1]\n\n       loss = clacModel(model, input_tensor, target_tensor, optimizer, criterion)\n\n       total_loss_iterations += loss\n\n       if iter % 5000 == 0:\n           avarage_loss= total_loss_iterations \/ 5000\n           total_loss_iterations = 0\n           print('%d %.4f' % (iter, avarage_loss))\n          \n   torch.save(model.state_dict(), 'mytraining.pt')\n   return model\n<\/pre>\n<div class='code-block code-block-4' style='margin: 8px 0; clear: both;'>\n<style>\n.guru99_incontent_31 {\n\tmin-height:280px !important;\n}\n<\/style>\n\n<!-- Tag ID: guru99_static_4 -->\n<div id='guru99_incontent_3' class=\"guru99_incontent_31\">\n  <script>\n    googletag.cmd.push(function() { googletag.display('guru99_incontent_3'); });\n  <\/script>\n<\/div><\/div>\n\n<h3>Step 4) Test the Model<\/h3>\n<p>The evaluation process of Seq2seq PyTorch is to check the model output. Each pair of Sequence to sequence models will be feed into the model and generate the predicted words. After that you will look the highest value at each output to find the correct index. And in the end, you will compare to see our model prediction with the true sentence<\/p>\n<pre>def evaluate(model, input_lang, output_lang, sentences, max_length=MAX_LENGTH):\n   with torch.no_grad():\n       input_tensor = tensorFromSentence(input_lang, sentences[0])\n       output_tensor = tensorFromSentence(output_lang, sentences[1])\n  \n       decoded_words = []\n  \n       output = model(input_tensor, output_tensor)\n       # print(output_tensor)\n  \n       for ot in range(output.size(0)):\n           topv, topi = output[ot].topk(1)\n           # print(topi)\n\n           if topi[0].item() == EOS_token:\n               decoded_words.append('&lt;EOS&gt;')\n               break\n           else:\n               decoded_words.append(output_lang.index2word[topi[0].item()])\n   return decoded_words\n\ndef evaluateRandomly(model, source, target, pairs, n=10):\n   for i in range(n):\n       pair = random.choice(pairs)\n       print(\u2018source {}\u2019.format(pair[0]))\n       print(\u2018target {}\u2019.format(pair[1]))\n       output_words = evaluate(model, source, target, pair)\n       output_sentence = ' '.join(output_words)\n       print(\u2018predicted {}\u2019.format(output_sentence))\n<\/pre>\n<p>Now, let&#8217;s start our training with Seq to Seq, with the number of iterations of 75000 and num of RNN layer of 1 with the hidden size of 512.<\/p>\n<pre>lang1 = 'eng'\nlang2 = 'ind'\nsource, target, pairs = process_data(lang1, lang2)\n\nrandomize = random.choice(pairs)\nprint('random sentence {}'.format(randomize))\n\n#print number of words\ninput_size = source.n_words\noutput_size = target.n_words\nprint('Input : {} Output : {}'.format(input_size, output_size))\n\nembed_size = 256\nhidden_size = 512\nnum_layers = 1\nnum_iteration = 100000\n\n#create encoder-decoder model\nencoder = Encoder(input_size, hidden_size, embed_size, num_layers)\ndecoder = Decoder(output_size, hidden_size, embed_size, num_layers)\n\nmodel = Seq2Seq(encoder, decoder, device).to(device)\n\n#print model \nprint(encoder)\nprint(decoder)\n\nmodel = trainModel(model, source, target, pairs, num_iteration)\nevaluateRandomly(model, source, target, pairs)\n<\/pre>\n<p>As you can see, our predicted sentence is not matched very well, so in order to get higher accuracy, you need to train with a lot more data and try to add more iterations and number of layers using Sequence to sequence learning.<\/p>\n<pre>random sentence ['tom is finishing his work', 'tom sedang menyelesaikan pekerjaannya']\nInput : 3551 Output : 4253\nEncoder(\n  (embedding): Embedding(3551, 256)\n  (gru): GRU(256, 512)\n)\nDecoder(\n  (embedding): Embedding(4253, 256)\n  (gru): GRU(256, 512)\n  (out): Linear(in_features=512, out_features=4253, bias=True)\n  (softmax): LogSoftmax()\n)\nSeq2Seq(\n  (encoder): Encoder(\n    (embedding): Embedding(3551, 256)\n    (gru): GRU(256, 512)\n  )\n  (decoder): Decoder(\n    (embedding): Embedding(4253, 256)\n    (gru): GRU(256, 512)\n    (out): Linear(in_features=512, out_features=4253, bias=True)\n    (softmax): LogSoftmax()\n  )\n)\n\n5000 4.0906\n10000 3.9129\n15000 3.8171\n20000 3.8369\n25000 3.8199\n30000 3.7957\n35000 3.8037\n40000 3.8098\n45000 3.7530\n50000 3.7119\n55000 3.7263\n60000 3.6933\n65000 3.6840\n70000 3.7058\n75000 3.7044\n\n&gt; this is worth one million yen\n= ini senilai satu juta yen\n&lt; tom sangat satu juta yen &lt;EOS&gt;\n\n&gt; she got good grades in english\n= dia mendapatkan nilai bagus dalam bahasa inggris\n&lt; tom meminta nilai bagus dalam bahasa inggris &lt;EOS&gt;\n\n&gt; put in a little more sugar\n= tambahkan sedikit gula\n&lt; tom tidak &lt;EOS&gt;\n\n&gt; are you a japanese student\n= apakah kamu siswa dari jepang\n&lt; tom kamu memiliki yang jepang &lt;EOS&gt;\n\n&gt; i apologize for having to leave\n= saya meminta maaf karena harus pergi\n&lt; tom tidak maaf karena harus pergi ke\n\n&gt; he isnt here is he\n= dia tidak ada di sini kan\n&lt; tom tidak &lt;EOS&gt;\n\n&gt; speaking about trips have you ever been to kobe\n= berbicara tentang wisata apa kau pernah ke kobe\n&lt; tom tidak &lt;EOS&gt;\n\n&gt; tom bought me roses\n= tom membelikanku bunga mawar\n&lt; tom tidak bunga mawar &lt;EOS&gt;\n\n&gt; no one was more surprised than tom\n= tidak ada seorangpun yang lebih terkejut dari tom\n&lt; tom ada orang yang lebih terkejut &lt;EOS&gt;\n\n&gt; i thought it was true\n= aku kira itu benar adanya\n&lt; tom tidak &lt;EOS&gt;\n<\/pre>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>What is NLP? NLP or Natural Language Processing is one of the popular branches of Artificial Intelligence that helps computers understand, manipulate or respond to a human in their natural language. NLP is the engine behind Google Translate that helps us understand other languages. What is Seq2Seq? Seq2Seq is a method of encoder-decoder based machine&#8230;<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":41,"featured_media":61095,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[75],"tags":[154,146],"coauthors":[478],"class_list":["post-1596","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-nltk","tag-convertbox-developer","tag-non-amp"],"taxonomy_info":{"category":[{"value":75,"label":"NLTK"}],"post_tag":[{"value":154,"label":"Convertbox-Developer"},{"value":146,"label":"Non AMP"}]},"featured_image_src_large":["https:\/\/www.guru99.com\/images\/seq2seq-model.png",371,130,false],"author_info":{"display_name":"Evelyn Clarke","author_link":"https:\/\/www.guru99.com\/author\/evelyn"},"comment_info":0,"category_info":[{"term_id":75,"name":"NLTK","slug":"nltk","term_group":0,"term_taxonomy_id":75,"taxonomy":"category","description":"","parent":0,"count":9,"filter":"raw","cat_ID":75,"category_count":9,"category_description":"","cat_name":"NLTK","category_nicename":"nltk","category_parent":0}],"tag_info":[{"term_id":154,"name":"Convertbox-Developer","slug":"convertbox-developer","term_group":0,"term_taxonomy_id":154,"taxonomy":"post_tag","description":"","parent":0,"count":823,"filter":"raw"},{"term_id":146,"name":"Non AMP","slug":"non-amp","term_group":0,"term_taxonomy_id":146,"taxonomy":"post_tag","description":"","parent":0,"count":1292,"filter":"raw"}],"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/posts\/1596","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/users\/41"}],"replies":[{"embeddable":true,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/comments?post=1596"}],"version-history":[{"count":1,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/posts\/1596\/revisions"}],"predecessor-version":[{"id":85072,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/posts\/1596\/revisions\/85072"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/media\/61095"}],"wp:attachment":[{"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/media?parent=1596"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/categories?post=1596"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/tags?post=1596"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.guru99.com\/wp-json\/wp\/v2\/coauthors?post=1596"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}