> ## Content Index
> Fetch the complete content index at: https://serpapi.com/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Real World Example of Machine Learning on Rails
- URL: https://serpapi.com/blog/real-world-example-of-machine-learning-on-rails/
- Published: 2022-02-09T21:10:49.000Z
- Updated: 2023-05-19T20:53:50.000Z
- Author: Emirhan Akdeniz
- Tags: AI ML, Google, Rails, Ruby

This week we'll talk about how to implement ML on Rails using [torch.rb](https://github.com/ankane/torch.rb), [torchtext-ruby](https://github.com/ankane/torchtext-ruby), and [gruff](https://github.com/topfunky/gruff) in detail.

**Changes in procedure this week:**  
\- Dumping of maximum word size (can be found on #1 of the blog post series)

### Creating Dataset

  
Before we start how we created a custom dataset using SerpApi engines, let me clarify one point. [Google Local Pack Results](https://serpapi.com/local-pack) are given or taken the same kind as [Google Local Results](https://serpapi.com/local-results). 

Former shows itself in the organic search results page (without using any extra parameters). But they represent the same thing. Their key fields are also the same. So we can train a model for classifying ingredients of Google Local Results, and then use it for Google Local Pack Results.

We decided to use a rake function to create our custom datasets. For that we’ll use a preset JSON file we will parse and then fill using SerpApi.

```Json
{
  "local_results": {
    "title": [],
    "place_id_search": [],
    "thumbnail": [],
    "rating": [],
    "reviews": [],
    "type": [],
    "phone": [],
    "address": [],
    "hours": [],
    "price": [],
    "description": [],
    "place_id": [],
    "lsig": []
  },
  "queries": [
    "McDonald’s",
    "Starbucks",
    "Subway",
    ...
  ],
  "params": "engine=google&tbm=lcl",
  "query_parameter": "q"
}
```

`local_results` here represents the type of results, all the keys within it are different keys we extract in Google Local Results.  
`queries` is a set of queries to create the dataset. In this case it contains famous restaurants and coffeehouses in the US.  
`params` are relevant parameters in a normal SerpApi Search.  
`query_parameter` represents the parameter key to be used in the search.

Rake command will use the path of the JSON file as an argument to fill it with the relevant data. Each result we get from SerpApi will be checked if its key names are the same as key names within the JSON and then it’ll be added inside the dataset.

Here is an example for Google Local Result and [Playground link for the example](https://serpapi.com/playground?&async=true&engine=google&gl=us&google%5Fdomain=google.com&hl=en&location=Austin%2C+Texas%2C+United+States&no%5Fcache=true&q=Starbucks&tbm=lcl):

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/ex_lcl.png)

Image of Examples

###   
Loading the Data

We need to create some class variables to be used within the custom data loader and pass some of them to the trainer.

```Ruby
    def initialize(json_file_path)
      super()
      @@all_letters = []
      @@data = File.read(json_file_path)
      @@data.split('').each {|letter| (@@all_letters << letter) unless @@all_letters.include? letter}
      @@n_letters = @@all_letters.size
    end
```

`@@all_letters` is filled with each unique character within the JSON file. This is helpful for creating an index for different kinds of data which we will come in a second.  
`@@data` is the JSON hash of the file we read.  
`@@n_letters` is our dictionary size. It’ll be the size of every unique character used to create our results.

  
We also need a way to get back our dictionary, and dictionary size in the training module:

```Ruby
    def self.declare_dictionary
      return @@all_letters, @@n_letters
    end
```

In order to avoid reading and writing issues with special characters, we need to transform them to Unicode:

```Ruby
    def self.to_unicode string
      I18n.transliterate string
    end
```

We need only the necessary parts we want to extract from `@@data` to work with it:

```Ruby
    def self.read_from_json
      data = JSON.parse(@@data)
      data = data[data.keys.first]
      data
    end
```

Each letter needs to be indexed within the dictionary (or alphabet, I use them interchangeably in this context) before we transform them into tensors.

```Ruby
    def self.letter_to_index letter
      letter = to_unicode(letter)
      @@all_letters.find_index letter
    end
```

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/ex1.png)

Sorry for the quality of handwriting :)

  
From there we make tensors out of them with the size (1 x @@n\_letters):

```Ruby
    def self.letter_to_tensor letter
      tensor = Torch.zeros(1, @@n_letters)
      tensor[0][letter_to_index(letter)] = 1
      tensor
    end
```

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/ex2.png)

Thus each word will be consisting of letter tensors:

```Ruby
    def self.word_to_tensor word
      tensor = Torch.zeros(word.size, 1, @@n_letters)
      word.split('').each_with_index do |letter, index|
        tensor[index][0][letter_to_index(letter)] = 1
      end
      tensor
    end
```

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/ex3.png)

###   
Creating The Model  

  
Initialize our model within a class:

```Ruby
    def initialize input_size, hidden_size, output_size
      super()
      @hidden_size = hidden_size
      @i2h = Torch::NN::Linear.new(input_size + hidden_size, hidden_size)
      @i2o = Torch::NN::Linear.new(input_size + hidden_size, output_size)
      @softmax = Torch::NN::LogSoftmax.new(dim: 1)
    end
```

We feed the input\_size which is …, hidden\_size which is …, output\_size which is … in our case.

`@i2h` represents `Input to Hidden`  
`@i2o` represents `Input to Output`  
`@softmax` here is the logarithm of softmax function in mathematics. Softmax is a probability distribution. In our case it’ll be the probability distribution of different keys within the SerpApi local results. Key with the highest probability will be given as an output.

Here's the visualization of the model:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/exmodel.png)

We need a way to forward the input data within the model.

```Ruby
    def forward input, hidden
      combined = Torch.cat( [input, hidden], dim:1)
      hidden = @i2h.call(combined)
      output = @i2o.call(combined)
      output = @softmax.call(output)
      return output, hidden
    end
```

We also need a way to initialize hidden layer zero tensor to be filled within the flow.

```Ruby
    def init_hidden
      Torch.zeros(1, @hidden_size)
    end
```

  
**IV - Training The Model**

There are necessary things we must define first. I’ll explain them in detail on next week’s blog post alongside some other details we will conclude. But simply put;

`@n_epochs` → Number of iterations a.k.a epochs  
`@print_every` → At which rate do we want to print the current situation of the model on the terminal.  
`@plot_every` → At which rate do we want to plot the progress to a graph.  
`@learning_rate` → Learning rate is responsible for the adjustment of weights within a machine learning model. In other words. It needs to be adjusted optimally.

Let me give a semantic example on learning rate:

Let’s assume you are teaching someone how to read. You taught the person the word `Scraper`, `Skipper`, `Metamorphosis`, and `Pneumonoultramicroscopicsilicovolcanoconiosis` (yes, a real word in English Language), alongside some other words. If you show `Scraper`, and ask him what this is;

- If he tells you it is `Scraper` it is a good learning rate.  
It might tell you that is is something else until it learns. But you’ll recognize a sweet spot of failures, and you’ll still have a hope that this person will learn how to speak.
- If he tells you it is `Skipper` it is a low learning rate.  
This means the person is lost in details. Too many similar letters will mean same word to this person.
- If he tells you it is `Metamorphosis`, it is a high learning rate.  
The gap in differentiating words is too high for this person. This person is a deductive person now.
- If he tells you it is `Pneumonoultramicroscopicsilicovolcanoconiosis`, it is a very high learning rate.  
No way he’s gonna learn.

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/exlr-2.png)

`@n_categories` → Number of categories, number of different keys in our case

`@n_hidden` → Size of Hidden layer. It could be tweaked to get different results. Also you may have different sets of hidden layers in front of each other with different sizes to get different results.

`@net` → Our Model

`@optimizer` → Stochiastic Gradient Descent function that is useful for adjusting weights between nodes to come up with an optimized model. Each iteration with positive outcome, match of inputted value, and outputted key in our case, will increase the model’s leniency to go from there given the input is similar, and each iteration with the negative outcome will decrease the model’s leniency to go from there given the input is dissimilar. With enough iteration, model will be optimized.

`@criterion` → Negative Log Likelihood is a function we use to determine the loss of a model. Loss is useful to observe the optimization of the model throughout different epochs.

```Ruby
  @n_epochs = 1000
  @print_every = 1000
  @plot_every = 1
  @learning_rate = 0.005
  @start = DateTime.now.to_f

  DataTools.new "ml/google/local_pack/data/local_pack_en-us.json"
  @all_letters, @n_letters = DataTools.declare_dictionary
  @data = DataTools.read_from_json
  @all_categories = @data.keys
  @n_categories = @data.keys.size
  @n_hidden = 128

  @net = GLocalNet.new(@n_letters, @n_hidden, @n_categories)
  @optimizer = Torch::Optim::SGD.new(@net.parameters, lr: @learning_rate)
  @criterion = Torch::NN::NLLLoss.new
  @current_loss = 0
  @all_losses = []
```

Now that we declared our necessary variables, let’s take a look at other functions we need.

This function is responsible for giving the highest possible output and its index within all categories:

```Ruby
  def self.category_from_output output
    top_n, top_i = Torch.topk(output.data, 1)
    category_i = top_i[0].item
    return @all_categories[category_i], category_i
  end
```

This function is responsible for picking a random value from an array:

```Ruby
  def self.random_choice arr
    max = arr.size - 1
    arr[rand(0..max)]
  end
```

This function is responsible for creating random training pairs. It picks a random category (key names in our case), picks a random word from that category, translates both of them into tensors and returns it. You can consider it as creating a random pop-quiz for the model.

```Ruby
  def self.random_training_pair
    category = random_choice(@all_categories)
    word = random_choice(@data[category])
    category_tensor = Torch.tensor([@all_categories.index(category)],dtype: :long)
    word_tensor = DataTools.word_to_tensor word
    return category, word, category_tensor, word_tensor
  end

```

This function is responsible for one epoch of training. It takes the category tensor and word tensor we created randomly, initiates a hidden layer, zeroes out the gradients (to keep it in buffers instead of overwriting), and then calls each letter tensor of the word tensor within the model. Output from that interaction is then used for measuring the loss of the epoch. Backward propagation is activated to accumulate buffers and then single optimization step is taken. Predicted output and the loss of the epoch are returned.

```Ruby
  def self.train category_tensor, word_tensor
    hidden = @net.init_hidden

    @net.zero_grad

    (0..word_tensor.size.first-1).each do |index|
      @output, hidden = @net.call(word_tensor[index], hidden)
    end
      
    loss = @criterion.call(@output, category_tensor)
    loss.backward
    @optimizer.step

    return @output, loss.data.item
  end
```

We also need a way to measure time for the overall process. It seems unimportant. But the training time of the model is truly one of the key features in scaling it.

```Ruby
  def self.time_since since
    now = DateTime.now.to_f
    seconds = now - since
    minutes = (seconds / 60).floor
    seconds = seconds - (minutes * 60)
    "#{minutes} minutes #{seconds} seconds"
  end
```

This next part is the brain of our training process where the data is trained. We train the model `@n_epochs` times. To calculate success rate, we’ll pass `total_passed` variable which’ll increase with each positive outcome.

At each epoch, we create a random training pair, train the model one time, get the current loss of our model, and check if the prediction is correct. Success rate will be created to measure the success of the model with the state of prediction. We’ll print the results at every `@print_every`. We’ll plot at every `@plot_every` to plot loss over epochs, and success rate over epochs.

```Ruby
  def self.iterate_epochs
    total_passed = 0
    success_rates = []
    (1..(@n_epochs+1)).each do |epoch|
      category, word, category_tensor, word_tensor = random_training_pair
      output, loss = train category_tensor, word_tensor
      @current_loss = @current_loss + loss

      guess, guess_i = category_from_output output
      state = guess == category ? "Passed" : "Failed"

      if state == "Passed"
        total_passed = total_passed + 1
      end
      
      success_rate = 100*(total_passed.to_f / epoch.to_f).round(6)

      if epoch % @print_every == 0
        puts "\n |Epoch: #{epoch} |\n "\
             "|Progress: %#{((epoch.to_f / @n_epochs.to_f).round(6))*100} |\n "\
             "|Time: #{time_since(@start)} |\n "\
             "|Loss: #{loss} |\n "\
             "|Guessed Value: #{word} |\n "\
             "|Guessed Field: #{guess} |\n "\
             "|Status: #{state} |\n "\
             "|Success Rate: %#{success_rate} |\n"\
             "----------------------------"
      end

      if epoch % @plot_every == 0
        @all_losses.append(@current_loss / @plot_every)
        success_rates.append(success_rate)
        @current_loss = 0
      end
    end

    plot_line_loss = Gruff::Line.new
    plot_line_loss.title = "Loss over #{@n_epochs} Epochs (At Every #{@plot_every} Epoch)"
    plot_line_loss.data :Loss, @all_losses
    plot_line_loss.write('ml/google/local_pack/predict_value/trained_models/rnn_value_predictor_loss.png')

    plot_line_sr = Gruff::Line.new
    plot_line_sr.title = "Succress Rate(%) over #{@n_epochs} Epochs (At Every #{@plot_every} Epoch)"
    plot_line_sr.data :Success, success_rates
    plot_line_sr.write('ml/google/local_pack/predict_value/trained_models/rnn_value_predictor_sr.png')
  end
```

We’ll also need to save the current state of the model to a file in order to use it:

```Ruby
  def self.save_model
    Torch.save(@net.state_dict, "ml/google/local_pack/predict_value/trained_models/rnn_value_predictor.pth")
  end
```

## Example Results

Note that the results here are not conclusive. This is a model trained with 1000 epochs (fairly low). Thus, these results are only to show the functionality of the implementation.  
  
Here is an example of a printed result:

```Ruby
 |Epoch: 1000 |
 |Progress: %100.0 |
 |Time: 0 minutes 5.2206950187683105 seconds |
 |Loss: 2.568265676498413 |
 |Guessed Value: Permanently closed |
 |Guessed Field: title |
 |Status: Failed |
 |Success Rate: %27.900000000000002 |
----------------------------
```

  
Here is an example of Success Rate over Epochs:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/rnn_value_predictor_sr.png)

As you can see, with enough training, it seems like it can get to a state where it’ll predict the overwhelming majority of the inputs correctly.

Here is an example of Loss over Epochs:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/02/rnn_value_predictor_loss.png)

As you can see, low level of training is exposing oscillation within the loss. However the trend seems to be healthy.

## Conclusion

I am grateful for the incredible support I received from the SerpApi team in crucial parts of the implementation. Also, I’m grateful for the creator and maintainer of gems we used to make this possible, and PyTorch for their incredible documentation.

 Next week, I’ll go deeper into the explanations of the model, make some tweaks in it, and share the comparative results. Thanks for reading.

Acknowledgments:

- Gems Used:  
[torch.rb](https://github.com/ankane/torch.rb)  
[torchtext-ruby](https://github.com/ankane/torchtext-ruby)  
[gruff](https://github.com/topfunky/gruff)
- C++ Libraries Used:  
[LibTorch 1.10.2, Linux, CUDA 10.2, cxx11 ABI](https://pytorch.org/get-started/locally/)
- Materials Repurposed From:  
[Blog Post](https://towardsdatascience.com/https-medium-com-dashingaditya-rakhecha-understanding-learning-rate-dd5da26bb6de)  
[Documentation](https://pytorch.org/tutorials/intermediate/char%5Frnn%5Fclassification%5Ftutorial.html)