> ## Content Index
> Fetch the complete content index at: https://serpapi.com/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# How a routine gem update ended up creating $73k worth of subscriptions
- URL: https://serpapi.com/blog/how-a-routine-gem-update-ended-up-charging/
- Published: 2022-01-07T22:21:45.000Z
- Updated: 2023-05-16T22:09:13.000Z
- Author: Julien Khaleghy
- Tags: Business

On November 5, 2021 (a Friday of course), we've deployed innocent-looking gem updates. Minor versions of Ruby on Rails, Ruby Sentry client, Ruby Slack client, http libraries, Puma, Devise, OmniAuth Ruby client, Mongoid, and a few test gems.   
  
However, something went very wrong.   
  
We saw odd Stripe errors on Airbrake. Then, on our Stripe account, we saw this:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2021/11/image-1.png)

In less than 1h, we've created 474 new subscriptions for a total of $73,271.36.  
  
We weren't expecting that much business.  
  
Our app for some reason was creating new subscriptions from old accounts that were canceled or disabled a long time ago. We had no idea why but we decided to react proactively as fast as we could, as the issue was still very ongoing.  
  
We immediately rolled back the gem updates to their safe older versions, disabled our Stripe API keys to lock our own application out, and refunded everyone impacted:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2021/11/image-2.png)

After refunding everyone, we manually double-checked the billing state of each account one by one and sent emails to apologize to each customer one by one; all 474 of them.  
  
We still didn't know why.  
  
Most of the updates seemed inoffensive. And after deeper inspection, it was indeed minor gem updates on several small things. We worked on several hypotheses: caching issues, odd race conditions, or some thread safety issues as Puma changed their threading model in that update.  
  
However, we finally found it. One method of our billing logic particularly seemed a code smell (Obligatory disclaimer: I personally wrote this.):

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2021/12/image-18.png)

This is part of a feature that seems good on paper: We offer to automatically renew your plan if you run out of searches on our service.   
  
The goal of `renew_early_protected` is to avoid the case of someone submitting multiple searches at the exact same time, subsequently getting their subscription renewed then their credit card charged several times.  
  
Notice the `or()` method at line 47.  
  
In Mongoid (The MongoDB driver for Ruby) `7.0.8` , `or()` meant filter documents that contain any of the argument conditions. In our case, filter users that have a specific ID AND that have either never had a `renewing_early_lock` or their `renewing_early_lock` is less than 1 hour ago.  
  
Here's what it got computed to for a specific user:

```ruby
computed_selector_in_mongoid_7_0_8 = {
  "_id" => BSON::ObjectId('59af54094-----------64'), "$or" => [{
    "renewing_early_lock_at" => {
      "$lte" => 2022-01-07 20:14:53.44744 UTC
    }
  }, {
    "renewing_early_lock_at" => nil
  }]
}
```

In Mongoid `7.3.3` , `or()` now means filter documents that contain any of the argument conditions OR any of previous method conditions! In our case, filter users that have either a specific id OR that have never had a `renewing_early_lock` OR their `renewing_early_lock` is less than 1 hour ago.  
  
Here's what it looks like for the same user:

```ruby
computed_selector_in_mongoid_7_3_3 = {
  "$or" => [{
    "_id" => BSON::ObjectId('59af54094-----------64')
  }, {
    "renewing_early_lock_at" => {
      "$lte" => 2022-01-07 19:00:09.571034 UTC
    }
  }, {
    "renewing_early_lock_at" => nil
  }]
}
```

Notice that the user ID selector got moved into an optional `or()`!   
  
This ultimately means we were charging random customers new subscriptions instead of the actual customer who wanted to renew early:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2022/01/image-3.png)

Which is obviously awful.  
  
We are deeply sorry if you were impacted by this issue. This is not acceptable.  
  
Even if that Mongoid shouldn't have changed existing methods behaviors between minor versions, my implementation was a true code smell as it was unclear what it did. This code should never have been deployed to production. But also and more importantly, the feature itself was a bad idea. Using our API to scrape search engine results shouldn't trigger a renewal of a credit card subscription in the first place. And we'll be removing that feature as soon as possible.  
  
Sorry again to all of our users.