Setting up multiple Okta orgs with the same Omniauth Oauth2 Strategy in Rails

Hello There,

I want to share what I did in order to support a second Okta Organization in a Ruby on Rails application using omniauth_oktaoauth gem.

I started with a basic spec to make sure I dont break anything:

RSpec.describe 'Sign in with Okta', type: :request do
  describe "Using Okta" do
    let(:user) { User.find_by email: user_email }
    let(:okta_oauth) do
      {
        provider: okta_provider.to_s,
        uid: "123456789",
        info: {
          name: "John Doe",
          email: user_email,
        }
      }
    end

    before(:each) do
      OmniAuth.config.test_mode = true
      OmniAuth.config.mock_auth[okta_provider] = OmniAuth::AuthHash.new(okta_oauth)
    end

    context "When using existing configuration" do
      let(:okta_provider) { :oktaoauth }
      let(:user_email) { "john.okta@existing.domain" }

      it "keeps working" do
        post "/users/auth/oktaoauth"
        follow_redirect!
        expect(user.email).to eq(user_email)
      end
    end
  end
end

With this, I proceed to write another spec to ensure the new option would work:

context "When using second okta org" do
  let(:okta_provider) { :second_okta }
  let(:user_email) { "john.okta@new.domain" }

  it "signs the right user in" do
    post "/users/auth/second_okta"
    follow_redirect!

    expect(user.email).to eq(user_email)
  end
end

After this, I started to use the normal steps:

# user.rb
devise :omniauthable, omniauth_providers: [:oktaoauth, :second_okta]

Then, modify the initializer to tell omniauth about the new strategy

# config/initializers/okta.rb
....
config.omniauth(:second_okta,
                Rails.configuration.second_okta.client_id,
                Rails.configuration.second_okta.client_secret,
                name: "second_okta",
                request_path: "/users/auth/second_okta",
                callback_path: "/users/auth/second_okta/callback",
                scope: "openid profile email",
                fields: %w[profile email],
                client_options: {
                  site: Rails.configuration.second_okta.url,
                  authorize_url: "#{Rails.configuration.second_okta.auth_issuer}/v1/authorize",
                  token_url: "#{Rails.configuration.second_okta.auth_issuer}/v1/token"
                },
                redirect_uri: "#{Rails.configuration.app.base_domain}/users/auth/second_okta/callback",
                issuer: Rails.configuration.second_okta.auth_issuer,
                strategy_class: OmniAuth::Strategies::Oktaoauth)

At the beginning, it looked easy to add a second option using the same strategy, but after dealing with omniauth internals and omniauth_oktaoauth source code itself, I found that I needed to specify name and strategy_class to override defaults, there's something inside the source code that did not work out of the box, I had to specify request_path and callback_path explicitly (I'll dig deeper later and send a patch if it's a bug).

After making those changes, it worked just fine.

Thanks!

Separate health check endpoint using puma

Puma offers a way to query its internal stats by enabling a controll app in a separate port, this can be useful when we need to know if the app is alive, this is different than normal health check endpoints because it does not get processed by rails at all.

To enable this functionality, all you need to do is to add this line in your puma.rb file:

activate_control_app 'tcp://0.0.0.0:9293', { no_token: true }

It will start a second web server in the port 9293 that can be queried by monitoring tools or even balancer healthcheck.

ecruz@Edwins-MBP % curl 'http://127.0.0.1:9293/stats'
{"started_at":"2021-10-15T21:39:55Z","workers":2,"phase":0,"booted_workers":2,"old_workers":0,"worker_status":[{"started_at":"2021-10-15T21:39:55Z","pid":44969,"index":0,"phase":0,"booted":true,"last_checkin":"2021-10-15T21:40:05Z","last_status":{"backlog":0,"running":5,"pool_capacity":5,"max_threads":5,"requests_count":0}},{"started_at":"2021-10-15T21:39:55Z","pid":44970,"index":1,"phase":0,"booted":true,"last_checkin":"2021-10-15T21:40:05Z","last_status":{"backlog":0,"running":5,"pool_capacity":5,"max_threads":5,"requests_count":0}}]}%
ecruz@Edwins-MBP %

Check the documentation for more options/usages

Profiles in docker compose, specify default services

It is common to have multiple services listed in the docker-compose file, sometimes with dependencies, but also, a normal workflow sometimes is to just start everything, like:

docker-compose -f docker-compose.yml -d

And it will start all servies listed with their dependencies, but, what happens if you need to add another service and you do not need it to start along with the rest? The answer is: profiles:

services:
  db:
    image: pg
  cache:
    image: redis
  rails:
    depends_on:
      - db
      - cache
    command: puma
  sidekiq:
    profiles: [ 'jobs' ]
    depends_on:
      - db
      - cache
    command: sidekiq

So, in this case, whenever you do docker-compose up -d it will start only rails with its dependencies: db and cache, it wont start sidekiq by default, but if you really want to start sidekiq, then you need to explicitly type it: docker-compose up sidekiq -d.

Eager load rails associations with nested scopes

It is common to apply some extra scopes when fetching AR relationships, for examples, if we have countries and states, we might want all the countries starting with the letter A and all their states that starts with the letter B, this will automatically create a n+1 query problem since it nees to iterate over each country and fetch all states, but, Rails provides a way to eager load these associations easily:

states_scope = State.where("name ilike 'b%'")
countries = Country.where("name ilike 'a%'")
# This is the magic
ActiveRecord::Associations::Preloader.new.preload(countries, :states, states_scope)

# Now you can invoke coutries.each(:states) and it wont cause queries N+1
countries.map {|country| { country.id => country.states.size }

Normally, you would have to define another relationship in order to eager load the association, but it is not needed using this approach:

class Country < AR::Base
  has_many :states
  has_many :states_starting_with_b, -> { where("name ilike 'b%'") }, foreign_key: :state_id, class_name: "State"
end

# Then
Country.includes(:states_starting_with_b).where("name ilike 'a%'")

But this approach does not scale, it requires to define tons of relationships

Docker compose services with health checks

If you need to add extra checks for services to be up and running before starting another one, you can use healtcheck property:

services:
  pg:
    image: pg/pg-12
    ports:
      - 5432:5432
    healtcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres" ]
      interval: 5s
      timeout: 5s
      retries: 5
  redis:
    image: redis:5.0.4-alpine
    ports:
      - 6380:6379
    healthcheck:
      test: [ "CMD", "redis-cli", "ping" ]
      interval: 5s
      timeout: 5s
      retries: 5
  app:
    image: railsapp
    depends_on:
      pg:
        condition: service_healthy
      redis:
        condition: service_healthy
    ports: ['3000:3000']
    command: |
      bash -c "bundle exec rails s"

Rollback a specific Rails migration

bundlde exec rails db:migrate:down VERSION=202101010000001

Where 202101010000001 is the migration you want to rollback

Control email deliveries in Rails with a custom Interceptor

You can easily tell Rails to control email delivering with a custom mailer interceptor, all you need to do is to implement the class method delivering_email:

class DeliverOrNotEmailInterceptor
  def self.delivering_email(email)
    mail.perform_deliveries = !email.to.end_with?('special-domain.com')
  end
end

# config/initializer/email_interceptors.rb
ActionMailer::Base.register_interceptor(DeliverOrNotEmailInterceptor)

How to check if an ActiveRecord association was already eager loaded

if you need to know whether to use a pre-laoded collection using eager loading or fech the collections, you can always call loaded? method:

if user.contact_phones.loaded?
  user.contact_phones.detect? {|phone| phone.primary }
else
  user.contact_phones.find_by(primary: true)
end

Using Gem::Dependency class manually to ensure version matching

I needed to add a mechanism to ensure that some actions of a controller were available only for specific versions, and I thought it was super similar of what Gemfile does, so I started to look how to use Gem::Dependency to solve this, and turns out it was super easy:

class Controller
  def feeds_one
    check_for_version_support('>= 1.0', '< 3')

    render json: SomeData.all
  end

  def feeds_two
    check_for_version_support('= 1.0')

    render json: SomeData.all
  end

  def feeds_three
    check_for_version_support('>= 2.1')

    render json: SomeData.all
  end

  private

  def check_for_version_support(*specification)
    checker = Gem::Dependency.new(action_name, specification)

    return if checker.match?(action_name, params[:version])

    raise VersionNotSupported, "Version not upported"
  end
end

Using Upsert with Rails

Some DBMS support natively something that behaves like update or insert, Rails recently added the method Upsert that can take advantage of this method.

It is useful to update information that do not need to run validations, meaning, in a super performant way, here's an example:

# We usually do this:
activity = current_user.activity
if activity 
  activity.update(last_seen_at: Time.current)
else
  current_user.activity.create(last_seen_at: Time.current)
end

But if you see, it does not need to run any validation, it just needs to update last_seen_at if exists or create a new one, it performs two queries: one to instanciate the activity object and a second one that performs the real update/insert statement.

That can be replaced with the following code and it will perform just a single query and it will take care to either create the record or update an existing one

Activity.upsert({ last_seen_at: Time.current, user_id: current_user.id}, unique_by: :user_id)

To make this really work, considering you use Postgresql, you have to add a unique index on user_id and modify default value on created_at and updated_at in a migration like this:

query = <<-SQL
  ALTER TABLE #{Activity.table_name}
  ALTER COLUMN created_at SET DEFAULT CURRENT_TIMESTAMP,
  ALTER COLUMN updated_at SET DEFAULT CURRENT_TIMESTAMP
SQL
execute(query)

Better usage of Rails logger

Logging usefull data is a hard task, but there's one specific method that helps to improve the experience of logging actual useful information: tagged. It adds extra tags to the log message making it easy to debug:

Rails.logger.tagged('Super App') do
  Rails.logger.info('Log Message')
end

This will result in somethin like this:

[Super App] Log Message

If you can see, it prepends usefull information, you could add personalized data to trace logs, for example:

class ApplicationController << ActionController::Base
  around_action :add_logger_tags

  def add_logger_tags
    Rails.logger.tagged(logging_tags) do
      yield
    end
  end

  def logging_tags
    [
      "Request Id: #{request.id}",
      "Session Id: #{session.id}",
      current_user && "User id: #{current_user.id}"
    ]
end

And you will have super nice logs to read

Creating an infinite loop in Ruby

If you need to iterate over a collection that its size is unknown, you can create an infinite loops in ruby easily, but if you need an index, then there's this option:

(1..).each do |page|
  results = ExternalService.get('/endpoint', page: page)
  parsed = JSON.parse(response.body)
  break if parsed['data'].empty?

  process_data(parsed['data'])
end

Count similar items in an array in Ruby

Before ruby 2.7, you would have to do some hacky code like:

array.inject({}) {|f, v| f[v] ||= 0; f[v] += 1 ; f }

But starting with Ruby 2.7, there's a nicer way:

array.tally

irb: [1, 2, 3, 4, 4, 5, 5].tally
=> {1=>1, 2=>1, 3=>1, 4=>2, 5=>2}

Printing queries executed by ActiveRercord

Starting of Rails 6, there's a handy way to specify the level of verbocity to ActiveRecord queries:

ActiveRecord::Base.verbose_query_logs = true

Full documentation here

No more adding a custom logger to ActiveRecord::Base.logger

Updating ActiveRecord models without loading an instance using Postgresql

There're sometimes that we need to update an ActiveRecord Model but it is not necesary to load an instance, the normal flow would be the following:

profile = UserProfile.find_by(user_id: id)
profile.update(last_seen_at: Time.now)

The problem with this is that we load a useless instance of UserProfile, it is not needed, in a high traffic sites, an extra select query can count a lot, but luckly, Rails has addressed this with upsert command:

UserProfile.upsert({ last_seen_at: Time.now, user_id: id }, unique_by: :user_id)

This will use native Postgresql upsert command to update a record if it exists or insert a new one and no select will be performed, everything in a single query instead of two.

To make it really work, you need to modify your created_at and updated_at columns to have default current_timestamp

Using Rails to migrate columns from JSON to JSONB in Postgresql

Postgres offers data type json to store any structure easily, but one dissadvantage is that filtering by properties stored in the json column are super slow, one simple fix before refactoring the whole implementation is to migrate the column to be jsonb, since it is stored in binary form, it supports indexes, a easy and safe way to do it is as follows:

class ModifyJSONDataDataType < ActiveRecord::Migration[6.0]
  def up
    add_column :table_name, :data_jsonb, :jsonb, default: '{}'

    # Copy data from old column to the new one
    TableName.update_all('data_jsonb = data::jsonb')

    # Rename columns instead of modify their type, it's way faster
    rename_column :table_name, :data, :data_json
    rename_column :table_name, :data_jsonb, :data
  end

  def down
    safety_assured do
      rename_column :table_name, :data, :data_jsonb
      rename_column :table_name, :data_json, :data
    end
  end
end

Then, using another migration(due the ability to disable transactions), add an index to it

class AddIndexToDataInTableName < ActiveRecord::Migration[6.0]
  disable_ddl_transaction!

  def change
    add_index :table_name, :data, name: "data_index", using: :gin, algorithm: :concurrently

    # You can even add indexes to virtual properties:
    # add_index :table_name, "((data->'country')::text)", :name => "data_country_index", using: 'gin', algorithm: :concurrently
  end
end

Inserting images in markdown posts

If you want to insert an image inside a markdown body, you can do it with:

![alt text](https://cdn.host.com/image.png "Title")

Example:

Invoking ActiveRecord::Migration functions in command line

If you need for some weird reason, to run migration commands in your rails console, just do the following:

ActiveRecord::Migration.add_index :table, :col, name: 'col_index'

Native Pub/Sub in Rails with ActiveSupport::Notifications

If you want to use pub/sub design pattern inside your rails app, there's no need to add extra dependencies, you can use ActiveSupport::Notifications to do the job

Example:

class Order
  # methods

  def complete
    update(complted_at: Time.zone.now, etc: 'some')
    ActiveSupport::Notifications.instrument("order_completed", { number: order.number })
  end
end

module Subscribers
  class SendConfirmationEmail
    def self.subscribe!
      ActiveSupport::Notifications.subscribe("order_completed") do |_name, _start, _finish, _id, params|
        order = Order.find_by number: params[:number]
        OrderMailer.confirm_order(order).deliver_later
      end
    end
  end
end

module Subscribers
  class UpdateCustomerCRM
    def self.subscribe!
      ActiveSupport::Notifications.subscribe("order_completed") do |_name, _start, _finish, _id, params|
        order = Order.find_by number: params[:number]
        CrmIntegration.update_customer(order.customer.email, order.total_amount)
      end
    end
  end
end

# etc

Rails find_by using relationships

If you want to find records via a relationship you can do it easily:

product = Product.joins(:variants).find_by(variants: { sku: 'SKU' }

Instead of:

product = Variant.find_by(sku: 'SKU')&.product

Measuring memory utilization

Ruby comes with the cool module Benchmark, but the only downside is that it only provides the time taken for each task.

A simple approach to get how many memory the ruby process is consuming is to call: ps -o rss:

memory_before = `ps -o rss= -p #{$$}`.to_i
Utils.heavy_memory_consumming_process
memory_after = `ps -o rss= -p #{$$}`.to_i

puts "The process took #{memory_after - memory_before} KB"

Editing a file using cat

If you have to modify a file because any editor is present (vi, vim, nano, etc). you can use cat to do it:

$ cat > filename.ext
write or paste the content

ctrl+d

And that would be it! Super helpful when debugging flaky tests that only fail in the CI server

When to eager load relationships in Rails and when it is not that good

Rails provides a way to eager load relationships when fetching objects, the main idea is to avoid queries N+1, but, when isn't a good idea when to do it?

Good

When rendering unique results that can not be cached, for example: table reports

Why? Most of the times you need to display related information

orders = Order.includes(:user, :line_items).completed

Try to avoid

When you use fragment cache

Why? Eager load is executed before the rendering, regardless the final result is already cached or not. If using eager loading, it will always be executed, but when allowing queries n+1 that query will be executed once to fill the cache, and that's it

products = Product.includes(:categories, variants: [:price]).search(keywords)

Use product.id & updated_at to fill a fragment cache and fetch the data from database only when needed, no extra info needed such as variants, categories, prices, etc

Connecting Ruby to AWS IoT Core using MQTT client

If you need to use Ruby to connect to Aws Iot Core, this is all you need:

require 'aws-sdk-iot'
require 'aws-sdk-secretsmanager'
require 'json'
require 'mqtt'

secrets_manager = Aws::SecretsManager::Client.new(
    region: ENV["IOT_AWS_REGION"],
    access_key_id: ENV["IOT_AWS_ACCESS_KEY"],
    secret_access_key: ENV["IOT_AWS_SECRET_ACCESS_KEY"]
    )

client = Aws::IoT::Client.new(
    region: ENV["IOT_AWS_REGION"],
    access_key_id: ENV["IOT_AWS_ACCESS_KEY"],
    secret_access_key: ENV["IOT_AWS_SECRET_ACCESS_KEY"]
    )

# Creates new ssl certificate
cert = client.create_keys_and_certificate(set_as_active: true)

# A policy named iot-mqtt needs to exist with permissions to publish and read
# any topic names
client.attach_policy(policy_name: "iot-mqtt", target: cert.certificate_arn)

# Stores the certificate in aws secrets manager
secrets_manager.create_secret(name: "iot_cert_pem", secret_string: cert.certificate_pem)
secrets_manager.create_secret(name: "iot_private_key", secret_string: cert.key_pair.private_key)

# Reads the certificate from aws secrets manager
cert_pem = secrets_manager.get_secret_value(secret_id: "iot_cert_pem").secret_string
private_key = secrets_manager.get_secret_value(secret_id: "iot_private_key").secret_string

# Connects to aws iot core endpoint using mqtts
mqtt_client = MQTT::Client.new(ENV["IOT_AWS_ENDPOINT"])
mqtt_client.cert = cert_pem
mqtt_client.key = private_key
mqtt_client.connect(MQTT::Client.generate_client_id("my-awesome-app-"))

# Publishes a message
message = { desired: { speed_limit: 35 } }
mqtt_client.publish("$aws/things/sensor_home/shadow/update", { state: message }.to_json)

# Listens to all accepted shadow updates
mqtt_client.get("$aws/things/+/shadow/+/accepted") do |topic, message|
    payload = JSON.decode(message)
    puts "Got #{topic}"
    puts "With #{payload}"
end

Getting Postgres version using rails console

If you are not sure what PG version your environment is using, execute this in the rails console:

ActiveRecord::Base.connection.select_value('SELECT version()')

Stubbing a block in rspec

There're at least two ways to stub a block in rspec, the first version is using and_yield

config = double('Config', enabled: true)
allow(app).to receive(:config).and_yield(config)
app.config do |config|
  expect(config.enabled).to be_truthy
end

The second version is receiving the block

config = double('Config', enabled: true)
allow(app).to receive(:config) do |_, &block|
  block.call(config)
end|
app.config do |config|
  expect(config.enabled).to be_truthy
end

How to see the full backtrace when running rspec

If you need to see the full backtrace of your tests, all you need to do is to add --backtrace to the command:

bundle exec rspec spec/model/user_spec.rb --backtrace

TODAY I LEARNED

27 posts by Edwin Cruz
@softr8