FacultyAWS

Usage

Each component of the gem needs to be required separately, and the appropriate AWS gem will need to be added to your Gemfile too.

See each section for details of requirements.

Notify Devs

gem 'aws-sdk-sns', '~> 1'
gem 'uoy-faculty_aws', '>= 0.9.0', require: 'faculty_aws/notify_devs'

Helper functions for notifying the devs of errors.

DB Connector

gem 'aws-sdk-rds', '~> 1'
gem 'sequel', '~> 5'
gem 'uoy-faculty_aws', '>= 0.9.0', require: 'faculty_aws/db_connector'

A class for connecting to a database using appropriate credentials.

Example use in sinatra apps:

configure do
  # Connect to database
  settings.db ||= {}
  settings.db[:noauto] = true unless in_lambda?
  DB = FacultyAWS::DBConnector.new(**settings.db).connection
end

DBConnector can pick up most configuration from the environment. It massages certain inputs before supplying them to Sequel - for example, for SQL logging you can specify DB_LOGGER='-' for stdout, or a filename.

BackgroundTask

gem 'aws-sdk-sqs', '~> 1'
gem 'uoy-faculty_aws', '>= 0.9.0', require: 'faculty_aws/background_task'

This module handles background tasks by sending them via an SQS FIFO queue.

Set ENV['BACKGROUND_QUEUE'] to the queue URL. In development mode, set ENV['BACKGROUND_INLINE'] (to yes, true or 1) - this bypasses the queue.

There is a lambda handler which you can set as the lambda handler in cloudformation:

  BackgroundTaskFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: <filename>.FacultyAWS::BackgroundTask::LambdaHandler.background_task_handler

You'll need to set up a file that loads all of your required libraries and code, and filename should refer to this file. For sinatra apps, this should not be part of lambda.rb as this loads rack and sinatra, and you probably don't want those in the background tasks; see FlexiLeave for an example of how to do this. For other apps (e.g. mailing list generator), this can go as part of the existing lambda.rb.

To register a task in your app:

# Register a task handler
FacultyAWS::BackgroundTask.new :my_task do |required:, optional: 'world'|
  puts "#{required}, #{optional}"
end

By default, exceptions raised in the handler will cause the lambda to exit with an exception, and notify the developers. These behaviours can be disabled individually:

FacultyAWS::BackgroundTask.new :my_task, notify_on_failure: false do ...
FacultyAWS::BackgroundTask.new :my_task, raise_on_failure: false do ...

To queue the task to run in the background:

# Queue an event
FacultyAWS::BackgroundTask.enqueue(:simple_task_with_no_params)
FacultyAWS::BackgroundTask.enqueue(:my_task, required: 'Hello')
FacultyAWS::BackgroundTask.enqueue(:my_task, required: 'Goodbye', optional: 'everyone')

You can optionally specify the message group ID and/or deduplication ID. The group ID defaults to the task name (one "lane" per task), and the deduplication ID is a hash of the message body by default.

FacultyAWS::BackgroundTask.enqueue(:foo, param: 'arg', message_group_id: 'group 2')
FacultyAWS::BackgroundTask.enqueue(:foo, message_deduplication_id: '9a8b7c')

You can use the current timestamp for deduplication; this also uses the message group ID to ensure messages from different groups cannot clash:

FacultyAWS::BackgroundTask.enqueue(:foo, timestamp_deduplication: true)

To run background tasks out of cron, set EventBridge to submit a JSON payload to SQS. It should be of the following form (params can be omitted):

{
  "task": "my_task",
  "params": {
    "named_parameter": "argument"
  }
}

Sending mail through SES

gem 'aws-sdk-ses', '~> 1'
gem 'mail', '~> 2'
gem 'uoy-faculty_aws', '>= 0.9.0', require: 'faculty_aws/mail'

FacultyAWS::Mail is a wrapper for sending email through SES. Emails can be written using the mail gem and passed to the helper.

Because the source ARN and from address will only differ by environment, we can set these globally when initialising the wrapper.

The wrapper will log the email when development is set rather than trying to connect to SES from your local machine.

An example of using this in a Sinatra app:

configure do
  MAIL = FacultyAWS::Mail.new(source_arn: ENV['EMAIL_SOURCE_ARN'], from: ENV['EMAIL_FROM'], development: development?)
end

#...

mail = Mail.new do
  # mail content here
end
MAIL.send(mail)

SMTP?

We have to use the SES API because we are sending email to an ARN that is part of another account. AWS does support SMTP for SES, but it's only available for the current account, so this doesn't work for us.

Development

Versioning

Your Gem's version is picked up automatically from lib/faculty_aws.rb. When any changes are pushed to master, after the normal CI tasks the pipeline will push to gemfury automatically. The usual workflow is:

• For minor changes, update VERSION and make the change in a single commit

• For anything else, create a branch and set VERSION to the version you're aiming to release for. Make the changes; when the branch is merged, the gem will be uploaded.

Note that gemfury will never overwrite an existing gem version, even if the old one is yanked!

Running tests

Tests can be run via rake: bundle exec rake spec - this doesn't run performance tests; they can be run separately via bundle exec rake perf.

You can also run rspec normally e.g. bundle exec rspec -fd.

Identifying Performance Tests

If you have performance tests that take a while, tag the context / describe block like this:

context 'when foo is bar baz', :perf do
  ...
end

Contributing

Bug reports and pull requests are welcome at https://github.com/university-of-york/faculty-dev-faculty_aws-gem