Skip to content

Caching: increasing timestamp precision #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fiedl wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Caching: increasing timestamp precision #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
41f3f81
caching: increasing the precision of the timestamps in the database t…
fiedl Apr 3, 2015
29981df
using mysql 5.6 on travis.
fiedl Apr 3, 2015
9cbe9b8
travis: removing host option from travis configuration after migratin…
fiedl Apr 4, 2015
87e0fb3
specify sock in travis db configuration.
fiedl Apr 4, 2015
0a1a30d
travis: trying to connect to database through 127.0.0.1.
fiedl Apr 4, 2015
f01d42a
working on dag link cache.
fiedl Apr 6, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,24 @@ env:
services:
- redis-server
before_install:
# INSTALL MYSQL 5.6
# (https://github.com/piwik/piwik/commit/20bd2e1c24e5d673dce3feb256204ad48c29f160)
# TODO: Remove when mysql 5.6 is provided by travis.
# Otherwise, our migrations will raise a syntax error.
- "sudo apt-get remove mysql-common mysql-server-5.5 mysql-server-core-5.5 mysql-client-5.5 mysql-client-core-5.5"
- "sudo apt-get autoremove"
- "sudo apt-get install libaio1"
- "wget -O mysql-5.6.14.deb http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.14-debian6.0-x86_64.deb/from/http://cdn.mysql.com/"
- "sudo dpkg -i mysql-5.6.14.deb"
- "sudo cp /opt/mysql/server-5.6/support-files/mysql.server /etc/init.d/mysql.server"
- "sudo ln -s /opt/mysql/server-5.6/bin/* /usr/bin/"
- "sudo sed -i'' 's/table_cache/table_open_cache/' /etc/mysql/my.cnf"
- "sudo sed -i'' 's/log_slow_queries/slow_query_log/' /etc/mysql/my.cnf"
- "sudo sed -i'' 's/basedir[^=]\\+=.*$/basedir = \\/opt\\/mysql\\/server-5.6/' /etc/mysql/my.cnf"
- "sudo /etc/init.d/mysql.server start"
- mysql --version
- mysql -e "SELECT VERSION();"
# /END MYSQL 5.6
- travis_retry gem update --system
- travis_retry gem install bundler
install:
Expand Down
1 change: 1 addition & 0 deletions app/models/active_record_associations_patches.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ def destroy(*records)
through_association.load_target
records.each do |record|
through_records_for(record).each do |through_record|
p "DELETE CACHE THROUGH HAS THROUGH ASSOCIATION."
through_record.delete_cache if through_record.respond_to? :delete_cache
end
end
Expand Down
2 changes: 1 addition & 1 deletion app/models/active_record_cache_extension.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ def bulk_delete_cached(method_name, objects)
end

def delete_cache
# p "DEBUG DELETE CACHE #{self}"
# p "DEBUG DELETE CACHE #{self.cache_key}/*"
Rails.cache.delete_matched "#{self.cache_key}/*"
end

Expand Down
8 changes: 8 additions & 0 deletions config/initializers/datetime_precision.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# In order to handle cache_keys properly, we need to store timestamps
# with µs precision.
#
# Trello: https://trello.com/c/jh6CpwdX/811-caching-zeit-auflosung
#
# Gist: https://gist.github.com/iamatypeofwalrus/d074d22a736d49459b15
#
Time::DATE_FORMATS.merge!({ db: '%Y-%m-%d %H:%M:%S.%6N' })
37 changes: 37 additions & 0 deletions db/migrate/20150403202151_increase_datetime_precision_for_timestamps.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# In order to handle cache_keys properly, we need to store timestamps
# with µs precision.
#
# Trello: https://trello.com/c/jh6CpwdX/811-caching-zeit-auflosung
#
# Gist: https://gist.github.com/iamatypeofwalrus/d074d22a736d49459b15
#
class IncreaseDatetimePrecisionForTimestamps < ActiveRecord::Migration
# Include non default date stamps here
# Key :table_name
# value [:column_names]
# NOTE: only MySQL 5.6.4 and above supports DATETIME's with more precision than a second.
TABLES_AND_COLUMNS = {
users: [:created_at, :updated_at],
groups: [:created_at, :updated_at],
dag_links: [:created_at, :updated_at],
profile_fields: [:created_at, :updated_at],
pages: [:created_at, :updated_at]
}

def up
TABLES_AND_COLUMNS.each do |table, columns|
columns.each do |column|
# MySQL supports time precision down to microseconds -- DATETIME(6)
change_column table, column, :datetime, limit: 6
end
end
end

def down
TABLES_AND_COLUMNS.each do |table, columns|
columns.each do |column|
echange_column table, column, :datetime
end
end
end
end
4 changes: 2 additions & 2 deletions demo_app/my_platform.rails4/Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ GIT
PATH
remote: ../..
specs:
your_platform (1.0.0)
your_platform (1.0.1)
acts-as-dag (>= 2.5.7)
acts_as_tree
bcrypt (>= 3.0.1)
Expand Down Expand Up @@ -227,7 +227,7 @@ GEM
yajl-ruby
font-awesome-rails (4.3.0.0)
railties (>= 3.2, < 5.0)
foreigner (1.7.3)
foreigner (1.7.4)
activerecord (>= 3.0.0)
foreman (0.78.0)
thor (~> 0.19.1)
Expand Down
4 changes: 2 additions & 2 deletions demo_app/my_platform.rails4/config/database.travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
test:
adapter: mysql2
encoding: utf8
reconnect: false
reconnect: true
database: my_platform_test
pool: 5
username: root
password:
host: localhost
host: 127.0.0.1
38 changes: 38 additions & 0 deletions ...ls4/db/migrate/20150403203339_increase_datetime_precision_for_timestamps.your_platform.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# This migration comes from your_platform (originally 20150403202151)
# In order to handle cache_keys properly, we need to store timestamps
# with µs precision.
#
# Trello: https://trello.com/c/jh6CpwdX/811-caching-zeit-auflosung
#
# Gist: https://gist.github.com/iamatypeofwalrus/d074d22a736d49459b15
#
class IncreaseDatetimePrecisionForTimestamps < ActiveRecord::Migration
# Include non default date stamps here
# Key :table_name
# value [:column_names]
# NOTE: only MySQL 5.6.4 and above supports DATETIME's with more precision than a second.
TABLES_AND_COLUMNS = {
users: [:created_at, :updated_at],
groups: [:created_at, :updated_at],
dag_links: [:created_at, :updated_at],
profile_fields: [:created_at, :updated_at],
pages: [:created_at, :updated_at]
}

def up
TABLES_AND_COLUMNS.each do |table, columns|
columns.each do |column|
# MySQL supports time precision down to microseconds -- DATETIME(6)
change_column table, column, :datetime, limit: 6
end
end
end

def down
TABLES_AND_COLUMNS.each do |table, columns|
columns.each do |column|
echange_column table, column, :datetime
end
end
end
end
27 changes: 16 additions & 11 deletions demo_app/my_platform.rails4/db/schema.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
#
# It's strongly recommended that you check this file into your version control system.

ActiveRecord::Schema.define(version: 20150325105893) do
ActiveRecord::Schema.define(version: 20150403203339) do

create_table "activities", force: true do |t|
t.integer "trackable_id"
Expand Down Expand Up @@ -62,8 +62,8 @@
t.string "descendant_type"
t.boolean "direct"
t.integer "count"
t.datetime "created_at"
t.datetime "updated_at"
t.datetime "created_at", limit: 6
t.datetime "updated_at", limit: 6
t.datetime "valid_to"
t.datetime "valid_from"
end
Expand Down Expand Up @@ -112,8 +112,8 @@

create_table "groups", force: true do |t|
t.string "name"
t.datetime "created_at"
t.datetime "updated_at"
t.datetime "created_at", limit: 6
t.datetime "updated_at", limit: 6
t.string "token"
t.string "extensive_name"
t.string "internal_token"
Expand All @@ -132,6 +132,10 @@

add_index "last_seen_activities", ["user_id"], name: "last_seen_activities_user_id_fk", using: :btree

create_table "my_structureables", force: true do |t|
t.string "name"
end

create_table "nav_nodes", force: true do |t|
t.string "url_component"
t.string "breadcrumb_item"
Expand All @@ -151,8 +155,8 @@
create_table "pages", force: true do |t|
t.string "title"
t.text "content"
t.datetime "created_at"
t.datetime "updated_at"
t.datetime "created_at", limit: 6
t.datetime "updated_at", limit: 6
t.string "redirect_to"
t.integer "author_user_id"
t.string "type"
Expand Down Expand Up @@ -181,8 +185,8 @@
t.string "label"
t.string "type"
t.text "value"
t.datetime "created_at"
t.datetime "updated_at"
t.datetime "created_at", limit: 6
t.datetime "updated_at", limit: 6
t.string "profileable_type"
t.integer "parent_id"
end
Expand Down Expand Up @@ -245,8 +249,8 @@
t.string "alias"
t.string "first_name"
t.string "last_name"
t.datetime "created_at"
t.datetime "updated_at"
t.datetime "created_at", limit: 6
t.datetime "updated_at", limit: 6
t.boolean "female"
t.string "accepted_terms"
t.datetime "accepted_terms_at"
Expand Down Expand Up @@ -286,6 +290,7 @@
t.datetime "updated_at"
end

Foreigner.load
add_foreign_key "attachments", "users", name: "attachments_author_user_id_fk", column: "author_user_id"

add_foreign_key "bookmarks", "users", name: "bookmarks_user_id_fk"
Expand Down
14 changes: 14 additions & 0 deletions spec/models/active_record_cache_extension_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,20 @@
@user = create(:user)
end

describe "#cache_key" do
subject { @user.cache_key }
it "should be the same before and after a reload from the database" do
cache_key_before_reload = @user.cache_key
@user.reload
subject.should == cache_key_before_reload
end
it "should allow to retrieve the cached value after reload from database" do
Rails.cache.write [@user, :foobar], "Value written before reload"
@user.reload
Rails.cache.read([@user, :foobar]).should == "Value written before reload"
end
end

describe "#cached" do
subject { @user.cached(:title) }

Expand Down
56 changes: 55 additions & 1 deletion spec/models/dag_link_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,59 @@
require 'spec_helper'

describe DagLink do
# Changes on dag links are reflected on attributes of dag nodes (e.g. Users, Groups, etc.).
# Therefore, it is important to invalidate or renew the cache of these objects.
#
describe "(Cache Callbacks)" do
before do

class User
def cached_group_names
cached { self.groups.pluck(:name) }
end
end
class Group
def cached_member_names
cached { self.members.collect(&:name) }
end
end

@user = create :user
@group = create :group
@membership = @group.assign_user @user

@user.cached_group_names
@group.cached_member_names
end

describe "after_destroy" do
describe "through the parent object" do
subject { @group.members.destroy(@user); time_travel(2.seconds); @user.reload; @group.reload }
it "should delete the cache of the associated child object" do
subject
Rails.cache.read([@user, 'cached_group_names']).present?.should be_false
end
it "should delete the cache of the associated parent object" do
subject
Rails.cache.read([@group, 'cached_member_names']).present?.should be_false
end
end
describe "through the child object" do
subject { @user.groups.destroy(@group); time_travel(2.seconds); @user.reload; @group.reload }
it "should delete the cache of the associated child object" do
subject
Rails.cache.read([@user, 'cached_group_names']).present?.should be_false
end
it "should delete the cache of the associated parent object" do
subject
Rails.cache.read([@group, 'cached_member_names']).present?.should be_false
end
end

end
end
end

# The dag link functionality is tested extensively in the corresponding `acts-as-dag` gem.
# This test is just to make sure that the integration is propery done. Therefore, some basic scenarios are tested here.
#
Expand Down Expand Up @@ -45,4 +99,4 @@ def setup_pages
end
end

end
end