My RubyFringe highlights

20 July, 2008

My new home town (not just the place I live, but now HOMEtown) rocks. The unspace crew did a freaking amazing job on RubyFringe. Joey has a huge list of notes on his blog if you want some nitty gritty. For me personally, here's the stand outs... - I have to say Damien Katz's presentation was extremely raw and powerful. The story of how CouchDB came to be will stick around in a lot of peoples minds for a very long time I'm sure. - Giles Bowkit's talk was just an assault on the senses. Wow. Almost want to go to Burning Man now. Looking over the shoulder of my cohort's laptop on the backchanel it seemed dead quiet in the 5 secs I peeled my eyes off stage. - Nick Sieger's Jazz presentation taught me more in 30 mins than any music class ever has (not that I attended many) did in school. Even though I love music. Love it to bits. But he broke it down so well I was stunned. Wow. Thank you. - Mr Grigsby. Resourceful. What more can I say. This dude plays the system hard. - Leila had a hard job having the last talk of the day, but her essence was on the money, do something people love (and therefore value) and you'll do well. If your customers love the stuff you build. You've got it made. She showed an extremely hard core passion for her company. Fantastic. - Less than 200 people seems a perfect number for networking. Not large enough that you're anonymous, not small enough so you're talking to the same people. Met so many great people. Hope to stay in touch. - Libin, my man, you should be THE official Canon product tester. I believe he was trying to hit 1k photos. at time of posting 413 odd photos were up on Flickr but he hasn't uploaded Sunday's yet... Surprisingly technology was less of a focus of most everyone's presentations. Instead of the usual tech stuff every day on blogs, to mix things up the talks really got into the hearts, minds, and souls of the presenters. All of which just can't be conveyed in the web medium. The tech presentations that were given were solid as well, it felt like just the right mix between screeds of cool innovative code and deep talks. It's amazing what can be achieved in just 30mins !

Read More

Toronto Rails Night - Merb Presentation

11 June, 2008

Last night was yet another successful Toronto Ruby on Rails Nite put on by Corina Newby. The Toronto ruby community is rocking, evidenced by the large turnouts and fresh faces we see everytime. I'd guess 40-50 of Toronto's best and brightest Rails developers filled out the room. The unspace guys put on some much appreciated beer and made sure we're all informed about the upcoming RubyFringe (can't wait, I'm really hoping Zed's presentation is as exciting as his blog rants posts). Along with the other presenters of the night Mike and Libin/Wes/Carsten (up on TorontoRB.ca sometime in the next day or so). I did a presentation on Merb - The pocket rocket framework. Slides are are here (you may have to click through to the blog post to view if you're reading this in a reader/aggregator). Resource links are at the end of this post, and some discussion notes follow.. Presentation background Intended for people who are interested in looking at Merb, collating key points from blog posts, irc dicussions, mailing lists, and own experiences getting an app running with Merb (well the opposite way round, trying to get an app running and finding out those answers!). Hopefully the presentation collected a fair few hours worth of investigation into 20 or so minutes to help other people out Sports Bikes vs Sports Cars Useful analogy between Sports Bikes (Merb) and Sports Cars (Rails). Said cars have all the bells and whistles, everything is packaged up nicely, and it's pretty hard to hurt yourself, however given the nice packaging it's pretty hard to modify things. Sports bikes (Merb) on the other hand, are much more open to tuning, playing with, go faster, but need a bit more care Same same but different Rails and Merb apps are very much of the same likeness, but there are differences. Don't expect to be able to port code over 1:1 (especially if you've got a big reliance on plugins), and don't pepper the merb mailing lists/irc channel for "please make xyx behave like it is in rails merb-core & merb-more Two gems you need to get started (technically you only need merb-core). -Core is the stripped down bare minimum to get merb running. -more contains all the nice packaged goods to help you along the way merb-gen is your friend throw out script/* you're now doing everything in merb gen for generating your app, models, resources etc Key code differences refer to slides, but the code used is difference Agnostic in a very good way Merb is agnostic when it comes to ORM, test framework, js framework, rack adapator. merb-gen behaviour changes based on component used so if you're using rpsec, then stories get created vs unit tests. This is one of the reasons I'm very much into the idea of Merb Cool features part controllers, action args, router, covered off in the slides. Full apps, flat apps, very flat apps and thin servers and large grains of salt For performance stats (which are to be taken with a very large grain of salt). Hello world action on a single controller, rails would be about 300 req/s full merb app would be round 500 req/s flat merb app 500-550 req/s, a very flat app marginally quicker again, front a very flat app with thin (merb -I veryflatapp.rb -p 3000 -e production -a thin) and woah hold on, 1000 req/s ! Question was asked about benchmark figures and I made the point, they're going to be all over the place - it's easy enough to benchmark for yourself, that's the only thing that's really relevant. We know Merb is quick, just how quick will depend on your own application and uses. Merb, Rails and you The inevitable discussion at the end around, should you use Merb and how does it relate to Rails. Some of the goodness in Merb looks to be slowly being backported to Rails, so over time Rails will benefit from all of the wizardry going on in Merb (hopefully!). For those that like Merb's approach, and willing to forsake the vast library of rails plugins, and/or have very specific performance requirements and/or need to hack a framework around to do what they want. Merb looks to be killer. Ultimately it's upto you, so evaluate carefully. Resources Lots of link goodness..... http://merbivore.com http://wiki.merbivore.com http://merbunity.com http://merborials.com/ http://groups.google.com/group/merb http://github.com/search?q=merb http://mwrc2008.confreaks.com/ http://www.slideshare.net/search/slideshow?q=merb http://www.slideshare.net/search/slideshow?q=datamapper

Read More

Toronto Ruby / Rails madness

05 May, 2008

If you're in Toronto and been living under a rock you probably haven't realised that every month, there are a number of events specifically for Ruby/Rails developers. There's the ever popular and well attended Rails Pub Nite - great for networking and an excuse to have a pint or two. If this sounds like your thing join the other 40 odd that head out on the third Monday of the month at the Rhino. Pete Forde @ Unspace is the man on the scene for that one, kudos for keeping that for over a 2 years now (I think, don't quote me!!). The newish kid on the block the Rails Project Nite. Project nite consists of 3 presentations every time showcasing projects or rails specific learnings, usually showing nitty gritty code on some fantastic projects so quite enjoyable for us nerds! There's been some great presentations, so make sure if you've got any interest get down there, again solid numbers show up. Corina Newby has been putting this together, major props there for getting the new event up and running. If you have an interesting project that you would like to present, just contact her and volunteer your name, and breif summary. Make sure you RSVP on the Project nite, if not you could miss out Finally there's the Toronto Ruby users group, meeting on second sunday of every month at the Linux cafe - I will fess up that I haven't been to this one, weekends being usually DIY disaster time it's harder for me to make that one :) So if you can't find something in between those 3 that spark an interest in Ruby related development learnings, well then... there's no hope for you More details on the Ruby Users Group, Sunday May 11th - http://www.trug.ca/ More details this months Project Nite, Tuesday May 13th- http://correlations.wordpress.com/2008/05/04/tuesday-may-13th-ruby-on-rails-project-night/ More details on the Rails Pub Nite, Monday May 19th - http://www.unspace.ca/innovation/pubnite/ Finally, what mention of the Toronto Ruby/Rails scene wouldn't be complete without a Rubyfringe plug, haven't got tickets, get that sorted out ! - http://rubyfringe.com See ya at one of the above ! [Post script] Not entirely Ruby/Rails related I would be shot if I didn't also mention the Toronto Software Developers Lunch or the Ajax Pub Nite.

Read More

require, RMagick, and case sensitivity

19 December, 2007

(At least on Leopard with Ruby 1.8.6) Ruby will try to reload rmagick if you get the case of rmagick different from the original require. And when that happens it all goes up the whop. We were using attachment_fu's image magick processor, then doing a require 'rmagick' seperately which resulted in tonnes of errors like this

/usr/local/lib/ruby/gems/1.8/gems/rmagick-1.15.10/lib/rmagick.rb:32:
  warning: already initialized constant PercentGeometry

Keep your case's of require's consistent. Even if this is file system dependent behaviour in mixed development environments you can imagine how weird problems could get!

Read More

The satisfaction of Rails

13 August, 2007

More work on XChain. In some spare hours on the weekend I implemented the basic structure of price catalogues for products. I can't tell you how long this took when previously doing it in PHP a few years back! (or even worse ASP a few years further back still). I'm sold on BDD, it was so quick to bang this out starting with the spec it first approach, mainly because all you need to do is 'rake spec', and bingo your code is tested... As follows, note code is cut down a little, but you get the picture:

# A product itself does not contain any price information, it has many price's in a 'pricing' model, that is linked
# to price types, eg "Standard Distributor" "Standard Retail"
class Product < ActiveRecord::Base
  has_many :pricings
  has_many :price_types, :through => :pricings
  belongs_to :order_line
end

#The pricing table, not only links to a product and price_type, it also has a discount
#model associated with it..
class Pricing < ActiveRecord::Base
  set_table_name 'pricing'
  belongs_to :product
  belongs_to :price_type
  belongs_to :discount
end

# discount linked to the pricing, and discount tables to enable volume based discounting.
class Discount < ActiveRecord::Base
  has_many :pricing
  has_many :discount_tables, :order => "qty ASC"
  #note that I don't want to do queries every time I ask get_discount, so I just iterate over the
  # tables to find the discount, rather than doing a find each time.
  def get_discount(qty)
    discount_tables.each do |dt|
      return dt.discount_applied if qty <= dt.qty
    end
    return discount_tables.last.discount_applied
  end
end


# In order to calculate price, we need to determine the pricing model for the order (taken from price type)
# then look up the unit price for the product according to the price type
# finally get the discount for the volume at that price type.
class OrderLine < ActiveRecord::Base
  belongs_to :order
  belongs_to :product
  def get_price(price_type)
    @pricing = product.get_pricing(price_type)
    price_as_ordered = ( get_unit_price * qty_ordered ) * get_discount
  end

  protected
  def get_unit_price
    @pricing.get_unit_price()
  end

  def get_discount
    @pricing.discount.get_discount(self.qty_ordered)
  end

You feel like you accomplished something when you sit down to work for a few hours and you get this...

=====================
rowans-computer:~/work/xchain rowan$ rake spec;
(in /Users/rowan/work/xchain)

DiscountTable
- should be valid

Discount when loading existing discount
- should return a max discount of 0.25 for a quantity of 10000
- should return a discount of 1 for a quantity of 20
- should return a discount of 1.5 for a quantity of 5
- should respond to get discount
- should have a discount table
- should have a name
- should belong to a pricing

Order when finding existing order
- should have a customer
- should have a billing address
- should have 3 order lines
- should have a purchase order number
- should have id = 1

Order with a single, invalid order line added
- should after changing the quantity be able to be saved
- should not be able to be saved as it has incorrect quantity
- should calculate to a correct amount

Order new draft order
- should be in the order collection list
- should belong to the correct customer
- should have an associated user who created it
- should be able to have it's address changed
- should be able to be changed to a new status

Order when creating a new order
- should when valid and saved, have one order status history item
- should be able to have an order line added
- should throw an error that it doesn't have line items
- should calculate return 0 dollars
- should have a default price type matching customer price type
- should be able to prefill an address
- should default to draft status
- should be able to instantiate

OrderStatus using a draft order status a super user
- should have a list of all available statuses

OrderStatus using a draft order status with no super user
- should not have all statuses available
- should have a list of next available status, when flow next is false, only including itself and the next higher status

OrderStatus
- should be able to find standard statuses

Product when using Product ID 1
- should be valid

OrderLine with valid order line
- should return correct discount levels (NOT IMPLEMENTED)
- should after changing qty return a different price (NOT IMPLEMENTED)
- should return a price of for x price type and y qty with a discountable price type
- should return a price of for x price type and y qty with a non-discountable price type
- should respond to get_price
- should be valid

Finished in 1.72758 seconds

40 examples, 0 failures, 2 not implemented

Read More

Syncing up external Filemaker data with Rails

28 May, 2007

One of the web apps I work on leverages a Filemaker database. My co worker Brent the expert Filemaker dude, and I had a serious problem to tackle - how to get Rails and Filemaker talking to each other. Now Filemaker has iffy SQL standard support and we didn't necessarily want our web app to run live from the Filemaker database. We spent some time looking at making an ActiveRecord adapter for FM but really at the end of the day it was too much like hard work. There are 3rd party solutions out there that can do this stuff for you, but if your needs dictate that they won't work - our experience may help you. I'm going to write up the theory behind this which has been live in production for a good 6 months + and in the coming weeks will release some code acts_as_syncable or similar... First off we wanted it occassionaly connected, one way synchronising. We had an installed instance of Filemaker Web Publishing engine so we could expose our data as xml views. We used a background rails (backgroundrb) process to query Filemaker every 5-10mins and ask for changed data, pull that data using REXML, and populate/update on the Rails side using ActiveRecord. Here's the mechanics... 1. Each object, needs a last_modified timestamp, whenever an update to the object that affects the data being synced, this last_modified timestamp needs to be updated (for all intents and purposes create an updated_at column on the filemaker side). 2. Next on the Filemaker side create a layout that exposes that data when queried via Web Publishing, as an xml result. We're going to do two different queries so your layout must support these - as per note ii. 3. On the Rails side we need to add two models, and extend the models we want synced. First I created a SyncLog model which will track when syncing occurs (just for informational purposes), then a ClassSyncLog, which will track the models synced, when they were synced, the last timestamp and id of each model synced. 4. For each model on the rails app side, we need a number of properties: - The corresponding id field name from Filemaker - A hash listing each rails field and it's corresponding name on the Filemaker side (we're assuming that these will be different, in our case they were wildly different) - Some call backs, pre process sync, and post process sync if you need to do stuff before/after syncing. 5. We need a model that will actually do the work of syncing the objects. This is the involved part of the process that does the grunt work of syncing so pay careful attention. 5.1 Pass in a list of Class names, for each class name do the following 5.2 Query filemaker using Net::HTTP from the last updated_at timestamp and see if any records have been updated since then, if so move on to 5.3 or go to the next class (well... almost there's a trick here **) 5.3 For that returned result set from Filemaker (from the HTTP Raw data returned), fill an REXML object from the returned xml file. For each of the records in the result set do the following 5.3.1 Find or create the rails side object from the corresponding id in the resultset, check if you can sync it per the pre process rules if you have any. 5.3.2 Iterate over each of the fields in the result set, see if it matches a field in the field mapping on the rails class you're syncing (4). If it does update the current object from the result set. 5.3.3 Finally save the current object and move onto the next record 5.4 Now before moving onto the next class you need to save a ClassSyncLog record, with the last timestamp of the last updated record, and the last id. So next time you have a starting point to retrieve the records. 5.5 Move onto the next class 5.6 After completing all classes write out a log entry to say it's been done. 6. Wait 5mins... and repeat. Of course there's some caveats. i. First is if you're dealing with 100'000's of records you do not want to grab just one resultset back - you're going to run out of memory and/or have problems with getting the full stream of data. So we get the result set query into groups of ~ 2000. ii. Related to this, lets say you have 100'000 records updated at 10:10:01 AM. The first request is going to be for all records greater than the last updated time, for example 9:30AM. So you'd get 2000 in on your 100'000 records, the next time in you ask for > 10:10:01 am, you're going to skip 98'000 records, so you want to do two queries first: a. Give me all records time = last_updated_time AND id > last_updated.id, which will catch the next 2000 records - repeat this one until you get no more results then: b. Give me all records time > last_updated_time - then go back to qry #a for the next repeat. iii. You will have performance issues, this isn't quick by any stretch of the imagination so be a little patient to work through the issues. We're running massive datasets so we often see updates that take hours. Not great but it's ok. iv. Watch your memory, this can be a long running process so throw in 'GC.start' to clean up memory along the way. (eg every resultset retrieved) v. To avoid ID collissions it's strongly recommended you use the MySQL master-master auto-increment-increment + offset scheme, so you stagger your ID inserts between your Filemaker DB(s) and MySQL DB(s). Again look for code in the coming weeks.. I'm wrapping our custom stuff up into an acts_as module for the (limited) audience this may help...

Read More