Toronto Ruby on Rails nite - not what you expect

15 January, 2009

So last night was the first Rails project night of the year and you know what ? I don't think I talked to one person about Rails itself other than the cursory "what do you do?" type conversations. What *was* interesting is I had no less than 4 different conversations with people about their professional development, be it their ideas or their careers - challenges they're facing, how to tackle things etc. It was kinda amazing, and made me realise just how important these project/pub/*camp evenings are. Being in a startup, or freelancer you don't have access to people to bounce ideas off, nor have access to professional development or HR teams. Last night really hammered this home. So if you're a team leader, a seasoned development vet, have built a business or two, etc and have a passing interest in Rails, you could do much worse than head out to one of the RoR related events, or any tech events, in Toronto. Ask someone what they do and you never know what kind of conversation you might have..

Read More

Tip of the day - APP_ROOT/config/httpd.conf

08 August, 2008

If there's one thing that's saved me no end of heartache it's including Apache conf settings for a Rails app or Merb/PHP/whatever, in the app's config folder. You get versioning for free. You don't forget your settings. You know where to find it every time. No other process can mess with it (not looking at anyone in particular CPanel...). Just make sure your web app can't mess with it :) i.e. create a file /var/rails/myrails/app/current/config/httpd.conf in which you place your host pointing to your rails app. Then in your apache conf file (normally /etc/httpd/conf/httpd.conf) ensure you include the file above. On my main app config file I try to pop in the top of the file the passenger config settings, as again it's easy to find, then any of the required vhost(s) that the app uses. Nice clean and in one place.

Read More

RubyFringe a new brand of awesomeness

20 July, 2008

Opening night and day 1 are over. Just about to go pick Krispy and head down for day two. Truly, my mate mates down at Unspace have done an AWESOME job. The vibe is amazing. A lot of "best conference ever" comments going round. Yes.. those are laptops waving in the air, and yes that is Zed on stage. Just look out for the mp3..

Read More

My RubyFringe highlights

20 July, 2008

My new home town (not just the place I live, but now HOMEtown) rocks. The unspace crew did a freaking amazing job on RubyFringe. Joey has a huge list of notes on his blog if you want some nitty gritty. For me personally, here's the stand outs... - I have to say Damien Katz's presentation was extremely raw and powerful. The story of how CouchDB came to be will stick around in a lot of peoples minds for a very long time I'm sure. - Giles Bowkit's talk was just an assault on the senses. Wow. Almost want to go to Burning Man now. Looking over the shoulder of my cohort's laptop on the backchanel it seemed dead quiet in the 5 secs I peeled my eyes off stage. - Nick Sieger's Jazz presentation taught me more in 30 mins than any music class ever has (not that I attended many) did in school. Even though I love music. Love it to bits. But he broke it down so well I was stunned. Wow. Thank you. - Mr Grigsby. Resourceful. What more can I say. This dude plays the system hard. - Leila had a hard job having the last talk of the day, but her essence was on the money, do something people love (and therefore value) and you'll do well. If your customers love the stuff you build. You've got it made. She showed an extremely hard core passion for her company. Fantastic. - Less than 200 people seems a perfect number for networking. Not large enough that you're anonymous, not small enough so you're talking to the same people. Met so many great people. Hope to stay in touch. - Libin, my man, you should be THE official Canon product tester. I believe he was trying to hit 1k photos. at time of posting 413 odd photos were up on Flickr but he hasn't uploaded Sunday's yet... Surprisingly technology was less of a focus of most everyone's presentations. Instead of the usual tech stuff every day on blogs, to mix things up the talks really got into the hearts, minds, and souls of the presenters. All of which just can't be conveyed in the web medium. The tech presentations that were given were solid as well, it felt like just the right mix between screeds of cool innovative code and deep talks. It's amazing what can be achieved in just 30mins !

Read More

If there's one piece of advice I could give ..

16 July, 2008

So I've been thinking about things, thinking about the state of web development. What could I write about to could help out some new recruits and budding developers. (Note this is a long one, so get some coffee, sit back, read, and I'd love to see your thoughts and opinions at the end ...). In a nutshell: I have a rather entrepreneurial friend back in New Zealand, he consulted at a Software company I worked for once. Over time we became good friends.. One day him and his business partner came up with a widget. And sold a few, and then needed to sell some more. Then one day he gave me a call. You can imagine as a web developer the conversation "Yeah some some company wants [insert horrendous figure here] to build us an e-commerce site" pffwaaaattt (coffee all over desk)... And so began a new relationship. Realising the scope of work was BIG, I actually called up one of my dev mates. "Oi, I've got some work for us, keen?" Gave him the specifics. We agreed to do it 50/50 (side note, later on I ended up taking it on 100%). We struck a deal. I drew up a contract and we got underway. It's one thing sitting at a desk job, saying something will work, and then meh, it doesn't, this happened or that happened, whatever the reason, your thing doesn't work. There's a kazillion reasons why something won't work, or can't be done, or is late etc. Depending on your character this will bother you, or not (hopefully the former!). It doesn't *really* teach you responsibility. Now, taking someones money personally, and promising delivery of something, with nobody else to shoulder the workload. Now that's responsibility. There's only you and your Mac/PC. So long as the two of you are both in working condition, and you haven't bitten off more than you can chew well there's no excuses. Everyone reading this should be familiar with the actual mechanics of building something, but most often you're shielded from the business elements. Why are you building what you're building ? Is some Business Analyst telling you chinese whispers. Do you really know ? It's all about the vision. This isn't some contrived management tactic, there's a reason people put it on the top of briefing/requirements docs, if you buy into the ultimate business goal, then you will be far better at doing what you need to do. There is a famous saying in New Zealand - "Make the boat go faster" an entire army of engineers, yachtsman, craftsman, tacticians were all about one single minded goal - making the boat go faster (and they did, Black Magic won the Americas Cup). In our case it was all about selling the widgets. By being both a friend, and supplier, you realise you're taking money from this company, your friend, to build something. They're going to have to make that money up by selling product. So, you'd better do what you can to help them sell that product. By keeping the vision at the forefront of your mind you're not going to use technology for technology's sake. You're not going to waste time spending 10 hours getting some little bit working when it doesn't matter. You are going to learn how to prioritise - especially if you're doing this in your hometime. Once you've wasted 3 evenings in a row and not gotten any further along the path to completing your objective you'll quickly learn what's important! This WILL make you a better developer. Too often I read posts about the top x number of things looking for when hiring a developer. Technical brilliance is IMNHO focused on WAY too much. By doing the process from start to finish, you'll realise, the minor technical bits are quite small in the overall scheme of things (unless of course you need to build some ninja fast product requiring complex calculations) Note as I've been writing this Ilya put up a great post mentioning the jelly jar, another way of looking at this issue You will hear and see a lot people who are singularly focused on one technology, one piece of the puzzle, they're specialists. That's great. But a piece of code, or a sql call, or a config file does not a web app make. If you have to do everything from ordering a server to getting the finished app up and ready to login, you'll realise there are a lot of pieces of the puzzle. By doing everything yourself you'll begin to realise the other side of the fence is not as nearly black magic as you thought it was. Gaining experience in all aspects, from server config, mysql setup, dns setup, playing with cron will make you a lot more well rounded. By becoming more experienced in different aspects, you'll realise you can do a lot more. [As a side note, this is also why I think working in a startup, is a damned good thing. Every dev should do it at least once!] Along the way, your client is going to ask you for things, lots of things. And you're going to have to manage that feature request cycle. Now who knows better - you or the customer ? The just say no philosophy that is becoming pervasive in web app development school of thought is scaring me. My client loves it when stuff gets implemented. I don't just say no every request. Where stuff doesn't make sense, or can be done better, I talk through it with them. But if I just said no everytime. Well the relationship not likely go far. Don't get me wrong, I see it's place, but you have to use judgement, and know where it applies. Don't take it literally. When you're close to the customer, you'll realise their daily pain. Walk through the system with them, understand every support request - what lead to it, why, what can you do better ? Now, why did I say in my opening statement maintaining an app. Long term there is no greater thing than maintaining what you wrote. In our day jobs, we move on, client's move on, the code moves on. But invariably the business doesn't - it stays there and so does the site. By maintaining something yourself you get an appreciation for the movement in technology(when we first wrote this app, the nearest thing to a published framework were the PEAR libraries, nobody had yet come up with a solid stack). Now 3-4 years later I'm rolling in Rails into the mix. You'll understand the value of commenting the tricky bits. it always comes up as a reminder, just last week I was all over some code questioning "WTF did I do here?" especially when the business decision hadn't been written down beside the code. Along with gaining a much fuller appreciation of clean, modular, readable and reusable code. Also you'll get the chance to refactor and it makes you (well at least it does for me) feel good. Some crufty bit of code you haven't touched for 6 months, and you get to make it look better and work better. Nice ! This might seem counter to what I've been saying. But I think this is how a lot of new technologies get adopted. People are playing on pet projects and attempt to use a new technology. Doing your own app, you get to make the choice. It's either going to work or it's not. And you have to live with the win's or losses. Whereas in your day job there might not be anyway on gods green earth you're going to implement technology x as the impact on a large team is huge, but on your own time - it's your choice. You get to play with jQuery for instance (whichs rock extremely hard by the way), or mix and match Rails and PHP. Can you imagine what a system architect is going to say when you propose that in your day job? No f-cking way. Of course, you have to pick your moments and rationalise why you're choosing that technology. What is it really doing better for you... This is where doing a project for yourself shines. I saw a great clip by Tony Robbins the other day, must say I've never seen him before, and not likely to buy his products, however he did say something that that resonated with me loudly - too many people blame the lack of resources time,money etc but ultimately their failures are not about lack of resources, but the lack of THEM being RESOURCEFUL. This is what I'm talking about with this post. Maybe it's a really long winded way of putting it, but by building and maintaining your own app, in order to do it successfully you'll have to be resourceful. You'll have to learn, prioritise, be accountable, responsible, problem solve, etc and it will all matter because.. You'll get this at the end of the day. That's what I appreciate in developers. People who when faced with a challenge won't back down. They will find different approaches to a problem. They will try and try to get something running, they'll go away, rethink the problem and have another crack at it. Once done, they'll polish it, make it look work better/faster/ etc to acheive the end goal/vision. Having done that they'll take pride in it and say "I did that, I think it's cool and it rocks, and here's why ... ". Going back to my original opening statement, doing a project yourself gives you the opportunity to do so, virtually unrestrained. You'll get a tonne of challenges from technical, emotional, financial, time, the whole gamut. But by taking on the responsibility to do it yourself, (or be responsible for delivery yourself - you could farm the whole lot out, then you've got a whole new set of problems!), it means you have to meet and break through each one of those challenges. The only constraints in general, are your time and motivation. A number of years down the track you can look back on it, and go 'damn that feels good' When I get the chance to interview people one of my questions is always Feel free to answer it here, I'd like to see what makes other people tick..

Read More

See Ruby/Rails does scale

24 June, 2008

Updated LinkedIn Blog Post If yellowpages.com or a whole bunch of other sites or the Friends for Sale facebook app wasn't enough to convince you, that yes, you are on the right track with Rails (no pun intended). Another solid Rails app has been mentioned in mainstream media http://blogs.zdnet.com/enterprisealley/?p=188 blogs about a Facebook app done by the LinkedIn Light Engineering Development team. Cranking out mac daddy throughput rate - 1 billion page views / month. Enough said. Go read the article and watch the (excellent) video. Then get back to the business of building hot Rails apps.

Read More

Toronto Rails Night - Merb Presentation

11 June, 2008

Last night was yet another successful Toronto Ruby on Rails Nite put on by Corina Newby. The Toronto ruby community is rocking, evidenced by the large turnouts and fresh faces we see everytime. I'd guess 40-50 of Toronto's best and brightest Rails developers filled out the room. The unspace guys put on some much appreciated beer and made sure we're all informed about the upcoming RubyFringe (can't wait, I'm really hoping Zed's presentation is as exciting as his blog rants posts). Along with the other presenters of the night Mike and Libin/Wes/Carsten (up on TorontoRB.ca sometime in the next day or so). I did a presentation on Merb - The pocket rocket framework. Slides are are here (you may have to click through to the blog post to view if you're reading this in a reader/aggregator). Resource links are at the end of this post, and some discussion notes follow.. Presentation background Intended for people who are interested in looking at Merb, collating key points from blog posts, irc dicussions, mailing lists, and own experiences getting an app running with Merb (well the opposite way round, trying to get an app running and finding out those answers!). Hopefully the presentation collected a fair few hours worth of investigation into 20 or so minutes to help other people out Sports Bikes vs Sports Cars Useful analogy between Sports Bikes (Merb) and Sports Cars (Rails). Said cars have all the bells and whistles, everything is packaged up nicely, and it's pretty hard to hurt yourself, however given the nice packaging it's pretty hard to modify things. Sports bikes (Merb) on the other hand, are much more open to tuning, playing with, go faster, but need a bit more care Same same but different Rails and Merb apps are very much of the same likeness, but there are differences. Don't expect to be able to port code over 1:1 (especially if you've got a big reliance on plugins), and don't pepper the merb mailing lists/irc channel for "please make xyx behave like it is in rails merb-core & merb-more Two gems you need to get started (technically you only need merb-core). -Core is the stripped down bare minimum to get merb running. -more contains all the nice packaged goods to help you along the way merb-gen is your friend throw out script/* you're now doing everything in merb gen for generating your app, models, resources etc Key code differences refer to slides, but the code used is difference Agnostic in a very good way Merb is agnostic when it comes to ORM, test framework, js framework, rack adapator. merb-gen behaviour changes based on component used so if you're using rpsec, then stories get created vs unit tests. This is one of the reasons I'm very much into the idea of Merb Cool features part controllers, action args, router, covered off in the slides. Full apps, flat apps, very flat apps and thin servers and large grains of salt For performance stats (which are to be taken with a very large grain of salt). Hello world action on a single controller, rails would be about 300 req/s full merb app would be round 500 req/s flat merb app 500-550 req/s, a very flat app marginally quicker again, front a very flat app with thin (merb -I veryflatapp.rb -p 3000 -e production -a thin) and woah hold on, 1000 req/s ! Question was asked about benchmark figures and I made the point, they're going to be all over the place - it's easy enough to benchmark for yourself, that's the only thing that's really relevant. We know Merb is quick, just how quick will depend on your own application and uses. Merb, Rails and you The inevitable discussion at the end around, should you use Merb and how does it relate to Rails. Some of the goodness in Merb looks to be slowly being backported to Rails, so over time Rails will benefit from all of the wizardry going on in Merb (hopefully!). For those that like Merb's approach, and willing to forsake the vast library of rails plugins, and/or have very specific performance requirements and/or need to hack a framework around to do what they want. Merb looks to be killer. Ultimately it's upto you, so evaluate carefully. Resources Lots of link goodness..... http://merbivore.com http://wiki.merbivore.com http://merbunity.com http://merborials.com/ http://groups.google.com/group/merb http://github.com/search?q=merb http://mwrc2008.confreaks.com/ http://www.slideshare.net/search/slideshow?q=merb http://www.slideshare.net/search/slideshow?q=datamapper

Read More

TorontoRB.ca

13 May, 2008

A while ago I was looking at all of the events going on in Toronto, mailing lists, *camps wondering how the heck does anyone get to know what’s going on?. Particularly those who are new to the scene ? After talking to a couple of people and realising the need for one central place to collect it all, I put together this blog. With my partners in crime, Corina and Pete, TorontoRB.ca will be kept up to date with when things are happening, goings on in the scene, opportunities to help out. Our vision is to keep you, our faithful Rubyist’s up to the play on what’s going on in our fair city. Add the feed to your readers, sit back, and soak up the centralised goodness. http://torontorb.ca If you're in Toronto, know of something I've missed. Have a blog, project, or anything that you think deserves a mention on there, send it into mail@torontorb.ca and it will get attended to in short order.

Read More

Minor gotcha and shout out to Passenger crew

06 May, 2008

Okay so yesterday I found a behavior problem with passenger (1.04) in that
request.env['PATH_INFO']
was nil, whereas under Mongrel it was being set as we were expecting to the current path. Within less than I guess 2 hours of reporting it on #passenger, code was put in place and committed to github. Now how's that for amazing work ? You've got to hand it to the boys over at the Passenger team. I sat back in my chair at that point and thought about the days before dealing with open source. I recall a dev team I was working with found a bug in number handling with .NET, what could they do ? For all the money paid in licensing fees etc ? Nothing. Just code around it. Days like yesterday I really enjoy the open source community, it certainly has some major advantages. I still have a lot of testing to do before I'd happily switch a production app over to Passenger, but looking forward to the promise of better deployment that's for sure!

Read More

Toronto Ruby / Rails madness

05 May, 2008

If you're in Toronto and been living under a rock you probably haven't realised that every month, there are a number of events specifically for Ruby/Rails developers. There's the ever popular and well attended Rails Pub Nite - great for networking and an excuse to have a pint or two. If this sounds like your thing join the other 40 odd that head out on the third Monday of the month at the Rhino. Pete Forde @ Unspace is the man on the scene for that one, kudos for keeping that for over a 2 years now (I think, don't quote me!!). The newish kid on the block the Rails Project Nite. Project nite consists of 3 presentations every time showcasing projects or rails specific learnings, usually showing nitty gritty code on some fantastic projects so quite enjoyable for us nerds! There's been some great presentations, so make sure if you've got any interest get down there, again solid numbers show up. Corina Newby has been putting this together, major props there for getting the new event up and running. If you have an interesting project that you would like to present, just contact her and volunteer your name, and breif summary. Make sure you RSVP on the Project nite, if not you could miss out Finally there's the Toronto Ruby users group, meeting on second sunday of every month at the Linux cafe - I will fess up that I haven't been to this one, weekends being usually DIY disaster time it's harder for me to make that one :) So if you can't find something in between those 3 that spark an interest in Ruby related development learnings, well then... there's no hope for you More details on the Ruby Users Group, Sunday May 11th - http://www.trug.ca/ More details this months Project Nite, Tuesday May 13th- http://correlations.wordpress.com/2008/05/04/tuesday-may-13th-ruby-on-rails-project-night/ More details on the Rails Pub Nite, Monday May 19th - http://www.unspace.ca/innovation/pubnite/ Finally, what mention of the Toronto Ruby/Rails scene wouldn't be complete without a Rubyfringe plug, haven't got tickets, get that sorted out ! - http://rubyfringe.com See ya at one of the above ! [Post script] Not entirely Ruby/Rails related I would be shot if I didn't also mention the Toronto Software Developers Lunch or the Ajax Pub Nite.

Read More

Rails + PHP sharing the same session

10 April, 2008

Ya know there's times in your life you do things that feel 'wrong'. This somehow feels that way, but one of my buddies wanted to know how it's done, so here it is: I've got an application I maintain in PHP which has been in production for a good number of years. It's stable, works really well, however like any application I need to keep putting in enhancements, the pain curve of doing this is exponential with the time I've been using Rails, so over a couple of years you can imagine the pain is pretty high. Last week I thought to myself, enough is enough. I'm sick of writing PHP, I could do this so much faster in Rails. I've been churning away on a rewrite in the background, in Rails as and where I have time but I wanted a 'fix it now' kind of solution. So, why not have the same virtual host serve out both Rails & PHP, so long as the Rails app can find out if a user is currently logged in, then I can piece by piece enhance/replace the PHP app. Turns out it was pretty damned simple. Here's a quick guide assuming 1- Both apps are sharing the same database 2- In the session you're storing a user_id of the logged in user 3- App uses cookies So here we go...

1. Virtual Host Config

The first thing we need to do is get Rails to run under the same vhost as the PHP scripts are run. This is easy, just adding a ProxyPass directive into the Apache vhost for the site back to your mongrel(cluster), so lets say we had a REST resource "some_resource", you'd add in the ProxyPass line below

    ...
    DocumentRoot "/path/to/your/php/app/html"
    ProxyPass /some_resource http://127.0.0.1:3000/some_resource
    ...

Note that in production this would pass through to a balancer of course. The unintended consequence of this is that you also can share all of your assets served from your existing app without having to duplicate them in your Rails app.

2. Sessions - PHP

The next thing for this to work is we need to know for the Rails app that the PHP app has logged a user in - we need to share the sessions. There's not too much black magic here. When you start your session in the PHP app (well at least mine!) PHP was storing in a named cookie eg "myappname" a hash value via
session_name( 'myappname' );
. This hash value is used to look up the session tmp files stored on the server, inside those session files are the serialized objects that PHP stores in $_SESSION global. So you have three options. 1. The "read from file system and unserialize the session" route 2. Store the session in memcached and have both PHP/Rails connect to memcached 3. Use the db to store sessions and logged in user_id's, then load the user on each page load. Note there is a page on this on the rails wiki, devoted to approaches 1 & 2. However if you're not storing a tonne of variables in the $_SESSION then approach number 3 - the path of least resistance - will likely be just dandy for you... In my database I created a shared_sessions table - using a rails migration (detailed in step 3 below) This stored a unique key "session_hash", as well as other fairly understandable columns "user_id", "updated_at". As we're logging in and out from the PHP app, this will mostly maintain the sessions table. So I added a quick couple of functions Application_SetSession() and Application_ClearSession(). Application_SetSession() simply writes into the shared_sessions table the hash value from the cookie $_COOKIE['myappname'], the corresponding user_id, as well as the time of update, it looks something like this:
function Application_SetSharedSession($user_id)
{
    global $DB;
    $session_hash = $DB->real_escape_string($_COOKIE['myappname']);
    $qry = "REPLACE INTO shared_sessions SET `session_hash` = '{$session_hash}', 
                      `user_id` = '{$user_id}', `updated_at` = NOW()";
    if ( !$DB->query($qry) ) throw new DBException($qry);
}
ClearSession() clears out the appropriate shared_session record. As you can well imagine ClearSession() is called when the user logs out, SetSession() when they login as well as every page load. Your PHP app will now be setting/clearing the shared_session records as appropriate.

3. Sessions - Rails

  rails myphphack
  cd myphphack
  script/generate model SharedSession
  script/generate model User
  script/generate scaffold SomeResource name:string ... 
Right now you need to fill out the SharedSession model's migration, and delete the user model migration (it's already in the database!). In the SharedSession migration, make sure you're setting the unique index on session_hash, otherwise REPLACE INTO won't work. So, first we need to load the user on the Rails side. To do this we need look up the SharedSession model for the session hash, from the cookie that was set by PHP. It's available of course, doesn't care who set it we have access to it.
class ApplicationController < ActionController::Base
  before_filter :restore_user
  ...
  protected
  def restore_user
    unless @user = User.restore_from_php(cookies[:myappname])
      redirect_to '/login.php'
    end
  end
  ...
end
On the user model, we now need a method to find the user from the session_hash:
class User < ActiveRecord::Base
  has_many :shared_sessions
  def self.restore_from_php(session_hash)
    find(:first, :include => :shared_sessions, 
          :conditions => ["shared_sessions.session_hash = ?", session_hash])
  end
end
Easy peasy.

4. Testing time

Run the migration, fire up a mongrel instance on 3000, hit the appropriate vhost, login to your PHP app and go to /some_resource. With any luck you'll be able to hit the /some_resource path with a logged in user.

Now then, it's not quite that simple

The astute ones will notice "Hey what happens if I login from PHP, see /some_resource, then come back directly to /some_resource in future - I could do this at any point in the future, it won't log me out". Exactly, you can handle this one of two ways. Either some nasty cron job thing to clear out the SharedSessions table, or do it as part of the load_user filter in the rails app - ie before restoring the user, SharedSession.delete_all("updated_at < DATE_ADD(NOW(), INTERVAL -15 MINUTES)") or whatever matches your current PHP session timeout settings. You also probably want to do some hash of the user agent, ip address etc and store this away in the shared_session table, then check against it whenever you're finding if a user exists, to try to eliminate cookie theft.

The fine print

It's pretty simple and it works in testing... YMMV. Test the hell out of it in your own application.

Read More

Tips of the day - Query Trace and Equality

07 March, 2008

Tip #1 Query Trace So you're profiling your application, cutting down your SQL queries, looking through the logs - "argh where did that query come from ?!" you say to yourself. Then you remember what Luke G said at the last Toronto Rails Project nite - check out Query Trace. Oh how this has made my life so much better. It tells you the stack trace at the point the query was executed, colorised nicely so you don't go blind tailing your log files. Here's a very simple example so you get the idea.. Query trace output Get the plugin from here https://terralien.devguard.com/svn/projects/plugins/query_trace/README Tip #2 Object equality vs checking id's When checking for ownership or association between objects, be careful you're not checking for equality of the objects. For example "did person x create this order y" the temptation is simply to go "if person == order.creator", where you're checking the objects themselves. Careful! As this is actually loading up the creator of the order if it's not already loaded (SELECT * FROM people WHERE person.id = x). Simply check if person.id = order.creator_id, as this won't require the order's creator to be loaded, eliminating one less hit to the DB. Little things like this littered through an application can start really slowing things down if you're not careful.

Read More

Rails vs Merb & ActiveRecord/Datamapper

19 February, 2008

If you're getting to this page from searching "Rails vs Merb" I suggest you read/view this presentation I did http://work.rowanhick.com/2008/06/11/toronto-rails-night-merb-presentation/ Following on from my presentation last week, I had someone mention I should look at Merb+Datamapper if I'm looking for something awesomely fast. So I decided to put it to the test. This is a *completely* contrived - I've done enough benchmarking to know this, so take the following figures with an (extremely) large grain of salt, I'm just posting this for interest. The view being rendered is a real world view that I'm using for a rewrite of a PHP/MySQL app, listing out orders by order status, along with their customer and originating country. I've got a database of Orders(770), OrderStatuses(17), Customers(300), Countries(244). I have a find by sql statement I'm using to pull upto 100 orders like so: "SELECT orders.id, orders.created_at, customers.name as customer_name, countries.name as country_name, order_statuses.name as status_name FROM orders LEFT OUTER JOIN `customers` ON `customers`.id = `orders`.customer_id LEFT OUTER JOIN `countries` ON `countries`.id = `customers`.country_id LEFT OUTER JOIN `order_statuses` ON `order_statuses`.id = `orders`.order_status_id WHERE (order_status_id < 100) ORDER BY order_statuses.sort_order ASC,order_statuses.id ASC, orders.id DESC LIMIT 100" The same statement is used in all 3 applications, along with the same generated html. All 3 apps were run daemonized in production mode using mongrel. The resulting page is comprised of a layout + js + css files, that each app is serving identically. Using Rails: 42.16 req/s (baseline) Using Merb + ActiveRecord: 57.80 req/s (1.37x) Using Merb + Datamapper: 72.98 req/s (1.73x) So lets up the order count to 11550. As the volumes of data goes up (and therefore database tuning becomes more important) these are the following figures... Using Rails: 22.82 req/s (baseline) Using Merb + ActiveRecord: 29.85 req/s (1.30x) Using Merb + Datamapper: 33.91 req/s (1.48x) Merb+Datamapper still outshines the competition, but the difference in performance isn't quite as significant, however once the DB is tuned, I bet life gets more interesting.... Again before I get lambasted, take the figures with a grain of salt, the times do vary quite significantly as there are variables aplenty, just look at the general trends.

Read More

How to Avoid Hanging Yourself with Rails

12 February, 2008

The law of unintended consequences. Evidently people are searching on figuring out how to leave this earth in Google and getting to this page because of the title with alarming regularity. If you're one of these people PLEASE see a psychiatrist, talk to someone you love, take a deep breath, realise that you've got your whole life ahead of you, whatever it takes. This evening I presented at the monthly TSOT Toronto Rails Project Nite. As promised here's the presentation and links to resources mentioned. Big thanks to TSOT, everyone who came, everyone else who presented and the spirit in the T. Ruby/Rails community. Presentation - How to Avoid Hanging Yourself with Rails Faker - faker.rubyforge.org 'mrj's ActiveRecord select/include patch - http://dev.rubyonrails.org/attachment/ticket/7147/init.5.rb include preloading optimisation - http://blog.codefront.net/2008/01/30/living-on-the-edge-of-rails-5-better-eager-loading-and-more/ Some points following on from this: - YMMV, times were produced using a consistent dataset, results are going to be all over the place dependent on columns, table size, indexes, ordering etc etc. - By testing with real (/faked) data, you can avoid (to some degree) the premature optimisation problem, you'll see what your app is going to perform like closer to a real world production environment. - Word on the street is Datamapper + Merb is wickedly fast, I'm going to run a test with the same dataset just to see what the difference is like - There were no MySQL optimisations preformed, just a vanilla MySQL install. - I'm not being negative on the core team about the select/include problem, it's a hard problem to fix, I'm just making people aware of it so you can save the banging-of-head-on-desk problem others of us have suffered. - If the presentation bored you to tears, you know it all already, live in Toronto, and are bored in your job - we may have a position for you. Talk to me... If you're in T. make sure you get yourself along to the Rails pub and Rails project nites, well worth it for networking and learning respectively. I hope to see every developer in T. putting forward a presentation on their projects, by collaborating and sharing we all learn immensely. Personally I've found every presentation exceedingly interesting - keep it up !

Read More

require, RMagick, and case sensitivity

19 December, 2007

(At least on Leopard with Ruby 1.8.6) Ruby will try to reload rmagick if you get the case of rmagick different from the original require. And when that happens it all goes up the whop. We were using attachment_fu's image magick processor, then doing a require 'rmagick' seperately which resulted in tonnes of errors like this
/usr/local/lib/ruby/gems/1.8/gems/rmagick-1.15.10/lib/rmagick.rb:32:
  warning: already initialized constant PercentGeometry
Keep your case's of require's consistent. Even if this is file system dependent behaviour in mixed development environments you can imagine how weird problems could get!

Read More

HTML_QuickForm in .... Rails ?

28 November, 2007

Caution Thinking Out Aloud: Okay the purists have just decided to kill me. Let the religious wars begin ! No seriously, everytime I do some coding in an old PHP app which is heavily dependent on HTML_QuickForm, I keep questioning how possible would it be to convert it across to Rails, and whether any other persons like myself would think all of their Christmases have come at once. Here's my hopefully rational thought processes behind this. Rails is a fantastic CRUD machine. If you want to build single model forms with validations in them, absolutely beautiful. Nothing could feel more clean and make you as a developer go 'thats sweet'. Fantastic. Now try and do a model-less form, with validations, or combine multiple models within one form. Things like the presenter pattern get bandied about. Real world case example - a form to choose options for defining a report. Nothing that you want to put in a model as it's not a represenation of a 'thing' (although some may argue it is). I wonder and am sitting on the fence of, whether to try implementing quickform for rails. For those that haven't been exposed to it, in a nutshell it's a brilliant little php library that lets you create a representation of a form, with rules, pass the form into a renderer which dumps out the html on a page, but that's not all, it assists you in processing the form on submission. So some code can look like this (not syntactically correct, but close enough), which does all of the hard work of writing out html, setting up client side js, and server side processing.
//page myform.php

$myform = new HtmlQuickForm('form_name', 'myform.php'); 
$abc = $myform->addElement('text', 'product_name', 'Product Name:'); 
$myform->addRule('product_name', 'required', 'You must enter a product name'); 
if ($myform->validate()) {
   //process form
   var_dump( $abc->getValue() ); 
} else {
   //setup form 
   $abc->setValue('123')
}

// then in any template which dumps out the rails code
$myForm->display(); 
It doesn't seem like much, but put half a dozen or more options on the form, different processing rules, and suddenly it's done a lot of the heavy lifting for you. What would it take to add this to Rails ? How would (I personally) fit it in. It doesn't really sit within either the model, view, or controller. The view should render the output, it doesn't really fit within a model as it's not a representation of a thing. It most closely sits in the controller, but you don't really want to clutter the controller with your form models. Here's some thoughts An initial idea, with everything defined in the controller, then dumped into the view by a @my_form.output
class FormController < Application 
  def do_form
    get_form
    if @myForm.posted 
       #do stuff
    else
       #populate stuff
    end
  end

protected
  def create_report
    @myform = RFormBuilder.new do |f|
       f.call_back_url = { :controller => 'form', :action => 'do_form' }
       f.name = 'myform'
    end
    @myform.add_text( :name => 'product_name', :label => 'Product Name', 
              :rule => { :required => :true, :msg => 'You must add this' } )

  end

end

OR Maybe we add in a new folder in the main app
  /app/
      /controllers
      /models
      /views
      /forms 
        myform.rb
And the treat myform.rb kinda like a model, but kinda not.

class MyForm < RFormBuilder
  defaults :name => 'my_form', :call_back_url => '/myform' 
  text_field :name => 'product_name', :label => 'Product Name', :required => :true
   
  #gets called when form is posted   
  def validate
    return true if field_product_type == 'abc'
  end 
end


# now in FormController 

 def do_form
    get_form
    @my_form = MyForm.new
    if @myForm.valid
       #do stuff
       render :text => @my_form.field_product_name
    else
       #populate stuff
    end
  end

What do you think ? Am I stark raving mad. Could this just work ? I think so and am willing to take up the challenge... Rowan

Read More

Gotta love *those* recruiters

01 November, 2007

From a blanket spam email I (and probably a good portion of the rails community just received) "I have a direct client who is seeking a strong Programmer who posses experience with Rudy on Rails. If you are qualified and interested, please send me your most current resume in a word.doc format to.." Yes, Rudy was actually put in bold, just to place even more emphasis on the spelling error.

Read More

The satisfaction of Rails

13 August, 2007

More work on XChain. In some spare hours on the weekend I implemented the basic structure of price catalogues for products. I can't tell you how long this took when previously doing it in PHP a few years back! (or even worse ASP a few years further back still). I'm sold on BDD, it was so quick to bang this out starting with the spec it first approach, mainly because all you need to do is 'rake spec', and bingo your code is tested... As follows, note code is cut down a little, but you get the picture:
# A product itself does not contain any price information, it has many price's in a 'pricing' model, that is linked
# to price types, eg "Standard Distributor" "Standard Retail"
class Product < ActiveRecord::Base
  has_many :pricings
  has_many :price_types, :through => :pricings
  belongs_to :order_line
end

#The pricing table, not only links to a product and price_type, it also has a discount
#model associated with it..
class Pricing < ActiveRecord::Base
  set_table_name 'pricing'
  belongs_to :product
  belongs_to :price_type
  belongs_to :discount
end

# discount linked to the pricing, and discount tables to enable volume based discounting.
class Discount < ActiveRecord::Base
  has_many :pricing
  has_many :discount_tables, :order => "qty ASC"
  #note that I don't want to do queries every time I ask get_discount, so I just iterate over the
  # tables to find the discount, rather than doing a find each time.
  def get_discount(qty)
    discount_tables.each do |dt|
      return dt.discount_applied if qty <= dt.qty
    end
    return discount_tables.last.discount_applied
  end
end


# In order to calculate price, we need to determine the pricing model for the order (taken from price type)
# then look up the unit price for the product according to the price type
# finally get the discount for the volume at that price type.
class OrderLine < ActiveRecord::Base
  belongs_to :order
  belongs_to :product
  def get_price(price_type)
    @pricing = product.get_pricing(price_type)
    price_as_ordered = ( get_unit_price * qty_ordered ) * get_discount
  end

  protected
  def get_unit_price
    @pricing.get_unit_price()
  end

  def get_discount
    @pricing.discount.get_discount(self.qty_ordered)
  end
You feel like you accomplished something when you sit down to work for a few hours and you get this...
=====================
rowans-computer:~/work/xchain rowan$ rake spec;
(in /Users/rowan/work/xchain)

DiscountTable
- should be valid

Discount when loading existing discount
- should return a max discount of 0.25 for a quantity of 10000
- should return a discount of 1 for a quantity of 20
- should return a discount of 1.5 for a quantity of 5
- should respond to get discount
- should have a discount table
- should have a name
- should belong to a pricing

Order when finding existing order
- should have a customer
- should have a billing address
- should have 3 order lines
- should have a purchase order number
- should have id = 1

Order with a single, invalid order line added
- should after changing the quantity be able to be saved
- should not be able to be saved as it has incorrect quantity
- should calculate to a correct amount

Order new draft order
- should be in the order collection list
- should belong to the correct customer
- should have an associated user who created it
- should be able to have it's address changed
- should be able to be changed to a new status

Order when creating a new order
- should when valid and saved, have one order status history item
- should be able to have an order line added
- should throw an error that it doesn't have line items
- should calculate return 0 dollars
- should have a default price type matching customer price type
- should be able to prefill an address
- should default to draft status
- should be able to instantiate

OrderStatus using a draft order status a super user
- should have a list of all available statuses

OrderStatus using a draft order status with no super user
- should not have all statuses available
- should have a list of next available status, when flow next is false, only including itself and the next higher status

OrderStatus
- should be able to find standard statuses

Product when using Product ID 1
- should be valid

OrderLine with valid order line
- should return correct discount levels (NOT IMPLEMENTED)
- should after changing qty return a different price (NOT IMPLEMENTED)
- should return a price of for x price type and y qty with a discountable price type
- should return a price of for x price type and y qty with a non-discountable price type
- should respond to get_price
- should be valid

Finished in 1.72758 seconds

40 examples, 0 failures, 2 not implemented

Read More

Using Rails with Flex to manage long running tasks

31 July, 2007

UPDATE Jul 2008 - The API for BackgroundRB has changed and docs improved significantly since this blog post. Read the latest API calls over here http://backgroundrb.rubyforge.org/rails/. Don't reference the Rails code below if using latest BackgroundRB As soon as you start doing anything with photos sooner or later someone says "It would be really nice to upload these in a zip file", which then leads into a whole rabbit warren of issues, one of which you will inevitably come across is how to deal with a long running task on the server, in terms of a) getting it to actually complete and b) telling the user what's going on. There are various hacks for doing these, like running a shell script and writing out tmp files all over the place. Just plain ugly. Enter Backgroundrb, probably one of the greatest little inventions for Rails. It lets you kick off a background process, outside of the main rails app process, with the option to interact with the rails models or not. The beauty of this plugin is a little thing called MiddleMan, as the name implies it lets you interact with your background processes (Workers) from your Rails app. So, this gives us the ability, from your Flex app to invoke a Worker on the server side, then monitor that Worker until it's finished, and/or kill off the worker if need be (for slacking on the job?). What do we need. 1. First off install BackgroundRB from here. 2. Next we need to create our worker in your Rails app dir 'script/generate worker ResizerWorker . Here's a simple chunk of worker code.
class ResizerWorker < BackgrounDRb::Rails
  def do_work(args)
    @total_images = 50
    @current_image = 1
    50.times do
      @current_image += 1
      sleep(1)
    end
  end

  def get_progress
    @current_image < 50 ? status = 'In progress' :  status = 'Finished'
    info = { :status => status, :current_image => @current_image, :total_images => @total_images}
    return info
  end
end
As you can see we have a do_work method which is generated by default, this is called whenever the worker is instantiated so it kicks off our work to be done. Then I added a get_progress method to return a hash of the worker's current status. 3. Great so now we need to kick it off, there's some options to do this automatically at timed intervals, but I'm not going to go into that here, all we're doing is getting our rails app, by a web request to start/stop and monitor the progress (the web request is then called from Flex) . So we create a watcher controller, with actions 'start_task', 'status' and 'stop_task' as below which is all pretty self explanatory:
class WatcherController < ApplicationController

  def start_task
    session[:worker_key] = MiddleMan.new_worker(:class => :resizer_worker)
    info = {:message => 'job_started', :key => session[:worker_key]}
    render :xml => info.to_xml(:dasherize => false)
  end

  def status
    @worker = MiddleMan.get_worker(session[:worker_key])
    if @worker
      info = @worker.get_progress
      if info[:status] == 'Finished'
        MiddleMan.kill_worker(session[:worker_key])
      end
    else
      info = { :status => 'Finished' }
    end
    render :xml => info.to_xml(:dasherize => false)
  end

  def stop_task
    @worker = MiddleMan.kill_worker(session[:worker_key])
    info = { :status => 'Task Stopped' }
    render :xml => info.to_xml(:dasherize => false)
  end

end
Gotcha! I found out the method names in the Documentation, on MiddleMan, were wrong compared to the version I had installed. I needed to use .new_worker, .get_worker and .kill_worker. 4. Fantastic, so go into console, do "rake backgroundrb:start" in your rails directory, then fire up a rails server. You should now be able to hit http://localhost:3000/watcher/start_task, http://localhost:3000/watcher/status, http://localhost:3000/watcher/stop_task and see the appropriate results. 5. Now you can fire HTTPService requests on the Flex side and consume the results as required. For example : (for the sake of berevity I haven't put in any timer function to recheck the status, just a manual click event)



	
		import mx.rpc.events.ResultEvent;
		import mx.controls.Alert;

		private function onSvcWatchTaskResult(event:ResultEvent):void
		{
			trace(event.result.valueOf());
			var info:XML = XML(event.result);
			pbProgress.setProgress( info.current_image, info.total_images );
		}

		private function onSvcStartTaskResult(event:ResultEvent):void
		{
			Alert.show('Task Started');
		}
	





	
		
		
			
			
			
		
	


And there you go. A few seemingly simple lines of code has knocked off one of the age old problems of dealing with long running tasks. I heart Rails (and Flex).

(As always with code examples, don't copy and paste, rewrite it yourself. You'll understand it better and won't copy in any errors I've made whilst cutting it down for blogging)

Read More

Syncing up external Filemaker data with Rails

28 May, 2007

One of the web apps I work on leverages a Filemaker database. My co worker Brent the expert Filemaker dude, and I had a serious problem to tackle - how to get Rails and Filemaker talking to each other. Now Filemaker has iffy SQL standard support and we didn't necessarily want our web app to run live from the Filemaker database. We spent some time looking at making an ActiveRecord adapter for FM but really at the end of the day it was too much like hard work. There are 3rd party solutions out there that can do this stuff for you, but if your needs dictate that they won't work - our experience may help you. I'm going to write up the theory behind this which has been live in production for a good 6 months + and in the coming weeks will release some code acts_as_syncable or similar... First off we wanted it occassionaly connected, one way synchronising. We had an installed instance of Filemaker Web Publishing engine so we could expose our data as xml views. We used a background rails (backgroundrb) process to query Filemaker every 5-10mins and ask for changed data, pull that data using REXML, and populate/update on the Rails side using ActiveRecord. Here's the mechanics... 1. Each object, needs a last_modified timestamp, whenever an update to the object that affects the data being synced, this last_modified timestamp needs to be updated (for all intents and purposes create an updated_at column on the filemaker side). 2. Next on the Filemaker side create a layout that exposes that data when queried via Web Publishing, as an xml result. We're going to do two different queries so your layout must support these - as per note ii. 3. On the Rails side we need to add two models, and extend the models we want synced. First I created a SyncLog model which will track when syncing occurs (just for informational purposes), then a ClassSyncLog, which will track the models synced, when they were synced, the last timestamp and id of each model synced. 4. For each model on the rails app side, we need a number of properties: - The corresponding id field name from Filemaker - A hash listing each rails field and it's corresponding name on the Filemaker side (we're assuming that these will be different, in our case they were wildly different) - Some call backs, pre process sync, and post process sync if you need to do stuff before/after syncing. 5. We need a model that will actually do the work of syncing the objects. This is the involved part of the process that does the grunt work of syncing so pay careful attention. 5.1 Pass in a list of Class names, for each class name do the following 5.2 Query filemaker using Net::HTTP from the last updated_at timestamp and see if any records have been updated since then, if so move on to 5.3 or go to the next class (well... almost there's a trick here **) 5.3 For that returned result set from Filemaker (from the HTTP Raw data returned), fill an REXML object from the returned xml file. For each of the records in the result set do the following 5.3.1 Find or create the rails side object from the corresponding id in the resultset, check if you can sync it per the pre process rules if you have any. 5.3.2 Iterate over each of the fields in the result set, see if it matches a field in the field mapping on the rails class you're syncing (4). If it does update the current object from the result set. 5.3.3 Finally save the current object and move onto the next record 5.4 Now before moving onto the next class you need to save a ClassSyncLog record, with the last timestamp of the last updated record, and the last id. So next time you have a starting point to retrieve the records. 5.5 Move onto the next class 5.6 After completing all classes write out a log entry to say it's been done. 6. Wait 5mins... and repeat. Of course there's some caveats. i. First is if you're dealing with 100'000's of records you do not want to grab just one resultset back - you're going to run out of memory and/or have problems with getting the full stream of data. So we get the result set query into groups of ~ 2000. ii. Related to this, lets say you have 100'000 records updated at 10:10:01 AM. The first request is going to be for all records greater than the last updated time, for example 9:30AM. So you'd get 2000 in on your 100'000 records, the next time in you ask for > 10:10:01 am, you're going to skip 98'000 records, so you want to do two queries first: a. Give me all records time = last_updated_time AND id > last_updated.id, which will catch the next 2000 records - repeat this one until you get no more results then: b. Give me all records time > last_updated_time - then go back to qry #a for the next repeat. iii. You will have performance issues, this isn't quick by any stretch of the imagination so be a little patient to work through the issues. We're running massive datasets so we often see updates that take hours. Not great but it's ok. iv. Watch your memory, this can be a long running process so throw in 'GC.start' to clean up memory along the way. (eg every resultset retrieved) v. To avoid ID collissions it's strongly recommended you use the MySQL master-master auto-increment-increment + offset scheme, so you stagger your ID inserts between your Filemaker DB(s) and MySQL DB(s). Again look for code in the coming weeks.. I'm wrapping our custom stuff up into an acts_as module for the (limited) audience this may help...

Read More

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.