Wednesday, November 18, 2015

useful Linux commands while developing rails web applications

Rails application best optimization concepts

Optimization is MUST of a Rails application specially for the high traffic Ruby On Rails application and for good UX. Here are some optimization tips which I already applied in my recent Rails projects.


Optimization of Rails Application and application server
Optimization of web server
  • Use nginx instead of Apache for better static resources(images,videos,css,javascript,text, html, etc) response
  • For static resources cached on client side for specific time(for example, 30 days), use this configuration
    For Apache, use the Location directive into your virtual host(be sure you have included expires and headers modules)
    Header unset ETag
    FileETag None
    ExpiresActive On
    ExpiresDefault "access plus 30 days"
For nginx location ~ ^/assets/ { expires 30d; add_header Cache-Control public; add_header ETag ""; break; }
view rawgistfile1.txt hosted with ❤ by GitHub
  • Make sure your web server is responding with gzip data
  • If your application faces heavy traffic then use load balancing. In one of my project, I used nginx as balancer on front end and 4 apache web servers on backend nodes. If you use cloud server like rackspace then you can use rackspace's builtin load balancer with your node servers or you can create your own load balancer
  • Use http reverse proxy like varnish cache server so that most of the requests are being served by the cache server instead of hitting your application server, see this link for varnish configuration
    http://abdul-barek-rails.blogspot.com/2012/07/integrate-http-reverse-proxy-cache.html
  • You can use http cache but be careful, its risky! because already requested requests(from a client) will not hit your application server(because of cache validation time), instead, the client will get response from its local cached data. To avoid this, you can change url.

  • Client Side Optimization
    • Try to use more Asynchronous requests using AJAX for better user experience and responsiveness in your application where you can.
    • Try to reduce number of requests which is the key optimization on client side
    • On production environment, use Rails asset pipeline feature so that your JavaScripts and css files are being combined and minified or compressed
    • Use css stripe to avoid loading more images(same type of images) so that you can reduce number of http requests for images
    • Try to reduce static resources weight(reduce size in KB or Byte)
    • You can use jquery lazy load plugin to load images lazily or you can use jquery appear plugin to load images/contents on when screen appears to view
    • To measure your application's load time and performance, use this google tool
      https://developers.google.com/speed/pagespeed/insights
    • For better inspection of your server's resources on browser, use firefox's firebug adons in firefox browser and see how much time all resources are taking to load and see how weight of resources are and see what are the unnecessary http requests and then reduce resources weight and load time and reduce unnecessary http requests as you can.
    Database Optimization
    • Apply database indexing on table's those columns which are being used for join, foreign key, ordering or on where clause
    • Apply data reporting, I mean instead of pulling big data result set on the fly, prepare them before they are being used, you can use any scheduler gem like rufus scheduler or delayed job to make stuff pre-ready.
    • Inspect slow query from your database server log file and optimize it
    You can use NewRelic for details servers logs, graphs, database logs and many many stuff to analyze more.

    Sunday, May 5, 2013

    Rails application best optimization concepts

    Optimization is MUST of a Rails application specially for the high traffic Ruby On Rails application and for good UX. Here are some optimization tips which I already applied in my recent Rails projects.


    Optimization of Rails Application and application server
    Optimization of web server
    • If your application faces heavy traffic then use load balancing. In one of my project, I used nginx as balancer on front end and 4 apache web servers on backend nodes. If you use cloud server like rackspace then you can use rackspace's builtin load balancer with your node servers or you can create your own load balancer
    • Using AWS Load Balancer for many EC2 instances is also a great way to divide traffic.
    • AWS Autoscaling is also an AWESOME automated way of scaling your app based on your traffic
    • Use nginx instead of Apache for better static resources(images,videos,css,javascript,text, html, etc) response
    • For static resources cached on client side for specific time(for example, 30 days), use this configuration
    • Make sure your web server is responding with gzip data
    • Use http reverse proxy like varnish cache server so that most of the requests are being served by the cache server instead of hitting your application server, see this link for varnish configuration
      http://abdul-barek-rails.blogspot.com/2012/07/integrate-http-reverse-proxy-cache.html
    • You can use http cache but be careful, its risky! because already requested requests(from a client) will not hit your application server(because of cache validation time), instead, the client will get response from its local cached data. To avoid this, you can change url.

    Client Side Optimization
    • Try to use more Asynchronous requests using AJAX for better user experience and responsiveness in your application where you can.
    • Try to reduce number of requests which is the key optimization on client side
    • On production environment, use Rails asset pipeline feature so that your JavaScripts and css files are being combined and minified or compressed
    • Use css stripe to avoid loading more images(same type of images) so that you can reduce number of http requests for images
    • Try to reduce static resources weight(reduce size in KB or Byte)
    • You can use jquery lazy load plugin to load images lazily or you can use jquery appear plugin to load images/contents on when screen appears to view
    • To measure your application's load time and performance, use this google tool
      https://developers.google.com/speed/pagespeed/insights
    • For better inspection of your server's resources on browser, use firefox's firebug adons in firefox browser and see how much time all resources are taking to load and see how weight of resources are and see what are the unnecessary http requests and then reduce resources weight and load time and reduce unnecessary http requests as you can.
    Database Optimization
    • PostgreSQL database is better than MySQL for large data sets
    • You could use AWS RDS to hold data for better latency and data backups
    • Apply database indexing on table's those columns which are being used for join, foreign key, ordering or on where clause
    • Apply data reporting, I mean instead of pulling big data result set on the fly, prepare them before they are being used, you can use any scheduler gem like rufus scheduler or delayed job to make stuff pre-ready.
    • Inspect slow query from your database server log file and optimize it
    You can use NewRelic for details servers logs, graphs, database logs and many many stuff to analyze more.

    Thursday, July 5, 2012

    integrate http reverse proxy cache server with Rails and Varnish



    • Setup Apache and make a virtual host to listen it on other port like 8080
    • Install varnish and listens it on port 80
    • For the Varnish, you need to do configuration on two files like bellow





    Here is the requesting flow: Browser ---> Varnish---->Apache2----->Application Server
    That means, when u request through browser, it request to Varnish(on port 80) and if varnish has cached then it served directly to client(browser) without interfering apache otherwise Varnish passes the request to apache...In this way you can scale your Rails app heavily and serve a lot of requests concurrently with low memory and low resource consumed.

    Some varnish useful commands:
    • Restart Varnish - /etc/init.d/varnish restart
    • Show Varnish statistics - varnishstat
    • Test to see response is coming through Varnish -
      curl -I http://www.your_app_url.com



    Friday, February 10, 2012

    Parallel Processing in Ruby On Rails

    I used ruby's fork when I was sending 60,000 emails from one of Rails applications and it is wise to split the big result set and send emails of each chunk parallely on forks for faster processing! Here is a sample ruby code for processing N length data.

    I tested it on Ubuntu 11, ruby 1.9.2, Rails 3.0.10

    Friday, December 16, 2011

    argument out of range and 0.0 issue on rails

    Argument out of range Exception on windows7 with Rails3 and model fields class type is float(0.0). Actually I was trying to use mysql2 gem on windows7 in Rails3.

    How I fixed it in my case:
    • My system was x86 typed(windows7) and I had 32 bit mysql client installed in my machine with Rails 3.1 and Ruby 1.9.2
    • I downloaded libmysql.dll (32 bit) and placed it on windows/system32 and ruby/bin folder
    • I restarted my laptop and FIXED!
    This is nothing but libmysql.dll version related issue with system!

    Saturday, February 27, 2010

    Dynamic Image from text/string in Rails

    Image will be generated dynamically from string or text:

    Use - require 'RMagick' in your class



    Pre-requisite : You must have rmagick and Imagemagick installed.
    Details: http://www.simplesystems.org/RMagick/doc/draw.html

    Monday, February 22, 2010

    How to develop rails plugin from scratch?

    Generate Plugin with command: ruby script/generate plugin hello_world
    It will create file system as bellow:
    - lib
        - hello_world.rb
    - tasks
        - hello_world_tasks.rake
    - test
        - hello_world_test.rb
    - init.rb
    - install.rb
    - uninstall.rb
    - README
    - Rakefile
    - MIT-LICENSE

    init.rb will be executed every time when your application runs. Generally hook code is to be included here like you want to make all methods of your plugin available in your app’s models, controllers, views and helpers

    Example:
    #All methods in module HelloWorld will be available in all #model’s object
    ActiveRecord::Base.class_eval do
        include HelloWorld
    end

    #All methods in module HelloWorld will be available in all controllers
    ActionController::Base.class_eval do
        include HelloWorld
    end

    #All methods in module HelloWorld will be available in all views and all helpers
    ActionView::Base.class_eval do
        include HelloWorld
    end

    lib/ hello_world.rb :

    Example:
    # All methods in this library will be available in all models, controllers, views and helpers if you write code as init.rb above

    module HelloWorld
        def say_hello
            return "Hello World"
        end
        def hello_text
            return “this is text”
        end
    end

    Now you need to do unit test your plugin’s methods. To do so follow:
    test/hello_world_test.rb:

    Example:
    require 'test/unit'
    require File.join(File.dirname(__FILE__),'../lib/hello_world.rb')

    class HelloWorldTest < Test::Unit::TestCase
        include HelloWorld # includes your library methods to test

        def test_this_plugin
            hello_world = say_hello
            assert_equal hello_world, "Hello World"
            assert_not_nil hello_world
        end

        def test_another
            assert_equal true,true
        end
    end

    To run these test cases, go to your plugin’s directory with command prompt and run like: ruby test/hello_world_test.rb

    You are done actually with plugin!

    Install.rb: It will be executed only once when the plugin is being installed.

    Uninstall.rb: This will be executed when you do uninstall your plugin

    README: In this file, you should write easy documentation of your plugin with examples. Also you may highlight yourself here.