Essential Node.js Modules for Web Application Development

In this blog post I have tried to explain essential node.js modules which may help you to make your web application robust and more users friendly.

1. Express Framework – Go to Website

Express is a minimal and flexible node.js web application framework, providing a robust set of features for building single and multiple pages, and hybrid web applications. It is complete package for your framework related requirements. Visit this website to know how it works and who uses express framework.

Install via npm: $ npm install –g express

2. Jade – Go to Website

Jade is a high performance template engine heavily influenced by Haml and implemented with JavaScript for node. Based on my experience it’s very easy to use and understand.  There are many instances while development, you will need HTML to Jade converter, so here it is for you.

Install via npm: $ npm install jade

3. My-Sql – Go to Website

My-sql module is driver for node.js. You can interact with any MySQL database, run any query, stored procedure. There are lot of configuration options available in this module which you can set depending upon your application needs. Interestingly this module also provide connection pooling mechanism which gives this module added advantage over other my-sql modules for node.js.

Install via npm: $ npm install mysql@2.0.0-alpha9

4. Mongoose – Go to Website

Mongoose is a MongoDB object modeling tool designed to work in an asynchronous environment. Mongoose provides a straight-forward, schema-based solution to modeling your application data and includes built-in type casting, validation, query building, business logic hooks and more, out of the box.

Install via npm: $ npm install mongoose

5. Session

Both express and connect has session management feature. If you want to learn how you can manage user’s session using express.js then read this interesting blog post.

Security is measure concern when working with any of the modules in this area. I don’t find any explanation on session replay attack and session hijacking attack prevention in express.js. So according to my experience you have to write your own logic to prevent session replay and session hijacking attack.

Also I would recommend you to read this blog post if you are really concerned about your website or web application security.

6. Nodemon – Go to Website

Nodemon will watch the files in the directory that nodemon was started, and if they change, it will automatically restart your node application. For use during development of a node.js based application. It’s a wrapper around node. Many times we keep changing our server side javascript files but we forget to restart node server, there this module helps us to automate this process.

Install via npm: $ npm install nodemon

7. Nodemailer – Go to Website

Nodemailer is an easy to use module to send e-mails with Node.JS (using SMTP or sendmail or Amazon SES) and is unicode friendly – You can use any characters you like.  Not only text message it also support large files as an attachment.

Install via npm: $ npm install nodemailer

8. Socket.io – Go to Website

Socket.IO aims to make real time apps possible in every browser and mobile device, blurring the differences between the different transport mechanisms. It’s care-free real time 100% in JavaScript. Real time dashboard, stock market applications are the best candidate where we can use this module. If you want to keep your users updated without refreshing page then there is no better option available then this module.

Install via npm: $ npm install socket.io

9. Async – Go to Website

Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. Originally it was designed for node.js but now you have option to use it on client side js if you wish to. If you are working with event driven programming language then module like this will be very handy and can save your lot of time.

Install via npm: $ npm install async

If you want to see practical examples for async module then I would suggest you read this blog post.http://www.sebastianseilund.com/nodejs-async-in-practice

10. Nodefly – Go to Website

Nodefly is application monitoring tool. You can monitor you application performance from anywhere in the world. When application go live you will always need some tool which continuously monitor your application performance and give you an alert whenever anything bad happens. Nodefly is very powerful module but it has only one requirement, internet connection on server. Nodefly is third party service which will collect performance metrics and sync it on their server.

Install via npm: $ npm install nodefly

11. Uglify – Go to GitHub 

Uglify helps you to make your web page lighter by making all JavaScript files minified on the fly. This means that developer will always work on non-compressed javascript version but when the same file reach to client (Brower) it will be loaded minified. This is the benefit of using node.js.  In today’s world loading web page faster is become essential otherwise no one would have time and care to visit your web page again.

Install via npm: $ npm install uglify-js@1

Flat File Database

Few days back, I was searching for database to store data. It may be big, small or medium size data. Operation which I was supposed to do was Select, Where and Aggregation nothing fancy. I started with mongoDB then I tried MySQL and lastly I ended with Flat CSV file. In this blog post, I have written my experience using all of the above systems.

I have inserted 34 Million+ rows in mongoDB, MySQL and Flat CSV file. Below are the sizes different system has created on disk.

mongoDB: 26 GB
mySQL: 13.37 GB
CSV File: 14.42 GB

My system’s configuration is
RAM: 4 GB
Processor: Intel i3 quad core

Insertion Speed
Record insertion speed in mongoDB is far better than MySQL. It’s almost 3-4 times faster than MySQL. Inserting record in CSV file has identical result as mongoDB. The reason is both are document oriented database.

 Select
I had done some simple select query with single field where condition in MySQL and mongoDB but result was very frustrating. After waiting for almost 20 minutes neither MySQL nor mongoDB has given any output. Then I had created index in MySQL but still result is identical. Even creating an index on such big table itself is big headache. It will take long to finish. In case of mongoDB it took almost 1.30 hour just to create an index and even after creating index mongoDB also had not be able to answer my query. Query was simple select with some where condition. You won’t believe guys I had tuned MySQL for optimal performance as recommended by MySQL itself. And also my table is MyISAM which is best engine for data retrieval.

 Aggregation
After simple select query I tried to fire some aggregation queries on MySQL and mongoDB database. Query which I had fired is simple group by on single column and sum of single column with some where condition on single field.

Select Plnt , Sum(Volumn) as Vol
From tbl_bigData
Where Region in (‘Gujarat’,’Delhi’,’Mumbai’)
Group by(Plnt);

This query is pretty straightforward and I was expecting result this time but despite of giving result, query was keep running. Another frustrating moment for me.

I have done all sort of tuning, indexing on my database in MySQL but I never achieved any output.

Then I have read somewhere about MapReduce in MongoDB. Firstly, I have created index of plnt, region column and then I submitted map and reduce function to mongoDB and then I executed MapReduce function in MongoDB. After waiting for 15 minute when I show console process was finished only 8% so without wasting time I stopped query execution.

Flat CSV File (Happy Moments Start)
After experimenting all of the above methods and tactics I got undesirable results, so I consulted one of my team members. He has very good knowledge dealing with flat file as database.

We had created method where we have to pass n-number of filter conditions (where conditions), n-number of aggregation parameters and resulting column index, because in CSV everything is based on index.

So using CSV file read not only I got desirable result but also good performance. Using single file read I am able to filter records and aggregate data all together. Interesting part is that you can return n-number of result set using single file read.

For example:
Result 1:
Filter : Region IN (‘Gujarat’,’Delhi’,’Mumbai’)
Select: Region, Plnt, Volumn

Result 2:
Filter : Region IN (‘Gujarat’,’Delhi’,’Mumbai’)
Aggregate: Sum(Plnt)
Select: Plnt, Sum(Plnt)

Yes, there are few limitations associated with flat file as database. If these limitations are unavoidable for you then this approach is not for you. According to me this approach is best use when you have big data say 100 GB+ data and you want some analysis on the that data.

Please correct me if I was wrong somewhere in this blog post.

Node.js Video Tutorial 1 of 8 – Introduction to node.js

In this

Node.js Video Tutorial

you will get an idea about how node.js works. You will also get an idea about how to install node.js. Using 5 lines of code you can create your own up and running web server.

get source code

Video by node.js creator himself. Video is is almost more than 1 hour long but I can guarantee you that you will know what you should know about before digging into node.js

What kind of application you can create in node.js
My answer is almost almost any application. You can create

  • Stock Market Application
  • Real Time News
  • Project Management Application
  • Web Based Game

or any application you can think of. Node.js is very fast try it and see what node.js can do for you.

Comment your thoughts.

Geographical SVG Map With Example

In web if you want to provide any geographical svg map related solution then consider map as some random shape. Every map is nothing but a random shape. And in web if you want to draw any shape SVG is the best option. SVG stands for Scalable Vector Graphics.

But to draw shape like this will require lot of skill and time. So, I find one website which gives us almost all countries detailed map in different formats like GIF, PDF, SVG and many other.

Please visit d-maps.com

I have taken India map from d-maps.com. This is SVG map.

Rest is depending upon your logic. :)

SQL Group By Age Range

SQL Group By Age Range

SQL Group By Age Range is tedious task because there is no direct method exists. So you need to apply some logic to get your desired result.

Let’s say you want to find total visitors count by specific age group.

For example result you want is something like
SQL Group By Age Range Output

Approach which I would suggest is simple and at the same time gives you facility to change age group range dynamically without changing your SQL query.

You have to do following things
1. Create master table for age group as shown below.
Create SQL Group By Age Range Master Table

2. Insert range data
Insert SQL Group By Age Range Master Data

3. Write SQL query using JOIN.

SELECT B.[RANGE] AS [Age Group(in Years)],COUNT(A.pk_ID) AS [Total Visitors]
FROM dbo.tbl_visitors A
RIGHT JOIN dbo.mstRange B ON A.AGE BETWEEN B.MINVALUE AND B.MAXVALUE
GROUP BY B.[RANGE],B.MINVALUE
ORDER BY B.MINVALUE

done !

Share your views!

Apply Culture In Silverlight Application

Culture In Silverlight Application plays a very important role. Silverlight application always sits on client browser and works as a plug-in. So you always need to keep in mind that control panel setting of different user might be different especially regional and language settings. Different user on different countries uses different date, time and number formats so while building Silverlight application you have to consider this fact and build your application accordingly and you need to make sure that you have applied culture in Silverlight application.

Recently I come across issue where application uses M/d/yy format date throughout application and it worked fine till user had not changed anything from regional settings. When user changed short date time format from control panel > regional and language settings application started giving errors of bad date and time. So I needed to apply default en-US culture in my application to resolve this issue. What I have done is I have applied en-US culture in current thread on application startup i.e. app.xaml.cs.

Here is the code for the same.

private void Application_Startup(object sender, StartupEventArgs e)
{
System.Globalization.CultureInfo c = new System.Globalization.CultureInfo("en-US");
c.DateTimeFormat.LongDatePattern = "dddd, MMMM dd, yyyy";
//YOU NEED TO PROVIDE ESCAPE SEQ IF YOU ARE USING SLASH IN YOUR FORMAT
c.DateTimeFormat.ShortDatePattern = "M\\/d\\/yyyy";
System.Threading.Thread.CurrentThread.CurrentCulture = c;
System.Threading.Thread.CurrentThread.CurrentUICulture = c;
}

I hope this will help others to solve the same problem.

JSON in Javascript Explained

What is JSON in Javascript?

JSON stands for Javascript Object Notation. JSON is nothing but an object which contains key value pairs. Advantage of JSON over other data format is it is lightweight. JSON is platform independent as well as language independent data exchange format.

JSON object example

Example:

var jsonObj = {id:1,name:”hiteshagja”};

In this example Keys are id and name and Values are 1(typeof int), hiteshagja(typeof string).

So If you write alert(jsonObj.name) in javascript you will get alert containing text hiteshagja. Isn’t this so amazing about JSON.

Add New Property(key) to already existing JSON object.

Example 1:

jsonObj.email=”hiteshagja24@gmail.com”;

Or

jsonObj[“email”]= ”hiteshagja24@gmail.com”;

This will add new key email to already existing jsonObj. You don’t need to write anything else other than this.

But if you write like

Example 2:

jsonObj.name=”hiteshagja.com”;

Or

jsonObj[name]=”hiteshagja.com”;

This will only update our JSON object key value from hiteshagja to hiteshagja.com.

JSON currently supports

String
Number
Object
Array
Boolean
Data types and
Null value

Get JSON object’s keys count

Object.keys(jsonObj).length

will return number of keys in given JSON object. In our case this will return 3(id,name and email).

Delete key

delete jsonObj[“email”]

this will remove email key value pair from jsonObj. Delete is a keyword which will accomplish this task.

String to JSON object

Example:

var jsonStr=’{“id”:”1”,”name”,”hiteshagja”}’;

Let’s say you have string value just like shown above and you want to convert it into JSON object.

var

jsonObj=JSON.parse(jsonStr);

Or

var jsonObj=eval(jsonStr);

will convert JSON string into object.

JSON object to String

In some cases you will be needed converting JSON object into string there you can use conversion like shown below.

Example:

var jsonStr= JSON.stringify(jsonObj);

This will convert jsonObj into string.

JSON Array

Till now you have seen how to create single JSON object. Now in some cases we will need to store data as an array. Let’s see how we can do so.

Example 1:

var jsonObj={id:1,name:”hiteshagja”,phone:[1111111111,2222222222]};

Here as you can see I have written phone value as an array.

You can access phone number using jsonObj.phone[0] just like you are accessing other array objects.

Example 2:

var jsonObj={

Products:[ {name:”Product1”

color:”Red”

code:1},

{name:”Product2”

color:”Green”

code:2},

{name:”Product3”

color:”Blue”

code:3}]

};

Example 2 is real time scenario where you will have array of products. jsonObj is array of products which contains 3 products in our example.

If you want to access product 2 then you can simple write

Products[1][Key].

I will keep updating this post whenever I will find anything interesting about JSON.

jQuery Signature Pad using HTML5 Canvas

With introduction of HTML5 Canvas we can easily create jquery signature pad plug-in. This article is all about jquery based signature pad that works in almost all modern browsers.

Recently I come across situation where I need to provide browser based Signature Pad and It should work fine in all the devices including apple devices. So I don’t have any other option left then HTML5 Canvas.

If you want to create a browser based signature pad then visit the link below.

http://thomasjbradley.ca/lab/signature-pad/

To save signed signature visit the link below. (CODE IS IN C#)

http://www.dotnetfunda.com/articles/article1662-saving-html-5-canvas-as-image-on-the-server-using-aspnet.aspx

Silverlight – Listbox Scrollbar Problem

Recently I have observed that listbox scrollbar has virtualization problem when we re-bind listbox. Let me explain this with an example.

I have added 50 items into listbox control programmatically. Here is the screenshot for the same.

Image after 50 Items bound to listbox

Normal Listbox Binding

Re-bind listbox

Re-bind button click without scroll bar move

As you can see I have added two buttons to Reset and to Re-bind listbox again.

If I click on any of the button everything will work as expected but real problem is when we move scroll bar to bottom of the listbox and we re-bind listbox data.

See what kind of problem you shall face.

Problem displaying data

Solution :

Use scrolltoview function after assigning ItemsSource to listbox. See code snippet below.

[sourcecode language="csharp"] List<ListBoxItem> a=GetItemsList(txt,count);

listbox1.ItemsSource = a;

if(a.Count()>0)

{

listbox1.UpdateLayout();

listbox1.ScrollIntoView(a[0]);

}

[/sourcecode]

Dealing with DateTime Formats in C#

Dealing with datetime formats in C# is very easy if you use framework functions properly. Recently I come across very ugly error which makes my application crashed. Actually it was problem converting DateTime for specific format. You should make your DateTime conversion independent of client computer’s culture setting. This thought in mind I am going to present this post.

1.       Convert String  to DateTime

Let’s take an example:

Suppose you have

[sourcecode language="csharp"] String strDate = "10/23/2011"; [/sourcecode]

Which is in MM/dd/yyyy format and you want to convert it in DateTime variable then you should write like

[sourcecode language="csharp"] DateTime dt = Convert.ToDateTime(strDate,System.Globalization.CultureInfo.InvariantCulture); [/sourcecode]

You could write above code if you are sure that the format is in MM/dd/yyyy. If the format is in say dd/MM/yyyy then DatTime.ParseExact is the perfect option for you.

Code above can be rewritten as

[sourcecode language="csharp"] DateTime dt = DateTime.ParseExact(strDate, "dd/MM/yyyy", System.Globalization.CultureInfo.InvariantCulture); [/sourcecode]

In all above code lines I have mentioned System.Globalization.CultureInfo.InvariantCulture parameter, which will make date format, culture independent.

2.       Convert Datetime to Specific Format

Let’s say you have

[sourcecode language="csharp"] DateTime dt = DateTime.Now; [/sourcecode]

and you want to convert it in different format. 

Example : Format : dd-mmm-yyyy

[sourcecode language="csharp"] String s = dt.ToString("dd/MMM/yyyy"); [/sourcecode]

But this will be overridden by current culture.

So you should do something like this

[sourcecode language="csharp"] String s = dt.ToString("dd/MMM/yyyy", System.Globalization.CultureInfo.InvariantCulture); [/sourcecode]

In short, CultureInfo parameter plays an important role.