Friday, December 18, 2015

Making JavaScript a more typed language - a safer way to use nodeJS

Javascript can be a great tool with how flexible it can be! Sometimes though the great parts are what also can make it hard to work with. Today I'd like to talk about two things that javascript provides and how I think they can be improved with a concept I'll call contracts.

1) First Javascripts ability for functions to take typeless objects/parameters.

Example:

function example( obj ) {}   //obj can contain any type of members and has no defined type

function example( a, b ) {} //a or b can be objects, strings, numbers etc. 


The problem here is knowing what to expect for a function, either the type or what variables are expected to be inside an object. Half of this problem could be solved through brute force. You could name the variables passed in. For instance, addTwoNumbers( firstNumber, secondNumber ). Somone still may call that function with a string when you are expecting a number. When objects are passed in to a function it is extremely hard to know what needs to be in the object without poking around in the function and this is very good way of introducing problems with your code!

2) Secondly Javascripts ability for functions to take any range of parameters.

Example:

function example( a, b ) {}    //can be called with  example(1) or example( 1,2,3)


Typlessness is great, but sometimes you'd like to make sure you know what you are getting!

Let's talk about the problem where people call your function with too few parameters and you are expecting all of them.

Example:
Your function is:    addTwoNumbers( first, second )
Someone calls it instead with:     addTwoNumbers( "5" );

Well you could solve this problem fairly easily by adding a bunch of conditional tests at the front of the function. For instance do this:

function addTwoNumbers( first, second, .., .., .. ) {

if ( typeof first === "undefined" or typeof second === "undefined" ... ) { throw "fail" }

}


This gets tedious very fast! Not only that but you don't really know if they pass in a string or number without further testing.

Introducing Contracts!


What if I told you that you could solve both problems at once and make your code much easier to read and understand! Well... you can with contracts!

A contract in this article is a template which describes how data must be structured. Contracts will be used to test the object passed in by a one line piece of code. It will also indicate what the function expects from what is passed in.

Example of a contract

Say we have a chart that you can add data to but you must provide a name and y coordinate. The function you might write will look like this:

function addChartPoint( chartPoint ) {


if ( !Verify( chartPoint, { Name: "", Y: 0 } ) {

   throw "missing data!";


chartPoints.push( chartPoint );

}



In this example, { Name: "", Y: 0} is the contract. In the verify() function it will check if chartPoint has the members Name and Y and then if Name is a string and Y is a number. Now you can see this makes it very easy to check the validity of your parameters AND let others know what is expected of what is passed in. You don't have to tediously write out checks for each item, how about that?!

So you are probably wondering what magic is in verify function. Well Javascript is so flexible that we can compare both objects by iterating keys to determine if they match or not to create a nice verify function.

Here is an example of a verify function. (Note this verify function does not check type in this example, just for names)

var verify = function( dataItem, template ) {

    var itemsToFill = Object.keys(template);

   

    for ( var index = 0; index < itemsToFill.length; index++ ) {

        var keyName = itemsToFill[index];

        var isObject = typeof template[keyName] === "object";

       

        if( typeof dataItem[keyName] === 'undefined') {

            //one of the expected types was not found!

            return false;

        }

        else if ( template[keyName].constructor === Array ) {

            for ( var item = 0; item < dataItem[keyName].length; item++ ) {

                if ( !verify( template[keyName][0], (dataItem[keyName])[item] ) ) {

                    return false;

                }

            }

        }

        else if ( isObject ) {

            if ( !verify( template[keyName], dataItem[keyName] ) ) {

                return false;

            }

        }

    }    
   

    return true;

};

exports.verify = verify;

As you can see this can be very powerful in making your functions easier to read and also test validity of parameters getting passed in! The code above also supports very complex templates but that is where the power lies! Try this one out for example:

function calculatePortfolioBalance( portfolio ) {


if ( !Verify( portfolio, { days: [ { stocklist: "", holding: { shares: 0, percent: 0}}] )

    throw "bad object passed";



}



This will make sure that we have a days array and that each days[0...n] contains a stocklist variable, holding object with children shares and percent. Pretty wild eh! This concept is even more powerful when you use it in conjunction with class declarations as well!




Wednesday, December 16, 2015

DynamoDB to javascript classes entity framework style with contracts

I am excited to show code from my previous post! Please read previous post if you haven't yet.

PortfolioBalance.db serves as a contract/template before it is filled with the object. The most important part about this is that it defines the object. I can't tell you how many times I end up trying to tab between the database webpage and trying to discern the right cell to figure out what it contains. Now it is very explicit and enforceable and can be done in levels! In the code below I do a trace if it doesn't find a variable, you could add more to enforce everything is there. If you look at PortfolioBalance.db.StocksPercent it is an array of 1 element. This single element is a template for objects in the array which I think is pretty cool. If you follow this pattern you will end up with beautifully formed data in your class. You won't ever need to tediously do constructor initialization or have to be cross referencing the database to find out what your class should contain! It is in one place!

Database.js

var createObjectFromTemplate = function( template, dataItem ) {
    var itemsToFill = Object.keys(template);
   
    var object = {};
   
    for ( var index = 0; index < itemsToFill.length; index++ ) {
        var keyName = itemsToFill[index];
        var isObject = typeof template[keyName] === "object";
       
        if ( template[keyName].constructor === Array ) {
            var array = [];
       
            for ( var item = 0; item < dataItem[keyName].L.length; item++ ) {
                array.push( createObjectFromTemplate( template[keyName][0], dataItem[keyName].L[item].M ) );
            }
            object[keyName] = array;
        }
        else if ( isObject ) {
            object[keyName] = createObjectFromTemplate( template[keyName], dataItem[keyName].M );
        } else if ( !dataItem[keyName] ) {
            //skip trying to set the item because it wasn't found in the data!
            console.log('!!!class is asking for member but not found in database ' + keyName );
        }
        else if ( dataItem[keyName].N ) {
            object[keyName] = Number(dataItem[keyName].N);   
        }
        else if ( dataItem[keyName].S ) {
            object[keyName] = dataItem[keyName].S;   
        }
        else if ( dataItem[keyName].L ) {
            object[keyName] = dataItem[keyName].L;
        }
        else {
            object[keyName] = dataItem[keyName];   
        }
    }    
   
    return object;
};

var convertToObjects = function( err, data, creationFunction ) {
    if ( err ) {
        return null;
    }   
   
    console.log( data.Items );
   
    var objects = [];
   
    for ( var index = 0; index < data.Items.length; index++ ) {
        var portfolioBalance = creationFunction();                                  //create a new class object
        portfolioBalance.db = createObjectFromTemplate( portfolioBalance.db, data.Items[index] );     //initialize the class object based on the db member, and row from database
        objects.push( portfolioBalance );
    }
   
    return objects;
};


PortfolioBalanceRepository.js

var getPortfolioBalanceForUserOnDate = function ( loginName, callback ) {
    var search = {
        TableName: exports.table.Name,
        KeyConditions: {
            "LoginName": {
                "AttributeValueList": [{
                    "S": loginName
                }],
                "ComparisonOperator": "EQ"
            },
        },
        ScanIndexForward: false,
        Limit: 1,
        ProjectionExpression: "#a,#b,#c,#d",
        ExpressionAttributeNames: { "#a": "StocksPercent",
                                    "#b": "CashPortfolioPercent",
                                    "#c" : "Date",
                                    "#d" : "PreviousCashPortfolioPercent"}

    };
   
    databaseLib.query( search, function( err, data ) {
        var objects = databaseLib.convertToObjects( err, data, function() { return new PortfolioBalance.PortfolioBalance(); } );
        callback( err, objects ) ;
    });
};



Portfolio.js:
function PortfolioBalance( ) {
    this.db = {
        LoginName: "",
        Date: "",
        CashBalance: 0,
        CashPortfolioPercent: 0,
        PreviousCashPortfolioPercent: 0,
        ReturnRate: 0.0,
        StocksPercent: [ { "StockId": 0,
                           "Percentage": { "Overall": 0, "Previous": 0, "Short": 0, "Buy": 0 },
                           "CompanyName": "",
                           "CurrentPrice": 0,
                           "Stock": "PM" }],
        StocksQuantity: "",
        TotalPortfolioValue: "",
        TradesForDay: ""
    };
   
    var parent = this;
}

exports.PortfolioBalance = PortfolioBalance;


Using Amazon DynamoDB inside of nodejs, entity frameworkish

Okay to all those wondering how do you integrate dynamodb with node and not lose your sanity I think I have the answer. I have a light background with entity framework and I'm not a huge fan with all of the routing that needs to get set up so here is my take on how to get going with minimal work and have a clean way of working with Dynamodb.

Goals of this design:
Keep plumbing/routing to a minimum
Write as little code as possible to get items from the database
Keep the class as clean as possible, datamembers for the class and functions for the class

Let's start with an example of a table with stock portfolio balances.

To start with create these classes -
1) database.js 
2) portfoliorepository.js
3) portfolio.js

Portfolio.js

First this is your class that does stuff! It has member variables, functions. What we want to do is add a special category for database variables. These can be treated as private if you like. They will represent what columns are in the database and will hold values to be written.

Ex:

function PortfolioBalance() {
    this.db = { 
        LoginName: "",
        Date: "",
        CashBalance: 0,
        CashPortfolioPercent: 0,
        PreviousCashPortfolioPercent: 0,
        ReturnRate: 0.0,
        StocksPercent: "",
        StocksQuantity: "",
        TotalPortfolioValue: "",
        TradesForDay: ""
    };
    //non database variables go here


    var parent = this;

    //class functions go here
}


PortfolioRepository.js 

Here is where stuff starts to get cool! The repository will only and should only be functions that query dynamoDB for data results. When the data gets returned we will use a function in database.js to create a list of 1 or more portfolio classes and return that. DO NOT add any other functions related to the class here! Also, do not create constructors for portfolio! As far as we are concerned, PortfolioBalance.db is the contract and all we need to populate portfolio balance. We can write a generic function in database.js to do this.


Ex:

var getPortfolioBalanceForUserOnDate = function ( loginName, date, callback ) {
    var search = {
        TableName: exports.table.Name,
        KeyConditions: {
            "LoginName": {
                "AttributeValueList": [{
                    "S": loginName
                }],
                "ComparisonOperator": "EQ"
            },

        },
        Limit: 1,
        ProjectionExpression: "#a,#b,#c,#d",
        ExpressionAttributeNames: { "#a": "StocksPercent",
                                    "#b": "CashPortfolioPercent",
                                    "#c" : "Date",
                                    "#d" : "PreviousCashPortfolioPercent"}

    };
   
    databaseLib.query( search, function( data, err ) {
        var objects = databaseLib.convertToObjects( data, err, function() { return new PortfolioBalance(); } );
        callback( objects, err ) ;
    });
};

Alright excitement level right now is 8/10. I will make that 10 on the next step. You can see this code deals with all your non-pretty dynamo db query stuff. That's all it should have and when it gets back a json result, that gets converted to a pretty list of PortfolioBalance() objects.

Database.js

Okay now we need some magic here to generate the classes.
For the convertToObjects( data, err, function() { return new PortfolioBalance(); } )

the code might be similar to follows ->


var initalizeObject = function ( object, dataItem ) {
    var itemsToFill = Object.keys(object.db);
   
    //this still needs to look for M/N/L and do the conversion!
    for ( var index = 0; index < itemsToFill.length; index++ ) {
        object.db[itemsToFill[index]] = dataItem[itemsToFill[index]];
    }
};

var convertToObjects = function( data, err, creationFunction ) {
    if ( err ) {
        return null;
    }   
   
    var objects = [];
   
    for ( var index = 0; index < data.Items.length; index++ ) {
        var newObject= creationFunction();                                  //create a new class object
        initalizeObject( newObject, data.Items[index] );     //initialize the class object based on the db member, and row from database
        objects.push( newObject);
    }
   
    return objects;
}; 



Some points here -> We are using object.db or portfoliobalance.db as a contract on what to grab from the query results. The code above does a straight copy meaning you will get myvalue: { n: 0 } or all the data type values amazon gives you. I am working on stripping these out in my database class inteligently so my class is as clean as possible. Also note, since object.db or portfolioBalance.db is a contract, we could also validate that the dataItem contains all expected keys! 


I may update this in the future with more code as I work on it, so hit me up in the comments if you are interested in more final code!