Creating a Facebook Messenger BOT with Laravel 5

Keeping in constant communication with your customers is a necessity for most businesses now days. Most companies do this via email and SMS messaging, but keeping in contact via Facebook Messenger can also be a great tool in increasing sales and customer satisfaction.

Even better, having a Facebook Messaging BOT to automate your customer’s initial incoming messages can cut down on costs, and help your company be more productive and efficient.

Over the last few months I was tasked with integrating Facebook’s Messenger API into a popular SaaS platform, so this tutorial will get your feet wet with the Messenger API in general, and then show you how to get started integrating the Messenger API with your company’s platform using Laravel 5.


** NOTE: For this tutorial, you will need a Facebook Developer account


First create a Facebook developer account (if you don’t already have one), then create an app, a page, and then add the messenger product.

This Quick Start Guide will guide you through the Facebook app procedure quickly.


Alright… Now let’s get started!


First we need to create our routes..
Create a route group in your routing file and add the following two routes.

app/Http/routes.php


Route::group(['prefix' => 'facebook'], function () {
    Route::get('message', 'FacebookController@verify');
    Route::post('message', 'FacebookController@message');
});

Next up, just like any Laravel route, we need to create our Facebook Controller with the following two methods that the above routes are referencing.

After our dependencies are injected in our constructor, we have our facebookVerify method. This method is used in order for our Facebook Messenger app’s webhook to verify that our route is alive and returning
a 200 response.


** NOTE: To understand why this method is needed, refer to the Facebook Messenger API docs.


Our second method, facebookMessage is the entry point for all incoming messages from our Facebook User.

First, we get our Request, process the object’s JSON and then pass our payload off to our message handler. Our message handler is located in in our Facebook Manager service.

Let’s take a look.

app/Http/Facebook/FacebookController.php


namespace App\Http\Controllers\Facebook;

use App\Http\Controllers\Controller;
use App\Services\Facebook\FacebookManagerIncomingMessageService;

use Illuminate\Http\Request;
use Illuminate\Http\Response;
use Illuminate\Routing\ResponseFactory;

class FacebookController extends Controller
{
    private $facebookManagerIncomingMessageService;
    private $responseFactory;

    public function __construct(
        FacebookManagerIncomingMessageService 
            $facebookManagerIncomingMessageService,
        ResponseFactory $responseFactory)
    {
      $this->facebookManagerIncomingMessageService = 
          $facebookManagerIncomingMessageService;
      $this->responseFactory = $responseFactory;
    }

    /**
     * @param Request $request
     * @return \Illuminate\Http\JsonResponse
     */
    public function facebookVerify(Request $request)
    {
        $verified = $this
            ->facebookManagerIncomingMessageService
            ->incomingMessageVerify(
                $request->hub_challenge, 
                  $request->hub_verify_token);

        if (!empty($verified)) {
            echo $verified;
        } else {
            return $this
                ->responseFactory
                ->json(Response::HTTP_FORBIDDEN);
        }
    }

    /**
     * @param Request $request
     * @return \Illuminate\Http\JsonResponse
     */
    public function facebookMessage(Request $request)
    {
        $message = $request->json()->all();

        try {
            $this
              ->facebookManagerIncomingMessageService
              ->incomingMessageHandler($message);
        } catch (\Exception $e) {
            return $e;
        }

        // we want to always send a 200 back 
        // to the messenger api - even with 
        // an issue on our side or messages 
        // start backing up and everything 
        // comes to a halt really quick
        return $this
            ->responseFactory
            ->json(Response::HTTP_OK);
    }
}

Now let’s take a look at the FacebookManagerIncomingMessageService.

This Service is used to process our incoming messages FROM the Facebook Messenger API, along with any handling (i.e. saving the message, creating a customer, etc).

Our entry point to our Facebook Manager service is the method incomingMessageHandler. This method contains a switch statement that is used to parse our JSON that is passed to it from the controller for the sender ID, the recipient ID, the actual message, and any other metadata that is needed for our application. For this tutorial all we care about is the sender and recipient IDS and the message. For more information on all the available fields reference the Facebook Messenger API docs.


Our next method setConversationParticipants is pretty self explanatory. It is used to set the IDs of both the sender and the recipient.


** NOTE: The sender ID and recipient ID reverse when we are parsing an ECHO message. To learn more about echo messages, and their purpose reference the Messenger API.


The next methods are our setFacebookPageByPageId and setFacebookUserBySenderId. These methods are used to set the recipient ID (non echo message), and sender ID (non echo message) in our class to use when needed. This let’s us know which message came from which party.


Code tracing through the code brings us to the createOrUpdateCustomer and getFacebookUserBasicInfo methods. These methods are used to check our database if our customer is already in our DB, and if so retrieve that model, if not we save this Facebook User as a new customer. Our Facebook User is referenced by the Facebook Sender ID that we parse out in the incoming JSON.

If this Facebook User is new and not a customer in our database, we then use our basic info method together with our Facebook User’s Sender ID to retrieve the user’s first name, last name, and avatar image from the Facebook Graph API.

Next is our setFacebookPageByPageId method. This method is how we keep track and reference the Facebook Page (your company’s page) that the Facebook User is talking to. In our Customer’s table we save the Facebook User’s senderID, and in our Page table, we save the Facebook Page’s page ID (usually the recipient ID if not an echo message).


OK… Enough talking… Let’s take a look at the actual code..

This is our FacebookManagerIncomingMessageService.

app/Services/Facebook/FacebookManagerIncomingMessageService.php


namespace App\Services\Facebook;

use App\Services\Facebook\FacebookManagerOutgoingMessageService;

use App\Contracts\Repositories\FacebookRepositoryInterface;
use App\Contracts\Repositories\CustomerRepositoryInterface;

use Illuminate\Contracts\Logging\Log;
use Illuminate\Database\Eloquent\ModelNotFoundException;

class FacebookManagerIncomingMessageService
{
    private $facebookManagerOutgoingMessageService;

    private $facebookPageRepository;
    private $customerRepository;
    
    private $facebookPage;
    private $facebookPageByPageId;

    private $customer;
    private $facebookUserBySenderId;
   
    private $message;

    private $log;

    public function __construct(
        FacebookManagerOutgoingMessageService $facebookManagerOutgoingMessageService,
        FacebookRepositoryInterface $facebookRepository,
        CustomerRepositoryInterface $customerRepository,
        Log $log
    ) {
        $this->facebookManagerOutgoingMessageService = $facebookManagerOutgoingMessageService;
        $this->facebookRepository = $facebookRepository;
        $this->customerRepository = $customerRepository;
        $this->log = $log;
    }

    /**
     * Verify the Facebook challenge.
     *
     * @param $challenge
     * @param $verifyToken
     * @return null
     */
    public function incomingMessageVerify($challenge, $verifyToken)
    {
        if ($verifyToken === $this
                ->facebookManagerOutgoingMessageService
                ->getVerificationToken()) {
            return $challenge;
        } else {
            return null;
        }
    }

    /**
     * Entry point for handling any incoming messages
     * from the Facebook Messenger Webhook
     *
     * @param array $message
     * @throws FacebookManagerIncomingMessageException
     */
    public function incomingMessageHandler(array $message)
    {
        $this->setMessage($message);

        if (!empty($message['object']) && 
          $message['object'] === 'page') {

          foreach ($message['entry'] as $pageEntry) {

            // iterate through each message and process
            foreach ($pageEntry['messaging'] as $message) {

             // set the correct sender & recipient
             $this->setConversationParticipants($message);

             switch ($message) {
                       
               // incoming facebook user message
               case (
                 !empty($message['message']) &&
                 empty($message['message']['is_echo']))):
                   $this->processFacebookUserMessage($message);
                   break;

               // incoming facebook echo message
               // echo messages are sent 
               // FROM 
               // the API when messages are sent 
               // TO
               // the API. Echo messages contain 
               // the message that was
               // (along with other metadata) 
               // to the API in order "echo" 
               // what was sent so you can 
               // process it (i.e. store the
               // message in your database)
               case (
                 !empty($message['message']) && 
                 !empty($message['message']['is_echo']):
                   $this->processFacebookPageMessage($message);
                   break;
                        
               default:
                   break;
            }
          }
        }
        return true;
    } else {
      throw new FacebookManagerIncomingMessageException(
        'Problem processing the incoming facebook message.');
    }
  }

   /**
    * Set our Facebook User's incoming message.
    * ** NOTE: Echo messages have the  
    * sender & recipient IDs reversed.
    */
    private function setMessage(
        $message)
    {
        $this->message = $message;
    }

    private function setConversationParticipants(
        $message)
    {
        // incoming facebook user visitor message
        if (empty($message['message']['is_echo'])) {
            $this->isEcho = false;
            $this->setFacebookPageByPageId($message['recipient']['id']);
            $this->setFacebookUserBySenderId($message['sender']['id']);
        } else { // this is an echo message - swap sender and recipient
            $this->isEcho = true;
            $this->setFacebookPageByPageId($message['sender']['id']);
            $this->setFacebookUserBySenderId($message['recipient']['id']);
        }
    }

    /**
     * Process the Facebook User's incoming message
     *
     * @param $message
     */
    private function processFacebookUserMessage($message)
    {
        return $this->saveMessageFromFacebookUser($message['message']);
    }
    
    /**
     * Check to see if there is a Facebook Page by this
     * Facebook Page ID. 
     * ** NOTE:This is the ID that is
     * associated with the Facebook Page. The Facebook
     * Page is the page that the user is sending messages to.
     *
     * @param $facebookPageId
     * @return mixed
     * @throws ModelNotFoundException
     */
    private function setFacebookPageByPageId($facebookPageId)
    {
  
        try {
            $facebookPage = $this->facebookPageRepository
                ->findByFacebookPageIdOrFail($facebookPageId);
        } catch (
            ModelNotFoundException $e) {
            throw $e;
        }

        return $facebookPage;
    }

    /**
     *  Check to see if there is an already existing
     *  customer for this Facebook customer
     *
     * @param $facebookSenderId
     * @return $this|null
     */
    private function setFacebookUserBySenderId(
        $facebookUserSenderId)
    {
        // check to see if we have a record of this customer already
        $this->customer = $this->customerRepository
            ->getFirstBy('facebook_sender_id', $facebookUserSenderId);

        // This Facebook user is not a customer yet
        // Create a new customer
        if (empty($this->customer)) {
            $this->customer = $this->createOrUpdateCustomer();
        }
    }

    /**
     * Create a new customer based on this Facebook User
     *
     * @return \App\Models\Customer
     */
    private function createOrUpdateCustomer()
    {
      $customerData = $this
        ->getFacebookUserBasicInfo();
       
      $customerData['first_name'] = 
        (!empty($customerFacebookData['first_name']))
          ? $customerFacebookData['first_name']
          : '';
      $customerData['last_name'] = 
        (!empty($customerFacebookData['last_name']))
          ? $customerFacebookData['last_name']
          : '';
      $customerData['avatar'] = 
        (!empty($customerFacebookData['profile_pic']))
          ? $customerFacebookData['profile_pic']
          : '';
      $customerData['facebook_sender_id'] = 
        (!empty($this->customerByFacebookSenderId))
          ? $this->customerByFacebookSenderId
          : '';

      if (empty($this->customer)) {
        $this->customer = $this
          ->customerRepository'
          ->create($customerData);
      } else {
        $this->customer->first_name = $customerData['first_name'];
        $this->customer->last_name = $customerData['last_name'];
        $this->customer->avatar = $customerData['avatar'];
      }

      $this->customer->save();

      return $this->customer;
  }

    /**
    * Use our Facebook Manager Outgoing Message
    * Service to send a message to the Facebook
    * Messenger API to query basic user info for
    * our Facebook User (first name, last name,
    * avatar, etc) and return this info so we
    * can use it to create our new customer.
    */
    private function getFacebookUserBasicInfo()
    {
        try {
            $response = $this
                ->facebookManagerOutgoingMessageService
                ->getFacebookUserBasicInfo(
                    $this->facebookUserSenderId, 
                    $this->facebookPage
                      ->facebook_page_access_token
                );

            $responseBody = $response->getBody();
            $facebookUserBasicInfo = json_decode($responseBody, true);

        } catch (\Exception $e) {
            $facebookUserBasicInfo = [];
        }

        return $facebookUserBasicInfo;
    }

   /**
    * Save or process the Facebook User's message
    */
    private function saveFacebookUserMessage($message) 
    {
      // ...
      // ...
      // Save or do something cool with your Facebook User's message.
      // ...
      // ...
    }

   /**
    * Save or process the Facebook Page's message
    */
    private function saveFacebookPageMessage($message) 
    {
      // ...
      // ...
      // Save or do something cool with the message that your Employee sent
      // FROM your Facebook Page to the 
      // Facebook User.
      // ...
      // ...
    }



** Note: This tutorial already assumes you have a Customer and FacebookPage Model created and that your database is in sync with them.



Now let’s take a look at our model retrieval method inside our Facebook Repository. This method is used to lookup our FacebookPage information (so the Facebook User (customer) is communicating with the correct Facebook Page (your employee)).

app/Repositories/EloquentFacebookRepository.php


namespace App\Repositories;

use App\Models\FacebookPage;

class EloquentFacebookRepository extends AbstractEloquentRepository
{
    // ...
    // ...

    public function findByFacebookPageIdOrFail($facebookPageId)
    {
        $facebookPage = $this
            ->model
            ->where(
                'facebook_page_id', $facebookPageId)
            ->first();
        if (is_null($facebookPage)) {
            $e = new ModelNotFoundException;
            $e->setModel(FacebookPage::class);
            throw $e;
        }

        return $location;
    }

    // ...
    // ...
}


Now that we know how to process incoming messages from our Facebook User (customer), let’s see how we can use our FacebookManagerOutgoingMessageService to send a message to the Facebook Messenger API from our Facebook Page (employee) and ultimately to our Facebook User (customer).

For HTTP requests we will be using Guzzle.


The first method getFacebookUserBasicInfo is how we query the Facebook Messenger API for basic Facebook User information. Reference the API docs to see all of the available information that you can retrieve with the Facebook User’s Sender ID.

The next method, our sendMessageToFacebookUser is used to send the actual message FROM our Facebook Page (employee) TO our Facebook User (customer). First, we retrieve our Page API key that is stored in our config file, our Page token (stored also in our FacebookPage table), and our Facebook User’s Sender ID, and then concatenate them along with the URI to send a message to our Facebook User.


Let’s take a look at the code…

app/Services/Facebook/FacebookManagerOutgoingMessageService.php


namespace App\Services\Facebook;

use App\Models\Customer;
use App\Models\;
use GuzzleHttp\Client as GuzzleClient;

class FacebookManagerOutgoingMessageService
{
 
    /** @var array facebook configuration */
    private $facebookConfig;
    private $guzzleClient;

    public function __construct(
        array $facebookConfig,
        GuzzleClient $guzzleClient)
    {
        $this->facebookConfig = $facebookConfig;
        $this->guzzleClient = $guzzleClient;
    }

    /**
     * Get the Facebook verification 
     * token from our config
     *
     * @return mixed
     */
    public function getVerificationToken()
    {
        return $this->facebookConfig['app_verification_token'];
    }

   /**
     * Get the Facebook User basic profile info
     *
     * @param $visitorByFacebookSenderId
     * @param $locationByFacebookPageAccessToken
     * @return mixed|\Psr\Http\Message\ResponseInterface
     * @throws Exception
     */
    public function getFacebookUserBasicInfo(
      $facebookUserBySenderId, $facebookPageAccessToken)
    {
        $apiUrl =
            $this->facebookConfig['graph_user_profile_api_uri'].
            $facebookUserBySenderId. '?access_token='.
            $facebookPageAccessToken;

        try {
            $response = $this
                ->guzzleClient
                ->request('GET', $apiUrl);
        } catch (\Exception $e) {
            throw $e;
        }

        return $response;
    }

    /**
     * Send the message to our the Facebook messenger API
     * and ultimately the Facebook User we're communicating
     * with.
     *
     * @param FacebookPage $facebookPage
     * @param array $requestArguments
     * @return mixed|\Psr\Http\Message\ResponseInterface|void
     * @throws Exception
     */
    private function sendMessageToFacebookUser(
        FacebookPage $facebookPage, array $requestArguments)
    {
        $apiUrl =
            $this->facebookConfig['graph_messenger_api_uri'].
            $facebookPage->facebook_page_access_token;

        try {
            return $this->guzzleClient->request('POST', $apiUrl, $requestArguments);
        } catch (\Exception $e) {
            throw $e;
        }
    }


And finally a quick look at your Facebook Configuration. Add the following (update with your values) the config tokens in your .env file.


// ...
// ...
// ...
FACEBOOK_APP_VERIFICATION_TOKEN = my_voice_is_my_password_verify_me
FACEBOOK_GRAPH_MESSENGER_API_URI = https://graph.facebook.com/v2.8/me/messages?access_token=
// ...
// ...
// ...


Now the only value that will be pulled from your database will be the Page Access Token, the Page ID, for your Facebook Page, and the Facebook User’s Sender ID for your Customer.


And that should be it! This was a long tutorial, so if you see any mistakes or have any questions on something that’s not clear and confusing… Just ping me! I’ll be glad to help.. 😀


Have fun coding! 😀

Adding Symfony 2 Routing to your legacy PHP application

Recently I’ve had to rewrite portions of a PHP legacy application that was written in a company’s proprietary legacy framework, in order for it to be test friendly and much more maintainable.

In order to accomplish this, I’ve added a few key Symfony components to their framework. The result was that the application was easier to test, easier to cache, and much much easier to maintain than it was previously.


Let’s take a look.


Their front controller was stripped and completely rewritten to implement Symfony’s HTTP Foundation and Symfony’s Routing.

Let’s take a look at a snippet of the front controller.
New Front Controller


include '../vendor/autoload.php';

use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\Routing\RouteCollection;
use LtApp\LtAppRoute;
use LtApp\LtAppKernel;

$routeCollection = new RouteCollection();
// inject the Symfony 
// route collection to our 
// App Route class (shown below)
$ltAppRoute = new LtAppRoute($routeCollection); 
$routes = $ltAppRoute->getRoutes();

// using a dependency injection container
// will be covered in another tutorial
$diContainer = include __DIR__.'/../app/LtAppContainer.php';

// create our Request object
$request = Request::createFromGlobals();

// Retrieving the LtAppKernel 
// service from my DI container.
// If you're not using a DI container, 
// you can just instantiate your version 
// of AppKernel (shown below) in place of this.
// Our response object is then returned 
// to the client.
$response = $diContainer
    ->get('ltAppKernel')
    ->handle($request);

// enable caching for our response
//$response = $diContainer
    ->get('ltAppKernelWithCache')
    ->handle($request);

// phone home
$response->send();

exit;


Now let’s take a look at our App Kernel.

Our App Kernel is the engine that will dynamically instantiate our application’s Routing and Controller objects. It is responsible for asking our matcher if the incoming Request object contains a URL string with a valid route, and if so, resolving the correct controller for that valid route.

Let’s take a look at a snippet of our App Kernel.

App Kernel


namespace LtApp;

use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\HttpKernelInterface;
use Symfony\Component\Routing\Matcher\UrlMatcherInterface;
use Symfony\Component\Routing\Exception\ResourceNotFoundException;
use Symfony\Component\HttpKernel\Controller\ControllerResolverInterface;

class LtAppKernel implements HttpKernelInterface
{
    private $matcher;
    private $resolver;
    private $dispatcher;

    // Injecting in our Url Matcher
    // and Controller Resolver.
    // Again this injected from my DI container. 
    // If you aren't using a DI container, 
    // just pass the interface objects as 
    // arguments when you instantiate 
    // your App Kernel from inside
    // your front controller.
    public function __construct
    (UrlMatcherInterface $matcher,
     ControllerResolverInterface $resolver)
    {
        $this->matcher = $matcher;
        $this->resolver = $resolver;
    }
    
    // Our handle method takes an 
    // HTTP Request object, 
    // the Symfony Http Kernel
    // so we have access 
    // to the master request, 
    // and a catch flag 
    // (we'll see why below)
    public function handle(
        Request $request, 
        $type = HttpKernelInterface::MASTER_REQUEST, 
        $catch = true)
    {
        // Next we take our HTTP request object and 
        // see if our Request contains a 
        // routing match (see our routes class 
        // below for a match)
        try {
            $request
            ->attributes
            ->add(
              $this
                ->matcher
                ->match(
                  $request
                    ->getPathInfo()
            ));

            // Our request found a match 
            // so let's use the Controller
            // Resolver to resolve our 
            // controller.
            $controller = $this
                           ->resolver
                           ->getController($request);
            
            // Pass our request arguments 
            // as an argument to our resolved 
            // controller (see controller below).
            // If you have form data, the resolver's
            // 'getArguments' method's functionality
            // will parse that data for you and 
            // then pass it as an array to your 
            // controller.
            $arguments = $this
                          ->resolver
                          ->getArguments(
                             $request, $controller);

            // Invoke the name 
            // of the controller that
            // is resolved from a match
            // in our routing class
            $response = call_user_func_array(
                          $controller, $arguments);

        } catch (ResourceNotFoundException $e) {
            // No such route exception 
            // return a 404 response
            $response = new Response(
                'Not Found', 404);
        } catch (\Exception $e) {
            // Something blew up exception
            // return a 500 response
            $response = new Response(
                'An error occurred', 500);
        }
        
        //**Note:
        // If you need any event listeners for specific
        // actions to take on your response object 
        // after returned from your business logic 
        // / controller and before you return 
        // to the client you can invoke them here.
        
        return $response;
    }
}

Now let’s take a look at the routing class that our App Kernel was
calling.
Routing


namespace LtApp;

use Symfony\Component\Routing\RouteCollection;
use Symfony\Component\Routing\Route;

class LtAppRoute
{
    private $routes;

    // pass in Symfony's 
    // Route Collection object
    public function __construct(
      RouteCollection $routeCollection)
    {
        $this->routes = $routeCollection;
        // ignore routeConfig
        // this I'm using for
        // something else
        $this->routeConfig(); // decouple later
    }

   
   // We register a route by
   // invoking the add method
   // with specific arguments.
   // The first argument is a 
   // unique name for this route.
   // Next we instantiate a Route
   // object with our route.
   // Our route is /docs/{ID}
   // Our associative array contains 
   // the controller that we
   // want to resolve with this route,
   // DocumentController and the 
   // method on that controller
   // that this particular route 
   // invokes viewDocumentAction.
   // separated by the :: 
   // token.
   // We can also pass in arguments
   // that contain a default value
   // if needed, and they will be
   // available in our method's 
   // parameters.
    private function routeConfig()
    {
     $this->routes
     ->add(
      'document_view', 
      new Route('/docs/{documentId}', 
      array(
      '_controller'=>
        'Lt\\DocumentPackage
           \\Controller
           \\DocumentController::
             viewDocumentAction',
      'action' => 
        'documents', // default value
      'documentId' => 
        '1' // default value
     )));
}


DocumentController (our resolved controller)


class DocumentController
{
  // ...
  // ...
  public function viewDocumentAction(
    Request $request, 
            $action, 
            $documentId)
  {
     // ... Grab services and models and
     // ... perform cool business logic
     $response = new Response();
     
     // return response object
     // back to App Kernel
     return $response;
  }
}


And there you have it. As you can see, this design decouples a lot of functionality from the application, which in this case, makes writing tests actually possible, where before automated testing wasn’t really possible.

And.. If you combine this with a Dependency Injection container, the result is even more flexibility and a code base that is much more maintainable.

Have fun coding! 😀

Supercharged PHP development with Vagrant, VMware, and IntelliJ IDEA

In my development environment I’ve been using Vagrant with Virutalbox for a while now; however, even with NFS, with the VM properly provisioned with enough resources, and running under Linux, the environment’s performance was still just barely tolerable. It’s performance compared to Ubuntu natively on a Samsung 850 pro SSD left me wishing for more.. A lot more.. 🙂


This tutorial will show you how to create a high performance PHP (or any language really) development environment using the JetBrains IntelliJ Idea IDE (with the PHP plugin), Vagrant 1.8.1 (latest version as of this post), VMware Workstation Pro 12, and good ole Ubuntu Trusty (14.04.3 with the vivid HWE stack). 😀 We will be using the vivid 3.19.43 kernel.


We also will be using native PHP 5.5.9 (but php-fpm, PHP 7, or HHVM would be even more awesome 😀 ) and XDebug as our remote debugger.


**Note: This tutorial assumes you have Vagrant, VMware, and Jetbrains IntelliJ Idea already installed and functioning on your host OS. It also assumes you already know how to install a guest OS in VMware.


This tutorial will be structured:

  1. First update/install all necessary Ubuntu packages in the VM.
  2. Configure our VM according to Vagrant standards.
  3. Configure Xdebug for Apache (although Nginx is a similar process).
  4. Turn our newly configured Ubuntu Virtual Machine into our own Vagrant base box with VMware as the provider.
  5. Configure our JetBrains IntelliJ Idea IDE (PHPstorm will work also) to work with our remote PHP interpreter and remote Xdebug debugger running in the VM.
  6. Lastly we will benchmark our VM development environment’s performance using wrk.

We have a lot to do… So… Let’s get started! 😀


**Note: This tutorial will use VMware Workstation 12 (but other VMware versions will probably work as well).


**Note: This tutorial also assumes you have a valid Vagrant VMware plugin license in order to interface Vagrant with VMware Workstation.


First do a minimal install of Ubuntu Trusty as a guest OS Virtual Machine using VMware Workstation.

If you don’t know how to do that, there are many tutorials available.

Pick one, install Ubuntu 14.04, and then come back. 🙂


**Note: When you install Ubuntu, since we’re creating a Vagrant base box, it is usually best to follow the Vagrant standard.

If you plan to share your base box with the public, using the Vagrant insecure settings is the standard way to configure it. This means the user should be vagrant with password vagrant, and the root password should be set to vagrant as well.

However, if this base box is only for your team and you don’t want it to be shared with anyone outside of your organization, then obviously the standard settings should be set to something much more secure.



OK….. So….. 🙂

At this point you should now have a minimal install of Ubuntu 14.04.3 (14.04.2 is fine also).


We’re now going to update the system, install the Ubuntu HWE stack, and then install all necessary packages for our LAMP stack and Vagrant Base Box.

Update repos and install/upgrade Packages:

vagrant@trusty:~$ sudo apt-get update

vagrant@trusty:~$ sudo apt-get install --install-recommends

vagrant@trusty:~$ sudo apt-get install build-essential libssl-dev

Upgrade Trusty to the LTS Enablement Stack:


// upgrade for better hardware support
// assumes using packages that are Multi-Architecture (both 32 and 64 bit)
vagrant@trusty:~$ sudo apt-get install --install-recommends linux-generic-lts-vivid xserver-xorg-core-lts-vivid xserver-xorg-lts-vivid xserver-xorg-video-all-lts-vivid xserver-xorg-input-all-lts-vivid libwayland-egl1-mesa-lts-vivid libgl1-mesa-glx-lts-vivid libgl1-mesa-glx-lts-vivid:i386 libglapi-mesa-lts-vivid:i386
 

At this point, if everything went well, you should be on the 3.19.0-43-vivid kernel.

Let’s check…

Check the kernel:

vagrant@trusty:~$ uname -r

The output should be…

Output:

vagrant@trusty:~$ 3.19.0-43-generic

Next, let’s install our LAMP stack.

Install the LAMP stack


vagrant@trusty:~$ sudo apt-get install apache2

vagrant@trusty:~$ sudo apt-get install mysql-server php5-mysql

// install php and any other php modules you need
vagrant@trusty:~$ sudo apt-get install php5-common php5-dev php5-cgi php5-cli php5-curl php5-xdebug php5-dbg php5-gd php5-mcrypt php5-oauth php5-memcache php5-redis php5-mongo libapache2-mod-php5


Next let’s make sure our NFS client is installed since we want Vagrant to spin up our VM using NFS (just like in Virtual Box) for greater performance.

Install NFS Client

vagrant@trusty:~$ sudo apt-get install nfs-common

Next, make sure you install any other packages you need in your stack.

Maybe you need Node? Mongo? Full MEAN stack? Redis? Java? Python? Ruby? Puppet/Chef?

Install any language, RDBMS, NoSQL DB, framework, or any platform that you need for you tech stack. Since our base box will be our template VM, we don’t want to have to install anything after it is created.

Lastly let’s install wrk so we can benchmark our HTTP requests. But first, let’s make sure git is installed and if it isn’t, we want to install it.

Install Git

vagrant@trusty:~$ sudo apt-get install git

Now let’s install wrk.

Install Wrk


vagrant@trusty:~$ cd /opt

// clone the github repo
vagrant@trusty:/opt$ git clone https://github/wg/wrk.git

vagrant@trusty:/opt/wrk$ cd ./wrk

vagrant@trusty:/opt/wrk$ make

// lastly add the exe to our env var PATH
vagrant@trusty:~$ cp wrk /usr/local/bin


Now we want to start configuring everything…

Let’s first configure our VM according to the Vagrant Base Box standard.

Since we’ve already created a user with a user/password combo as vagrant/vagrant and root has the password vagrant, the first step is done.


Next, we don’t want to be asked for a password when using sudo.

Let’s fix that.

Password-less Sudo


vagrant@trusty:~$ visudo

// append to the end and comment out any line containing "requiretty"
vagrant ALL=(ALL) NOPASSWD: ALL


Now we want SSH to be fast, so let’s make sure “UseDNS” is set to “no”

Check UseDNS


vagrant@trusty:~$ cd /etc/ssh

vagrant@trusty:/etc/ssh$ sudo vim ssh_config

// append "UseDNS no" to the file if its not already contained
UseDNS no


Next up, let’s create the insecure public key the Vagrant will look for so Vagrant can log us in with our key instead.

Set our insecure public key for the user Vagrant


// make sure we're in home
vagrant@trusty:~$ cd ~

vagrant@trusty:~$ ls -la

// create .ssh directory if .ssh does not exist in /home/vagrant
vagrant@trusty:~$ mkdir .ssh

// change .ssh to permission 0700
vagrant@trusty:~$ sudo chmod -rf 0700 .ssh

// create the insecure vagrant public key
vagrant@trusty:~$ cd .ssh

vagrant@trusty:~/.ssh$ vim authorized_keys

// paste into our authorized_keys file
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key


Now we need to make sure our authorized_keys file has the correct permissions.

Set Permissions


vagrant@trusty:~/.ssh$ sudo chmod 0600 ./authorized_keys


Next, up let’s enable and configure Xdebug and while we’re at it, let’s also enable mod_rewrite.

Enable Rewrite Module


// enable mod_rewrite
vagrant@trusty:~$ sudo a2enmod rewrite


Now let’s configure Xdebug.

Configure XDebug Module


vagrant@trusty:~$ cd /etc/php5/apache2/conf.d


**Note: If the symlink for xdebug.ini isn’t in conf.d, symlink the config file xdebug.ini from /etc/php5/mods-available/20-xdebug.ini to /etc/php5/apache2/conf.d/20-xdebug.ini



// configure xdebug
vagrant@trusty:/etc/php5/apache2/conf.d$ sudo vim 20-xdebug.ini

// add to xdebug.ini
[XDebug]
zend_extension=location_to_whatever_version_you_are_using.so
xdebug.remote_enable=1
xdebug.remote_host=192.168.33.1
xdebug.remote_port=9009
xdebug.remote_log="/var/log/debug/xdebug.log"


Next up, let’s configure a Vhost to be used as a template. Our Vhost will point to our NFS shared directory between our host and guest OS.

Create Vhost template


vagrant@trusty:~$ cd /etc/apache2/sites-available

vagrant@trusty:/etc/apache2/sites-available$ sudo vim vagrant-dev.conf

 
// Add these contents
< VirtualHost *:80>
    ServerName vagrant.dev
    DocumentRoot /var/www/html
    ErrorLog ${APACHE_LOG_DIR}/vagrant_error.log
    CustomLog ${APACHE_LOG_DIR}/vagrant_access.log common

    # uncomment if using php-fpm
    #ProxyPassMatch ^/(.+\.(php)(/.*)?)$ fcgi://127.0.0.1:9000/var/www/html/$1

    # don't use these settings in prod
    < Directory /var/www/html>
        AllowOverride all
        Options MultiViews Indexes FollowSymlinks
        Require all granted

        # not necessary if elsewhere or in .htaccess
        < IfModule mod_rewrite.c>
            RewriteEngine On
        < /IfModule>
    < /Directory>
< /VirtualHost>

Now we want to enable our vagrant-dev.conf vhost.

vagrant@trusty:~$ sudo a2ensite vagrant-dev.conf

Now let’s symlink our “html” directory to our soon to be Vagrant’s default shared directory.


**Note: You can point the vhost’s document root directly to the shared directory also.


Symlink to Shared Directory


vagrant@trusty:/etc/apache2/sites-available$ cd /var/www

vagrant@trusty:/var/www$ sudo ln -s /vagrant_data html


Next, since we don’t want to run a DNS server for a development environment, let’s manually point to this vhost.

Edit /etc/hosts


vagrant@trusty:/var/www$ cd /etc
vagrant@trusty:/etc$sudo vim hosts

// Add to your hosts file
127.0.0.1 vagrant.dev


That should be it for our VM configuration!

Now shutdown the VM, and let’s turn it into a base box with VMware Workstation as the provider.


Now we’re back on our host OS.

Navigate on the host OS, to the directory where your Virtual Machine is stored.

Navigate to Base box directory


dlaroche@laptop-dev:~$ cd baseboxes/Ubuntu-trusty

There should be files with extensions .nvram, .vmdk, .vmsd, .vmx, .vmxf, and some log files.

We need to first create a metadata.json file that Vagrant will read when creating the base box. This file can be detailed, containing provider and versioning info, but we are only going to use the bare minimum for this tutorial. You can learn more on the Vagrant site for more info on base box meta data, since this probably doesn’t meet your organization’s needs.

Create Vagrant metadata.json


dlaroche@laptop-dev:~/baseboxes/Ubuntu-trusty$sudo vim metadata.json

// Add to the file
{
  "provider": "vmware_workstation"
}


Now lets finally create the base box! 😀

We want to tar all of the files in the VM directory into a .box archive.

Create VMware Base Box


dlaroche@laptop-dev:~/baseboxes/Ubuntu-trusty$ tar cvzf ubuntu-trusty.box ./*


Now that we have our VM as a base box, let’s now add it to Vagrant.

Add Base Box to Vagrant


dlaroche@laptop-dev:~/baseboxes/Ubuntu-trusty$vagrant box add our-organization/ubuntu-trusty ubuntu-trusty.box


Now let’s take a look at our list of base boxes.
List Base Boxes added to Vagrant


dlaroche@laptop-dev:~/baseboxes/Ubuntu-trusty$vagrant box list


Now let’s configure our Vagrant init file, by going to the root directory of our code.

Create Vagrantfile


dlaroche@laptop-dev:~$cd workspace/test

dlaroche@laptop-dev:~/workspace/test$vagrant init 


Now add to your code’s Vagrantfile inside “Vagrant.configure” the following.

Add to Vagrantfile


config.vm.box = "larochetech/ubuntu-14.04.3_64"
config.ssh.insert_key = false
config.vm.provider "vmware_workstation"

config.vm.network "forwarded_port", guest: 80, host: 8080

// we configured xdebug to communicate to this network
config.vm.network "private_network", ip: "192.168.33.10"

// use nfs for greater performance
config.vm.synced_folder ".", "/vagrant_data", nfs: true

// Give the vm all of your cores and half of your ram.
// Setting the gui to true allows us to see the VM running
// in VMware. If you don't care about that, you can leave 
// it out.
config.vm.provider "vmware_workstation" do |v|
    v.gui = true
    v.memory = 8192
    v.cpus = 4
end


Now let’s edit our host OS hosts file to point to Vagrant.

Edit /etc/hosts


dlaroche@laptop-dev:~/workspace/test$ sudo vim /etc/hosts

// Add to your hosts file the private network of your VM
192.168.33.10 vagrant.dev


Now let’s spin up our VM using vagrant! 😀

Start VM through Vagrant


dlaroche@laptop-dev:~/workspace/test$ vagrant up

Since this is the first time the VM is spun up, Vagrant should provision it according to the VagrantFile settings, and if everything goes according to plan, you should have the VM running in VMware! 😀


Now that the VM is running under Vagrant… Finally… We’re ready to configure our IDE! 😀


Launch JetBrains IntelliJ Idea and install the PHP and Vagrant Plugins. You can search for them under “Settings -> Plugins”.

After they’re installed, restart the IDE and go to “Settings -> Languages & Frameworks -> PHP -> Servers” and create a new server.

For the “host” add “vagrant.dev”

Next, click “Use path mappings” and for “File/Directory” set it to the root directory for your code base (on your host OS).

For “Absolute path on the server”, set it to the shared Vagrant directory “/vagrant_data”.

Now click “Apply”.


Now go to “Settings -> Languages & Frameworks -> PHP” and click the “…” and create a new “Remote Interpreter”.

In the Vagrant plugin tab, enter the SSH credentials for the user “vagrant” we created in our VM, and under “PHP executable”, click the “Reload/Refresh” button.

This should search for the remote PHP interpreter running inside the VM and load it. After it is found click “Apply” to save and exit.


Next, we need to configure XDebug in the IDE. Go to “Settings -> Languages & Frameworks -> Debug” and make sure “Can accept external connections” is checked, and change the “Debug port” to “9009” (which is what we set Xdebug to inside the VM).

Click “Apply” to save.


That should be it! 😀

Now go to vagrant.dev

If everything is correct, you should be running your code through the VM’s PHP interpreter and XDebug should be communicating back to your IDE from the VM on port 9009. 😀


Now all thats left to do is to test our NFS performance in the VM with http requests using wrk.



Benchmark NFS Performance


dlaroche@laptop-dev:~/workspace/test$ vagrant ssh

// run a 60 second test
vagrant@trusty:~$ wrk -d60s http://vagrant.dev


Output – Vagrant NFS with VMware Provider:


Running 1m test @ http://vagrant.dev
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
  Latency    11.31ms   17.66ms 569.96ms   91.76%
  Req/Sec    650.24    245.14     1.01k    75.72%
  
77209 requests in 1.00m, 0.97GB read
Requests/sec:  1286.56
Transfer/sec:  16.49MB


Output – Native Ubuntu:


Running 1m test @ http://test.dev
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
  Latency    11.20ms   21.44ms 426.39ms   92.76%
  Req/Sec   786.50    374.96    2.00k    60.73%

93149 requests in 1.00m, 1.44GB read
Requests/sec:   1550.07
Transfer/sec:   24.60MB


And that’s it! Everything should be working..

If you run into any problems, either comment below, or ping me at info@daryllaroche.com

Happy coding! 😀

The Node.js Concurrency Paradigm: Asynchronous I/O

Node.js relies on asynchronous Input and Output (I/O) in order to increase the computing efficiency and speed of an application. This article will take a basic look at what asynchronous programming actually means, and why asynchronous I/O is important in a language like Node.js.


For Node.js performance to be actualized, a certain programming paradigm must be followed. Node.js is an event-driven architecture, and relies on asynchronous (non-blocking) input/output calls, along with the Node.js event loop to gain performance. This means that calls made in Node.js to the file system, event loop, or any resource, must be done asynchronously.


Asynchronous programming, informally, means code that is not waiting for a call to return. This is done using callbacks and registering them with an event-listener.


Let’s take a look at the difference between a simple asynchronous (non-blocking) call versus a synchronous (blocking) call.


Simple Synchronous Example:


// synchronous (blocking)
var request = prepareData();

// twiddle thumbs waiting for response
// from remote web service
var response = getRemoteData(request);

// display results
console.log(response);


First, notice the getRemoteData(request) call. If we’re on the client-side, and we make a synchronous call, the user’s browser is in a waiting state and frozen until the call is completed. The rest of the code cannot be executed until this call has returned.


If we’re on the server-side, and in Node.js we have an incoming request added to the event loop, but this request is a synchronous call, then we have a problem. If this synchronous call is to read or write to the file system, when executed, the entire event queue will be blocked and in a waiting state. No other calls in the event queue can be performed, because the server is halted and waiting for the blocking call to finish.

The speed and through-put of our application ultimately depends on how fast each synchronous call is completed. Since Node.js is single threaded the entire application will be blocked and waiting, which is obviously very bad.


Now let’s take a look at the same simple calls, but this time performed asynchronously.


Simple Asynchronous Example:


// Asynchronous (non-blocking)
var request = prepareData();

// call getRemoteData and continue to 
// execute next line of code
getRemoteData(request, function(response){ // pass a callback
  // display results when done
  console.log(response);
};


We begin here again by passing the request to getRemoteData(request, callback), but we also pass a callback as the second parameter. Remember we can do this because JavaScript supports functions as “first class citizens”.

Our inline anonymous callback function will fire, and the response will be passed as a dynamic argument when getRemoteData() returns.

What is important is our code continues to execute the line after getRemoteData() immediately. It does not wait for the call response to return. This is referred to as an asynchronous (non-blocking) call.


Programming in a purely asynchronous fashion is what makes Node.js so powerful.


Now, it is true that many mainstream languages support concurrency. And it is also true, that they also have asynchronous I/O support. However, the problem is that the majority of these languages have only concurrency and asynchronous “features”. In most languages, concurrency support was added long after the language’s release.


Node.js nearly only supports asynchronous programming and was developed purely with the asynchronous paradigm as a fore-front of its design.


And that’s it!
Happy asynch coding and JavaScript all the things!
😀

The Responsibility of the Node.js Event Loop

Everyone always talks about the Node.js event loop and how its important, but what exactly is it responsible for?

Well… Let’s take a look! 😀

The event loop (event queue) in Node.js is a loop that is responsible for processesing incoming requests. This loop “contains” a queue of requests, and each request in the event loop is a callback.


Node.js Event Loop

The Node.js Event Loop

Since the Node.js event loop contains a queue, it processes the requests like a traditional queue First-In-First-Out (FIFO) fashion. Node.js will continue processing each callback in the event loop until there are no more callbacks left to process.


In order to do this, Node.js registers with the Operating System. When an request arrives, Node.js is alerted, and the event loop processes the request. Any other subsequent connections that are made are queued until Node.js has finished with the current request.


The event loop allows Node.js to scale efficiently by accepting (queueing) many requests at once. Efficient scaling is one of the benefits that Node.js (with the event loop) has over traditional web servers. The event loop is single-threaded. This allows Node.js (when used properly) to require far less computational resources than traditional web servers.

Traditional web servers like Apache or Nginx are multi-threaded. Multi-threaded web servers typically cannot scale to the magnitude that Node.js can without greatly increasing the system’s computing resources. Since multi-threaded web servers create a new thread for each incoming request, this increases both CPU and RAM requirements of the system for the same amount of traffic. This increase can be either positive or negative (depending on your application), but the amount of computing resources for multi-threaded web servers will be greater than the requirement for single-threaded ones.


And that’s it on the Node.js event loop for now!

Happy coding! 😀

Review of JavaScript Invocation Patterns

JavaScript Invocation patterns can be tricky for developers new to the language. This tutorial will take a look and review the four different invocation patterns that JavaScript offers, and the behavior that is expected when using them.

A function invocation in JavaScript refers to passing control and parameters to another function during execution. This may not seem very interesting at first, but the functional ability that the developer can gain can be powerful.

If you have programmed in other languages, especially in many of the popular Object Oriented languages that contain the this parameter, you will quickly find that in JavaScript, the context execution of a function can change the functionality of a function in ways that are typically not expected.

Let’s dive into this in detail. Each function in JavaScript contains two implicit parameters, the this parameter that you use to call object properties with, and another parameter called the arguments parameter.

Along with these two implicit parameters, JavaScript also contains four different invocation patterns.


Let’s take a look.


The Four JavaScript Invocation Patterns:

  1. Constructor Invocation
  2. Function Invocation
  3. Method Invocation
  4. Apply Invocation


**Note: Each invocation pattern in JavaScript changes the initialization context of the this parameter.


Let’s take a look at how each pattern works.


The Constructor Invocation Pattern


Again, JavaScript is classless, prototypal inheritance language, but some of the syntax JavaScript uses camouflages its true nature. Some of JavaScript’s syntax makes the language appear to have classical inheritance, but it doesn’t.

When a function object in JavaScript is invoked with the new keyword, then that object is created with a “hidden link” to that function’s prototype member, and the this parameter will be bound to that new object.


Let’s take a look at an example.


Constructor Invocation Example:


// create a constructor function
// called HelloName
var HelloName = function(name) {
  this.say = name
};
// Give all instances of HelloName
// a public method called getSay
HelloName.prototype.getSay = function() {
  return this.say;
};
// create an instance of HelloName
var helloNameObj = new HelloName("Daryl");
// hello Daryl
console.log('hello' + helloNameObj.getSay());

When functions in JavaScript are created with the new keyword, they are called constructors. We add functionality to constructor functions by adding properties to the functions prototype.


**Note: By convention, the first letter of a constructor function is always capitalized.


Constructor functions in JavaScript can be very useful; however, using the “constructor style” method for creating functions is not recommended. JavaScript contains a better method built into the language, that accomplishes the same behavior. This method is to use Object.create, and we will look at why this is a better option later in another tutorial.


The Function Invocation Pattern


A function is not a property of an object when the function is invoked using the Function Invocation Pattern. If the function is invoked with this pattern, the this parameter becomes bound to the global object, and this can become a problem.


**Note: This was considered a design flaw when the language was developed, because binding to the global object is bad. We won’t go into complete detail of why it is bad in this tutorial, but we will look at a commonly used example of why. There are a lot of reasons, so just trust me (or Google it), binding globally is a nightmare waiting to happen.. 😛


OK, let’s first take a look at the Function Invocation Pattern, and then since binding to the global object is bad, we’ll take a look at how a standardized work around is implemented when a function is invoked inside a method.

Function Invocation Example:


// invoked as a function
var addTogether = add(1, 2); // 3
// using our helloNameObj object
// modify helloNameObj to contain a
// moneyMakerAlgorithm method
var helloNameObj.moneyMakerAlgorithm = function() { 
  var that = this; // work around
  var extraMathProcessing = function() {
    // "that" holds the correct "this" value now
    that.value = add(that.value, that.value);
  }
  // invoke as a function
  extraMathProcessing(); // "this" is now bound to global object
}
// invoke moneyMakerAlgorithm as a method
helloNameObj.moneyMakerAlgorithm();
// output result of value
console.log(helloNameObj.value);


The Method Invocation Pattern


As mentioned, when a function is stored as a property of an object, that property is called a method. When a method is invoked, the this parameter is bound to that method, and that method is a function object.


Let’s take a look at an example.


Method Invocation Example:


// create an object literal with 
// a method. The input param is
// optional, but there should be
// validation checking for if input
// is actually of data type number
var helloNameObj = { 
  value: 1,
  increase: function(num){
    this.value += num;
  }
};
helloNameObj.increase(1); // value = 2
helloNameObj.increase(3); // value = 5


The Apply Invocation Pattern


Since JavaScript supports a functional programming paradigm, then functions can be “first-class citizens”, and since functions are objects, then functions can contain other functions, called methods.

The implications of these features are that JavaScript contains an “Apply Invocation pattern”. This pattern, enables the creation of an array of arguments that can be invoked on a function directly. When doing this, we can apply the context of the this parameter to the context of our choosing.


Let’s take a look at some examples.


Apply Invocation Example:


// apply takes two parameters
// first - is the this parameter
// second - is the array of arguments
// to apply on the function
var helloNameArr = [2, 3];
var addTogether = add.apply(null, helloNameArr); // 5

Apply Invocation Example:


// create an object literal with a 
// say property like in the constructor
// invocation pattern
var helloNameObj {
  say: 'hello Daryl'
};
// Unlike above in the constructor
// invocation pattern, helloNameObj
// does not inherit from 
// HelloName.prototype, so
// we do not have a getSay method
// on helloNameObj, but we can
// invoke getSay on helloNameObj
// by using the apply keyword again
var say = HelloName.prototype.getSay.apply(helloNameObj);
// hello Daryl



Now that we went through all the invocation patterns, let’s take a look at an interesting array parameter that JavaScript makes available.


JavaScript Arguments Parameter



JavaScript contains an array parameter, that is implicitly available to functions when they are invoked. The arguments array allows functions to be invoked with extra arguments. This includes ones that are not already defined as the functions parameters. The implications of this, is that functions can be invoked with an unspecified number of arguments.

Let’s take a look.

Arguments Parameter Example:


// invoked as a function
var addStuff = function() {
  var result=0; 
  for(var i=0; i < arguments.length; i++) {
    result += arguments[i];
  }
  return result;
}
console.log(addStuff(1,2,3); // 6


And that's it for the JavaScript invocation review.
Happy coding 😀


Proxy Node.js Application Requests through Apache

Running a Node.js application along side other non node web applications can work well if configured correctly. This tutorial will show you how to easily configure a Virtual Host to proxy Node.js application requests through Apache.


First, we need to make sure that we have Node.js running on a port other than the port Apache is running on (usually port 80). Our goal is to run the Node.js server on a port other than 80, and then use Apache to proxy incoming Node.js application requests to the port that the Node.js server is running on.


First, we need to make sure that Apache has the mod_proxy.so and mod_proxy_http.so modules loaded. If they are not loaded, go uncomment them in the Apache configuration file, and then restart the server.

Next, we need to configure an Apache Virtual Host just like we would any other VHost, except we want to proxy all domain requests for your Node.js application to the Node.js server.


**Note: The process is also the same if only a portion of an application is using Node.js.


Let’s take a look.

Apache Virtual Host for the Node.js Application



    ServerName www.yoursite.com
    ServerAlias yoursite.com
    DocumentRoot /your/site/location
    ErrorLog logs/yoursite.com-error_log
    CustomLog logs/yoursite.com-access_log common

    ProxyRequests off
    
        Order deny,allow
        Allow from all
    
    
        ProxyPass http://127.0.0.1:3000/
        ProxyPassReverse http://127.0.0.1:3000/
    




**Note: This configuration assumes that the node.js server is already running locally on port 3000, but it could be configured to use any unused port and/or an IP, that points to a system that is not the same as the one that Apache is running on.


Next, we need to tell Apache to proxy all node.js requests to node.js. This is done in our location directive with ProxyPass. All domain requests for www.yoursite.com will proxy to the node.js server running locally on port 3000.

Our ProxyPassReverse is used if/when our application performs a redirect, however its usually good practice to add even if your application isn’t redirecting.

For example, if our Virtual Host proxies incoming requests to our node.js application running at localhost on port 3000, and if our application performs a HTTP 301 or 302 redirect for one of those requests, there we be an error.

Our response headers will contain a location of http://127.0.0.1:3000/your/site/location/redirected-location when sent back to Apache.

And as you can tell, if Apache sends this location back to the client, then there will more than likely be an error.


ProxyPassReverse solves this by rewriting the location in our response headers so that Apache sends yoursite.com/redirected-location back to the client’s browser.


After that, save your Virtual Host configuration for your node.js application, and restart Apache.


And that’s it!

All incoming requests to yoursite.com should now be proxied to your node.js application.


Happy Coding! 😀

JavaScript Async and Callbacks versus Promises

JavaScript Async & Callbacks

Asynchronous calls and callbacks are how JavaScript is able to respond to user events that occur after the browser has loaded the DOM. This type of programming is commonly referred to as event-driven programming.

In event-driven programming, once an event occurs, its registered callback to that event fires off its functionality. An event could occur from many varying incidents, such as removing an input field from a web form, or retrieving data from a remote web service.

Let’s take a look

Async call with a Callback


// get data from remote url
getJSON(url, function(err, data){
  // check for errors
  // no errors - process data
  // check data condition
  // check another condition
  // else do this condition
});

The data returned from calling our remote web service, fires the callback function with either the data or an error returned as a dynamic argument.

Using a callback for a simple event is fine. However, if an event contains multiple conditions that need to be checked, then callback functions can become hard to read and maintain. In these more complex cases, JavaScript Promises are the solution.


JavaScript Promises
JavaScript promises are considered a best practice when “calling back”. Promises make callback functionality easier to read by eliminating the need for convoluted error and data conditional checking.

The reason is due to a specific condition will not fire, unless it occurs.

Let’s take a look

Async Using a Promise


// our promise
var promiseData = getRemoteData(url);

promiseData.then(function(data){
  // do something interesting
  // with the returned data  
});

We create a promise variable named promiseData and then call the remote web service like before.


JavaScript promises have a then method defined on them, and this is the location where we create our callback functionality.


If any side effects occur, such as an error, we don’t have to check for this condition, because our then method will only fire the callback if the data is returned. If the data does not return, the then method never executes.


And that’s it! There’s obviously a million other examples between the two, but this gives an simple clear cut practical solution on why you would use promises.

Happy coding! 😀

MySQL Replication on CentOS through an SSH Tunnel

Replication in MySQL is keeping databases somewhat “in sync”, minus the latency at least.. So let’s pray to the computer gods (maybe Ava in Ex Machina?) 🙂 that we don’t run into any weird, complex, race conditions..

There are multiple ways to do data replication in MySQL and if your database isn’t very big, then true MySQL replication is probably overkill.

First, you could just write a bash script that runs as a cronjob on the host machine to mysqldump and save. Then on the client write and run a bash script as a cronjob to grab from the host and import the mysqldump into the database. But this obviously isn’t going to be ideal with anything other than small ‘toy’ databases.

So we need some awesome automated replication-like feature that only updates child databases with changes that are made in the parent database as it changes. Have no fear, MySQL has had such a feature built into its core pretty much since its inception!

Now, if you’re reading this, I’m sure you know the benefits of an SSH tunnel, but if you don’t, well…. SSH tunneling is a cheap (and ghetto) way to keep our data transfer done in a secure fashion while at the same time minimizing open ports and keeping our attack surface minimal.

Enough said.. So let’s get started!


**NOTE: Setting up an SSH tunnel is beyond the scope of this tutorial unfortunately, so search for one of the million tutorials on how to setup up a SSH Tunnel (with keys) if you need help and then come back..

**ALSO NOTE: Data replication is NOT a substitution for a proper data backup policy.. You’ve been warned…


First we need to configure our Master. The “master” MySQL server is the one that will be changing. The data that changes in the Master will then propagate and be replicated in the slave(s).

Edit /etc/my.cnf


# place this is under [mysqld] and NOT [mysqld_safe]
[mysqld]
server-id=1

log-bin=mysql-bin

# binary logs take massive disk space, this removes logs after 10 days.
# Change to an appropriate time frame for your needs.
expire_logs=10

# change database_name to the db name to be replicated
binlog-do-db=database_name

Next, let’s create a MySQL user to specifically do the replication.

Create replication user


# login to mysql
]$ mysql -u root -p

# create a user called replicator 
# (he's the king of all replicators, 
# in case you were wondering..) :D
mysql> CREATE USER 'replicator'@'127.0.0.1' 
IDENTIFIED BY 'Aw3someSecurePassw0rd111!';

Next, grant permissions to the new replicator account.

NOTE: We only create this account on the master, this will be the account the slave uses to access the master via the slave machine.


mysql> GRANT REPLICATION SLAVE ON *.* 
TO 'replicator'@'127.0.0.1' IDENTIFIED 
BY 'Aw3someSecurePassw0rd111!';

mysql> quit;

Now restart the MySQL server


]$ sudo service mysqld restart

Next, we need to check our master’s status.

Check Master server status


]$ mysql -u root -p

mysql> USE database_name;
mysql> FLUSH TABLES WITH READ LOCK;
mysql> SHOW MASTER STATUS;

You should see an output similar to:
+———————–+———-+—————+——————+
| File   | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+———————–+———-+—————+——————+
| mysql-bin.000001  | 107  | database_name | |
+———————–+———-+—————+——————+

Take note of what is under “File”, meaning write down mysql-bin.000001 along with the “Position” of 107. You need these values on the slave(s).



Next perform a mysqldump of the database that you’re replicating. If you need more help than this, google how to mysqldump a mysql database and pick one of the million tutorials. Umm, make sure you bring air freshener. 😛

**NOTE: Don’t exit from the mysql terminal, open a new terminal to do the dump. Trust me.

Mysqldump the database


]$ mysqldump -u root -p 
--opt database_name > database_name.sql


Next copy the db dump to your slave machine and then create the same database on the slave and import the dump. If you need help on this, there’s also many tutorials available by asking the google god.. Errr I mean Ava. 🙂


Now we need to edit /etc/my.cnf on the slave machine

Add the following to /etc/my.cnf


# again make sure all of this is under [mysqld] and NOT [mysqld_safe]
[mysqld]
server-id=2

Next we need to restart the mysql daemon

Restart MySQL


]$ sudo service mysqld restart

Now on the slave let’s create an ssh tunnel (assuming you have ssh working properly)

SSH tunnel on the slave


# IP ADDRESS: ip address of the master
# PORT: any port you have free
]$ sudo ssh -f  -L :127.0.0.1:3306 -N

# example: ssh -f 23.23.29.129 -L 4444:127.0.0.1:3306 -N

This will ssh to the master server
-f requests ssh to go to the background
-L specifies the local port (client/slave)
to forward to the remote host on 3306
-N don’t execute a remote command



Next let’s test the tunnel


]$ mysql -h 127.0.0.1 -P 4444 -u any_valid_user -p



If everything is valid and good to go with the tunnel, then we
need to login to mysql on the client (slave) and configure a few
things..


On a MySQL slave server type


# substitute your values that were output 
# earlier when checking the master's status
mysql> CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000001',
MASTER_LOG_POS=107,
MASTER_HOST='127.0.0.1', MASTER_PORT=3305,
MASTER_USER='replicator',
MASTER_PASSWORD='Aw3someSecurePassw0rd111!';

# start the slave
mysql> start slave;

# and then type
mysql> show slave status\G



If everything went according to plan, you should see:
Slave_IO_Running: Yes
Slave_SQL_Running: Yes



And that’s it! 😀

Everything written to the master db should propagate to the slave(s) almost instantly.


**Note: With this basic setup, do not write to a slave or replication will more than likely break, induce problems and ultimately a headache…


You can add read_only flags, replicate only specific tables, and just have a more detailed and complicated use case, but that’s out of the scope of this tutorial.


This is just a basic tutorial to help get your feet wet! But if you have a question or need help on a more complex setup, just ask! 😀

The Decorator Pattern in Java and Javascript

The Decorator Pattern is a highly used Object Oriented structural design pattern, that gives the ability to attach additional responsibilities to an object dynamically, without sub-classing.


When this pattern is used correctly, it can be extremely powerful due to the flexibility it offers.


So… Grab yourself a cup of coffee, or better yet… Some of THIS and let’s get to work! 😀


Now you’ve probably seen the Decorator Pattern implemented multiple times in many Object Oriented languages, so to change it up a bit, we’ll compare the implementation in both Classical inheritance and Psuedo-Classical inheritance with some Prototype inheritance functionality.


**NOTE:  The JavaScript implementation of the decorator pattern is not pure Prototypal inheritance.

It’s referred to as “Pseudo-Classical” inheritance with a few aspects of Prototypal inheritance. I explain why once we get to the JavaScript implementation.


Before we start, let’s talk about why you would even consider using the Decorator pattern, by giving a practical example.


Suppose we’re designing a system for a restaurant…

Now, in our restaurant’s system, we have a drink application that contains a subsystem which calculates the price of different drinks that this restaurant sells.

However, since we don’t live in a perfect world, we assume that not every customer will want to order a drink exactly the way it’s specified on the menu.

Since declining custom drink orders is bad for business, and the number of possible drink combinations are high, we are going to have to think about our design thoroughly.


As you can probably already see, a typical inheritance hierarchy isn’t going to work by itself…

Sub-classing for every current and future drink type *combination* would become a maintenance nightmare, therefore another approach must be taken.


**NOTE: Before we start, I also want to make a point that this design is not what you would do for any real application, much less a production application.


Why?

Well for one, we wouldn’t just throw all of our concrete subclass “decorations” (Red Bull, Orange Juice, Monster, etc) all into a single hierarchy under the Decorator.

Clearly Red Bull and Orange Juice have nothing in common…

Well, besides being a liquid… But with good Object Oriented Analysis and Design, along with being a production restaurant application, that is clearly not enough.


Well… Now that I think about it…
That’s assuming that this application will *actually* need to be maintained in the future.

If it didn’t, we could just throw everything in one gigantic function…
Maybe main()?
lol.. I kid.. I kid… 😀


Why is it not enough?

Well, what would happen if we wanted to decorate our drink with fruit?
maybe a cherry?
how about a lime?
or what about a napkin?
maybe a straw?


As you can see, those objects aren’t anywhere near related to each other.

The only thing a cherry and a napkin have in common, is that some drinks may use both of them.. lol 😀


So these issues (along with many others) are the reason our Object Oriented design for the entire restaurant system needs to be thoroughly thought out.


The reason I have it implemented like this, is to keep it simple..

By doing so, we can focus all of our attention on the actual Decorator Pattern, and what it is doing, without confusion.


So, without further ado….

*Clears throat… Mic check 1…2…3…* 😀


—- Introducing the Decorator Pattern —-


Decorator Pattern UML
decorator_pattern


Java Output
decorator-ss


The Decorator Pattern implementation in Java (Classical Inheritance)

Drink.java  –  Client Code

public class Drink {

  // Client Code
  public static void main(String[] args) {
  
    // create a drink with brand of vodka
    Vodka drink = new GreyGoose();

    // display vodka brand and price
    System.out.println(drink.getBrand() + 
      " $" + drink.calculatePrice());

    // decorate the drink with a mixer
    // mix the drink with a large Red Bull
    Vodka mixedDrink = new RedBull(drink, "Large");

    // decorate the drink with another mixer
    // mix the drink with a medium Orange Juice
    mixedDrink = new OrangeJuice(mixedDrink, "Medium");
        
    // decorate the drink with yet another mixer
    // mix the drink with a Medium Monster
    mixedDrink = new Monster(mixedDrink, "Medium");

    // Display the type and price of the drink's
    // mixed ingredients, along with the drink total,
    // and format output to 2 places
    System.out.printf("Mixing a drink of " + 
      mixedDrink.getBrand() +
      "\nTotal price is: $%.2f", 
      mixedDrink.calculatePrice());
    }
}

Vodka.java

public abstract class Vodka
{
  protected String brand = "Unknown Vodka";

  protected String getBrand()
  {
    return this.brand;
  }

  protected abstract double calculatePrice();
}

GreyGoose.java

public class GreyGoose extends Vodka
  public GreyGoose()
  {
    this.brand = "Grey Goose";
  }
  
  public double calculatePrice()
  {
    return 32.99;
  }
}

VodkaDecorator.java

public abstract class VodkaDecorator extends Vodka
{
  protected Vodka drink;
  protected abstract double sizePrice(String size);
  protected abstract String getBrand();
  protected abstract double calculatePrice();
}

RedBull.java

public class RedBull extends VodkaDecorator
{
  private String size;
  private double price;

  public RedBull(Vodka drink, String size)
  {
    this.drink = drink;
    this.size = size;
    this.price = this.sizePrice(size);
  }

  protected double sizePrice(String size)
  {
    double price;   
    switch (size) {
      case "Small":
        price = 2.99;
        break;
      case "Medium":
        price = 4.99;
        break;
      case "Large":
        price = 6.99;
        break;
      default: // unknown size - super wings
        price = 8.99;
      }
    return price;
  }

  public String getBrand()
  {
    return this.drink.getBrand() + 
      "\nand a " + this.size +
      " Red Bull which gives you wings for $" +
      this.price;
  }

  public double calculatePrice()
  {
    return this.price + 
      this.drink.calculatePrice();
  }
}

OrangeJuice.java

public class OrangeJuice extends VodkaDecorator
{
  private String size;
  private double price;

  public OrangeJuice(Vodka drink, String size)
  {
    this.drink = drink;
    this.size = size;
    this.price = this.sizePrice(size);
  }

  protected double sizePrice(String size)
  {
    double price;   
    switch (size) {
      case "Small":
        price = .99;
        break;
      case "Medium":
        price = 1.99;
        break;
      case "Large":
        price = 2.99;
        break;
      default: // unknown size - super orange
        price = 3.99;
      }
    return price;
  }

  public String getBrand()
  {
    return this.drink.getBrand() +
      "\nand a " +
      this.size +
      " Monster energy for $" +
      this.price;
  }

  public double calculatePrice()
  {
    return this.price +
      this.drink.calculatePrice();
  }
}

Monster.java

public class Monster extends VodkaDecorator
{
  private String size;
  private double price;

  public Monster(Vodka drink, String size)
  {
    this.drink = drink;
    this.size = size;
    this.price = this.sizePrice(size);
  }

  protected double sizePrice(String size)
  {
    double price;   
    switch (size) {
      case "Small":
        price = 3.99;
        break;
      case "Medium":
        price = 5.99;
        break;
      case "Large":
        price = 7.99;
        break;
      default: // unknown size - super energy
        price = 9.99;
      }
    return price;
  }

  public String getBrand()
  {
    return this.drink.getBrand() +
      "\nand a " + 
      this.size +
      " fruity Orange Juice for $" +
      this.price;
  }

  public double calculatePrice()
  {
    return this.price +
      this.drink.calculatePrice();
  }
}



OK, let’s talk about what’s going on…


First, starting in our client code (Drink.java), we create a new drink by instantiating the concrete subclass Grey Goose as a type Vodka.


Next, we “decorate” our drink of Grey Goose with some Red Bull…

We do this by instantiating a concrete subclass of one of our “decorations”, and then inject our drink object (Grey Goose) into our Red Bull object.

By decorating (wrapping) our drink object, this will allow us to polymorphically call calculatePrice() on both objects (when unwrapping), since both Vodka subclasses and VodkaDecorator subclasses are of type Vodka.


**NOTE: Notice how our VodkaDecorator not only inherits from our abstract base class Vodka, but it also composes a Vodka object.


If you haven’t already seen why, having our VodkaDecorator inherit from Vodka and compose Vodka is extremely important.

Our decorations (RedBull, Monster, etc.) need to have the same “interface” as the object they’re decorating.

With VodkaDecorator, we’re using inheritance in order to gain the same super-type as our drink objects, but we’re using composition in order to gain behavior.


Gaining behavior through composition is where our flexibility comes from. If we didn’t, and gained our behavior through inheritance alone, then any needed behavior would have to be determined at compile time only from what our Vodka base class can offer.
Since we’re using composition we can mix our decorations in any order at runtime!


And that’s really all there is to it.. We can use as many or as few decorations that we need for our drink.

We could pass our drink object to multiple RedBull objects, or one Red Bull, one Orange Juice, and one Monster (like the implementation does), or none at all..




OK, now let’s discuss the implementation in JavaScript…


The decorator pattern isn’t really relevant in JavaScript like it is in statically typed languages like Java.

Extending objects at runtime are common with dynamically typed languages, and this behavior is native in JavaScript.

Because it’s more or less built into JavaScript, the behavior that we’ve seen the decorator pattern provide isn’t really needed.

Furthermore, when JavaScript’s native behavior isn’t sufficient, another pattern called a Mixin is usually much better suited.

However, let’s still take a look at what it looks like in JavaScript…


**NOTE: This JavaScript implementation is referred to as pseudo-classical inheritance, not prototypal.

Even though JavaScript is a classless language, this implementation still simulates classes and treats the simulated classes like a class based heirarchy.


This is why this type of inheritance implementation is referred to as “pseudo-classical”.

In true prototypal inheritance we have no need for classes, whether real or simulated, so abstracting a classical inheritance hierarchy doesn’t make much sense.


The Decorator Pattern implementation in JavaScript


Run output in JSFiddle



Drink.js  –  Client Code

var main = (function() {
  var drink = new GreyGoose('Grey Goose');
  alert(drink.getBrand() + 
  " $" +
  drink.calculatePrice());

  var mixedDrink = new RedBull(drink, "Large");
  alert(mixedDrink.getBrand() +
    "\nTotal price is: $" +
    mixedDrink.calculatePrice());
})();

Vodka.js

var Vodka = function() {
  if (this.constructor === Vodka) {
    throw new Error("Abstract");
  }
};

Vodka.prototype.getBrand = function() {
  if (this.brand === undefined) {
    this.brand = "Undefined Vodka Brand";
  }
  return this.brand;
}

Vodka.prototype.calculatePrice = function() {
  throw new Error("Abstract Method");
}

GreyGoose.js

var GreyGoose = function(brand) {
  Vodka.apply(this, arguments);

  this.brand = brand;
};

GreyGoose.prototype = Object.create(Vodka.prototype);
GreyGoose.prototype.constructor = GreyGoose;
GreyGoose.prototype.calculatePrice = function() {
  return 32.99;
};

VodkaDecorator.js

var VodkaDecorator = function() {
  if (this.constructor === VodkaDecorator) {
    throw new Error("Abstract");
  }
};

VodkaDecorator.prototype.sizePrice = function(size) {
  throw new Error("Abstract Method");
 };

VodkaDecorator.prototype.getBrand = function() {
  throw new Error("Abstract Method");
 };

VodkaDecorator.prototype.calculatePrice = function() {
  throw new Error("Abstract Method");
}

RedBull.js

var RedBull = function(drink, size) {
  this.drink = drink;
  this.size = size;
  this.price = this.sizePrice(size);
}

RedBull.prototype = Object.create(VodkaDecorator.prototype);
RedBull.prototype.constructor = RedBull;

RedBull.prototype.sizePrice = function(size) {
  return function() {
    var price;
    switch (size) {
      case "Small":
        price = 2.99;
        break;
      case "Medium":
        price = 4.99;
        break;
      case "Large":
        price = 6.99;
        break;
      default:
        price = 8.99; // unknown size - super wings
        break;
    }
    return price;
  }(); // private method by closure
};

RedBull.prototype.getBrand = function() {
  return this.drink.getBrand() +
    "\nand a " + this.size +
    " Red Bull which gives you wings for $" +
    this.price;
};

RedBull.prototype.calculatePrice = function() {
  return this.price +
    this.drink.calculatePrice();
};



With the exception of not decorating our drink with as many decorations, the JavaScript implementation is exactly the same as the Java implementation.

Within the confines of JavaScript we use the same inheritance hierarchy, and we add in some prototypal aspects to create the same decorator pattern behavior.


The explanation of prototypal functionality is out of the scope of this tutorial, but using this behavior (and some hacky throwing of errors) we can mimic classical inheritance in JavaScript.

In actuality, implementing the behavior of the decorator pattern in JavaScript (in a non psuedo-classical way) is much simpler.


So there you have it!
The decorator pattern offers powerful flexibility when used appropriately and it will prove itself to be a powerful tool in your Object Oriented toolbox.

Have fun! 😀