I’m a skeptic, satiated by large raw data sets, analysis & inference

Speak to anyone who knows me, and they will likely characterize me as a skeptical, pessimistic, anxious, intense, and persistent individual.

If someone sends me a spreadsheet and then calls me to walk me through the numbers my immediate assumption is that the purpose of the follow-up call is to shape my perception. If someone provides me a composite of the figures without the raw data, visible formulas and documented data sources, I also assume manipulation. With this said I am a realist, and I am willing to accept manipulation, but I am honest about acceptance rather than convincing myself otherwise. I am just wired to be vigilant.

For me the glass being half-full represents a lack of fear of it being half-empty, I am motivated to refill the glass by the reality that it is half-empty and what is likely an unhealthy fear of dying from dehydration, but it works for me. From my perspective, the half-empty glass is not depressing or a demotivator it is a potential reality. Now don’t get me wrong, I know there is water in the glass and death is not imminent, but I am incredibly aware and grateful for the opportunity to find a water source to refill my glass.

I spend my days listening to dozens of pitches, where I need to focus, why I need to do x or y, what I am missing out on by not doing x or y, etc… The pitches almost always start with a half-full perspective, selling the positive but it’s amazing how when it doesn’t go the way the pitchman expects the approach shifts to the half-empty perspective, relying on FOMO (fear of missing out) as a last ditch attempt at motivation.

Now let’s face it, no one likes to miss out, but as a realist, I recognize that I can’t do everything, so decisions are required. Forks in the road appear every minute of every hour of every day, and I am greeted at each fork by a host espousing the merits of their righteous path. For someone like me, these decisions need to be my own, driven by raw data (as raw as it can be), analysis and inference. I try to check the near-term outcomes at the door and focus on and visualizing the long-term strategic outcomes, the vision. In my mind tactical activities require little to no thought, they just happen. For example, a visionary looking for a more sustainable model for garbage disposal doesn’t stop taking their garbage to the curb every Monday and Thursday. Accepting what is and executing without much thought IMO avoids paralyzation and makes room in your life and brain for what will be.

So now we arrive at the origin of this blog. I have to make personal and professional bets on where the market is going, what it most relevant and where I should focus my time. Of course, I have a subjective opinion on where I believe the market is going but I like to validate my opinions with some data and with so many people, organizations and news outlets selling their version of the future the question becomes, how do I validate my opinions objectively. Social chatter is meaningful to me as is sentiment analysis. The great news is with a little Python, the use of some APIs and the ELK stack it’s pretty easy to collect data from social media platforms, store it, analyze it and draw some conclusions. One such question that is very relevant to me is what technologies and what OEMs (original equipment manufacturers) have mindshare? I’ve been pulling social media data for a few weeks using #hashtags to see what techs and OEMs have the most buzz; I have also been doing sentiment analysis to see if the buzz is good or bad.

Here is my view of the market using social media buzz to determine mindshare (it actually feels pretty on the money):

The world has changed, are you paying attention?

This blog is the result of a restless night where I pondered a recent event where the idea (or existence) of NOC (Network Operations Center) was conveyed as a key component of the ITSM (Information Technology Service Management) paradigm. I find this to be an uber interesting topic and position given that the world has moved (and continues to move) in every way from a centralized to a disaggregated and distributed model. I believe this is true in computing (think cloud, microservices, twelve-factor apps, etc…) and it’s true in the area of human capital management and service delivery.

I thought I would share some of my opinions on the topic, my position as well as some anecdotes that I believe support my thoughts.

First, let me start by saying that we are engaged in a war, a war for human capital, a war where the best knowledge workers don’t look anything like what they looked like twenty years ago, they live in the shadows, digital nomads inhabiting a digital universe.

When I think NOC, here is what I envision:

The above is a picture of the NOC from the movie WarGames which was released in 1983, this was cool and impressed the audience, but it was 35 years ago! It’s probably obvious from looking at my blog header that I am a big WarGames fan. Let’s stay with the Hollywood portrayal of tech for a moment because I think it’s relevant.

Fast forward from 1983 to 2001, 18 years later, and the NOC has given way to the lone hacker, with umteen monitors (quite a setup) working alone to High Voltage by The Frank Popp Ensemble.

Disaggregation and decentralization have become a pervasive theme, message and a way of life. Nowhere is this more evident in than in the Open Source community. Disaggregation and decentralization coupled with a shifting culture that has shifted the motivation of the knowledge worker has given way to an unprecedented pace of innovation which would otherwise be impossible.

The Open Source statistics are truly staggering: https://octoverse.github.com/

Couple what the Open Source movement has taught us about the power of disaggregated and decentralized teams with “the surprising truth about what motivates us” and you’ll realize that the disaggregated and decentralized cultures being built are unlike anything we could have dreamed. The passion, commitment, engagement, communication, execution, and velocity are astounding.

Ask yourself where people (yourself included) go for help, how they build communities, what are trusted sources of information, etc…
Where do developers look for help? StackOverflow, Slack, IRC, Quora, etc…?
Where does the average person look for help? Facebook, YouTube, Twitter, etc…?

These are all platforms which enable the construction of disaggregated and decentralized communities which create cultures, subcultures, increase engagement, provide better time to value, etc… Are there no lessons to be learned here? There are lessons to be learned, and many are learning these lessons and adapting their engagement models.

I am a techie, and I believe that substance will always prevail over style and the question I continually ask myself as I adjust to keep up with a market which is innovating and changing at an unprecedented pace is how to define the culture? Is what we are doing relevant today and does it put us on a trajectory to where we’ll need to be in 24 months?

And now we have arrived at my thoughts regarding the NOC.

JetBlue made a bold move (which others followed) shifting from reservation call centers to hiring booking agents who work virtually, and their customer service is consistently rated the highest in the industry.

Relevant business models do NOT focus on resource colocation; they focus on resource capability, availability, and collaboration. I would go as far as to say that colocation favors style over substance.

The cultures we build need to focus on leveraging technology to deliver a great total customer experience (TCE). I believe that a 5.3” screen in the hands of hundreds of engineers, elegant engagement processes, procedures, and tools deliver a better TCE than a 60” monitors on the wall in a room with ten engineers with landline phones. Cultural agility over environmental rigidity.

The focus and value here is NOT a finite set of L1, L2 and L3 shift workers in a NOC. Big screen TV’s on the wall, the Bat Phone and people sitting at a desk are style decisions which have no direct correlation to the ability to deliver substance. Our focus needs to be on how to engage and nurture the best knowledge workers the market can offer. Our mission needs to be the creation and cultivation of a culture which fosters engagement. Our ability to engage and escalate to a subject-matter expert (SME) at any time, to improve the TCE by building equitable partnerships which deliver distinct value, with a meaningful escalation path that focuses on MTTW (Mean-Time-to-Workaround) while in parallel determining a root cause and resolution lies in our culture.

We must understand that the world has changed.  We live in a world where seemingly forward-thinking paradigms are obsolete before they are implemented.  The path to success relies on agility and accountability, not rigidity and responsibility.

POTUS Sentiment Analysis

Just finishing up a class where I have been doing a lot of social media mining and analytics and I thought it would be a fun given all the banter about #fakenews to do my own POTUS sentiment analysis.

Note:  The above processes are running and the visualizations below are live.  Refresh page to refresh data.

Potus Sentiment

Disclaimer:  This is real data pulled from the twitter API.  I can assure you there is zero motivation or interest here beyond the raw data, the sheer volume of data given the topic, and of course I realize this is relevant to many others which is what makes it interesting.

Quickly Gather RecoverPoint Replication Stats

It’s been a while since I posted, I think I got to caught up in writing lengthy posts (which I often never completed) rather than just publishing content as I have it and as my personal time allows.  This post is the start of a new philosophy.

Last week I had a need to quickly grab some replication stats from RecoverPoint and I thought I would share the process and code I used to do this.

Prerequisites:  plink, sed, awk, head, tail and egrep

Note:  Because this is not a tutorial I am not going to talk about how to get the requirements configured on your platform.  With that said you should have no issues getting the prerequisites work on Windows or Linux (for Windows Cygwin may be a good option).

The resulting output is a CSV which can be opened in Excel (or whatever) to produce a table similar to the following:

image

My Iomega ix2 and my new 3 TB USB drive

Purchased 3 TB Seagate USB 3.0 drive from Amazon (http://amzn.to/TpduBU)

Waited… Very excited to connect to my ix2….

A few days later my 3 TB USB expansion drive arrived.  I hurried to unpack and connect to my ix2 expecting plug-and-play.  I plugged by no play.

clip_image001

An overwhelming feeling of sadness consumed me, followed by WTF then the joy of knowing I could and would hack this to make it work.

Knowing that this Iomega thing had to be running Linux I began to scour web for how to enable SSH with Firmware Version 3.3.2.29823

Found plenty of how to information on Firmware Version 2.x but 3.x (Cloud Enabled Firmware) is a bit more sparse.

Finally to enable SSH:  http://ip/diagnostics.html

clip_image002

SSH now enabled, opened PuTTY and SSH to device.

Username:  root
Password:  soho

Boom!  In….

clip_image003

A quick “df -h” shows my currently configured capacity:

clip_image004

A quick “cat /proc/scsi/usb-storage/4” followed by a “fdisk -l” reveals the drive is being seen by the ix2.

clip_image005

clip_image006

Created partition on /dev/sdc, “fdisk /dev/sdc”

clip_image007

Now what?

Hmmmmm…. Maybe I can create a mount point on /mnt/pools/B/B0, seems logical.

clip_image008

clip_image009

Whoops forgot to mkfs.

Run “mkfs /dev/sdc1”

clip_image010

“mount /dev/sdc1 /mnt/pools/B/B0/”

clip_image011

Hmmmm…..

“umount /dev/sdc1”

Tried to partition with parted (core dumps, ix2 running ver 1.8.  pretty sure GPT partition support not ready for primetime in vet 1.8)

Let see if I can get a new version of parted.

Enabled apt-get (required a little work)

cd /mnt/pools/A/A0
mkdir .system
cd .system

mkdir ./var; mkdir ./var/lib/; mkdir ./var/cache; mkdir ./var/lib/apt; mkdir ./var/cache/apt;  mkdir ./var/lib/apt/lists; mkdir ./var/lib/apt/lists/partial; mkdir ./var/cache/apt/archives;  mkdir ./var/cache/apt/archives/partial; mkdir ./var/lib/aptitude

(I think that is all the required dirs, you will know soon enough)

cd /var/lib
ln -s /mnt/pools/A/A0/.system/var/lib/apt/ apt
ln -s /mnt/pools/A/A0/.system/var/lib/aptitude/ aptitude
cd /var/cache
ln -s /mnt/pools/A/A0/.system/var/cache/apt/ apt

run “apt-get update”

Should run without issue.

run “aptitude update”
Note:  Should run without issue.

clip_image012

Jettison that idea, not enough space on root and /mnt/apps to install new version of parted and required dependencies.

New approach:

run “dd /dev/zero /dev/sdc”

Let run for a minute of so to clear all partition info (ctrl-c) to stop

Download EASEUS Partition Master 9.2.1 from filehippo (http://www.filehippo.com/download_easeus_partition_master_home/)

Install EASEUS Partion Master 9.2.1 on Windows 7 desktop
Connect 3 TB Seagate USB drive to Windows 7 desktop
Partition and format partition ext3 using EASEUS Partion Master 9.2.1
Note:  This takes a little while.

Once complete I connected the drive to my Iomega ix2

Voila!

clip_image013

clip_image014

Cleaned up the “/mnt/pools/B directory” I created earlier (“rm -rf /mnt/pools/B”)

Reboot my ix2 (make sure I didn’t jack anything up) and enjoy my added capacity.

image

Pretty sick footprint for ~ 4.5 TB of storage (1.8 TB of it R1 protected).

Mapping RDM Devices

This post was driven by a requirement to map RDM volumes on target side in preparation for a disaster recovery test.  I thought I would share some of my automation and process with regards to mapping RDM devices that will be used to present RecoverPoint replicated devices to VMs as part of a DR test.

Step 1:  Install VMware vCLI and PowerCLI
Step 2:  Open PowerCLI command prompt
Step 3:  Execute addvcli.ps1 (. .\addvcli.ps1)

Step 4:  Execute getluns.ps1 (. .\getluns.ps1)

Step 5:  Execute mpath.ps1 (. .\mpath.ps1)

Step 6:  Get SP collect from EMC CLARiiON / VNX array
At this point you should have all the data required to map the RDM volumes on the DR side.  I simply import the two CSVs generated by the scripts into excel (scsiluns.csv, mpath.csv) as well as the LUNs tab from the SP Collect (cap report).
Using Excel and some simple vlookups with the data gathered above you can create a table that looks like the following:
I could probably combine these three scripts into one but under a time crunch so just needed the data, maybe I will work on that at a later date or maybe someone can do it and share with me.

Verizon Actiontec Router and Local DNS

I have been really busy and not posting much, but I have my home lab pretty much built out and have a bunch of new projects in the hopper, more on that in future posts.  If you have FIOS like I do you probably have a Actiontec router provided by Verizon.  When building out my home lab I wanted to use my Actiontec router as my DNS server, for obvious reasons, the web interface became frustrating pretty quickly.  So many clicks and the ability to only enter a single host registration at a time:

image

The ability to edit DNS from telnet is actually really nice on the Action tech router.  Commands are petty simple.

1) Enable Telnet on the router (Advanced –> Local Administration)

image

2) Once telnet is enabled, you can now telnet to your router using the same credentials used with the web interface.

image

3) After the telnet session is established there are basically three commands you need to be familiar with:

  • dns_get:  lists all DNS server entries
  • dns_set:  adds a DNS entry
  • dns_del:  deletes a dns entry

The syntax is pretty simple:

  • dns_get:  used by itself to list all DNS entries
  • dns_set: dns_set ID HOSTNAME IP_ADDRESS (e.g. – dns_set 1 host1 192.168.1.100)
  • dns_del:  dns_del ID (e.g. – dns_del 1)

This method of adding and removing DNS entries from the Actiontec router is significantly faster than using the web interface.

I use a Google Doc spreadsheet to track my IPs and build the command to add and remove DNS entries.  I have shared my template here:  https://docs.google.com/spreadsheet/ccc?key=0Alfr2Qqx-moWdE43YTFZLVRtRWM1X3VsdXY2UmFBVUE

Best Remote Connection Tool

I have tested a ton of tabbed remote connection tools.

RDTabs (http://www.avianwaves.com/tech/tools/rdtabs/):  Like it for pure RDP, no SSH, http, etc…

Terminals (http://terminals.codeplex.com/):  Slow and a little buggy IMO

Remote Desktop Manager (http://devolutions.net/):  Over built app, not portable, etc…

I am now using mRemoteNG (http://www.mremoteng.org/):  Love it!

image

This fits all my needs.  Supports all the protocols that I require, no install portable version available which is perfect for me.  I have the portable version in my dropbox (http://www.dropbox.com/) folder so I can launch on any machine and have all my connections readily available.  I can add connections anywhere and they sync’d via  dropbox.  The perfect solution for me.  The app is light weight and fast, give it a try.

App that provides dramatic productivity improvements (for parents)

So this may seem like a strange post, as most people will think that I am going to be talking about a an IDE application, a RAD tool, a CRM application or some sort of text-to-speech processor, regardless of what you are expecting I can almost guarantee you will be expecting something a little more sexy than what you are about to see (especially if you are not a parent).

I think this app is so useful I am not only posting to my appoftheday.org blog but also to my gotitsolutions.org blog because it is that good.

Let me provide some background.  I have two wonderful little girls, a 5 year old and a 6 month old, for anyone with children we all we have retooled the human machined (ourselves) to have a CPU that is focused on work and coprocessor that deals with our children while we try to focus (we can flip this paradigm as well).  I have to say my time slicing skills are second to none, you learn how to work in 2 min slices while breaking away for 30 seconds to lend some CPU cycles to an often overheating parental coprocessor.  I often read emails back later that had the same thought double typed, missing words, etc… this is because I am processing too much information, my mental programming is way off.  I have this huge array of things I need to do, things I am doing, things I am being told to do, things my kids want to do, yadda, yadda, yadda…. Let’s just say that the that I often suffer pointer corruption which leads to memory leaks, corruption and eventually a segmentation fault (in non techie lingo this is know as a freak out, but this is a technical blog hence the techie speak).

So to the point of the post.  There is this brilliant lady named Julie Aigner-Clark the founder of The Baby Einstein Company, absolute best videos for kids under the age of one to help cool down the coprocessor (why didn’t I start filming shiny lights and hand puppets 10 years ago).  My 5 year old will even watch the videos.  There is this great website site called YouTube where you can find Baby Einstein videos as well as other great videos like Oswald, WordGirl, Hannah Montana and The Pink Panther (a few of my older daughters favorites) So you are probably asking what relevance does this have.  I will explain, be patient, I know how difficult this probably is because you 6 month old wants to eat and your 5 year old wants you to “Play Barbies” with her.

I am in my office trying to work and my daughter comes in, she wants me to stop what I am doing to play with her, I attempt to stall and concentrate at the same time (very difficult).  I eventually sit her on my lap (applies to 6 month old and 5 year old) and open YouTube in my browser and start playing our favorite Baby Einstein or WordGirl video.  Good so far.  I pop out the video window from youtube.com and resize my excel sheet and attempt to work, here is a screen shot of what I am left with:

image

So on the left my daughter(s) can sit on my lap and watch the vide while I work on the spreadsheet on the right.  Now here is the issue, I only have 3/4 of the screen which can be a little annoying, if I need to use another app it can be a big issue.  So what is the effect of me switching windows:

image

Oh no, the video moved to the background, scramble to resize the browser window to avoid a complete meltdown.  Reflexes are not that good so I rarely accomplish the goal.

Now for the introduction of a must have application that dramatically improves productivity, focus and sanity.  The app is called DeskPins and simply it allows you to pin any window to the foreground so lets look at a couple of examples of how I use this.

I follow the same process as before with finding a video on YouTube, popping out the video windows but now I pin the video window to the foreground.

image

Now I can maximize my spreadsheet (far better) and without the video moving to the background, I can move the video window around as needed.  I can open FireFox and not worry about losing the video to the background.

image

The app works on 32 and 64 bit versions of Windows (I am running on 32 bit XP, 32 bit Win 7 and 64 bit Win 7) and has become an invaluable tool for me.  Hopefully this post helps with some use case examples and helps other parents occupy their children in times of need.  Enjoy!

Intro to Unisphere session at #emcworld

  • Unified management platform for CLARiiON, Celerra and RecoverPoint
    • Unified look and feel across all aforementioned products
    • Support for all CLARiiON >= FLARE 19
  • Release data Q3 2010
  • Only functions at the presentation layer
    • Will not impact the CLI so scripts will not be impacted by Unisphere
  • Views
    • Dashboard
      • Unified alerting sorted by severity
      • System list in the top toll bar provides the ability to drill down from the domain level to the physical system to view details
    • System View
      • Graphical hardware depiction (finally)
    • Summary Views
      • Storage
        • Bye bye CAP reports!!!!!  (thank you)
      • Replication
    • Note:  View Blocks are widgets that are placed on the screen somewhere that Aggregate data graphically displaying utilization, capacity, etc…
      • Ability to customize views, these views are tied to the client not the user so they will stay on the client not move with the user
  • Right-click provides context menu similar to what is currently available in Navisphere
  • Array data is cached locally on Unisphere client
    • This will hopefully help out performance
    • Data collection from array is time stamped so you can ensure you are working with the correct data set
    • A refresh (manual) needs to be performed to query the array and refresh the Unisphere cache
    • Once the data is cached it can be manipulated
  • Context sensitive help and actions
  • Replication Manager (RM) and USM (Unisphere Service Manager) integration (via link and launch)
    • If apps not installed it launches PowerLink and grabs RM or USM
    • Performs the install
    • Launches the app
  • USM replaces NST and unifies the ability to perform service tasks on CLARiiON or Celerra
    • USM adds the ability to track FCOs, Support Advisories, etc… via RSS feeds
    • Also provides the ability to launch to the proper location to research things (e.g. – EMC Support Forums), USM remains as the presentation layer, no need to jump around between apps.
    • Service requests can be created directly from USM

Interoperability Matrix

Platform Native Managed
CLARiiON CX, CX3, CX4 FLARE R30+ FLARE R19+
Celerra DART 6.0+ DART 6.0+
CLARiiON AX4 TBD FLARE R23+
  • Navisphere Off-Array being replace by Unisphere client and server
    • Support for Navi Analyzer off-array
  • Navisphere manager license supports Unisphere, no need for an upgrade

Overall the app looks really nice considering most of us are used to using a fractured set of tools between CX, NS and RP.  Will be interested in seeing how Unisphere helps us map the use of the array.  It appears to still be a JAVA based application so the jury is out on performance until I see it.

Audience questions:

  • Can you get to engineering mode from Unisphere
    • Unanswered.  Translation: Assume access to engineering mode on the CX will still require us to log into Navisphere.
  • Support for Centera on the roadmap
    • Answered, Yes.  No timeframe given.
  • What is the user experience like on a very large CLARiiON that is busy
    • Answer, it depends.  Translation:  It will likely still be slow.