The best thing and the worst thing….

It has been said that the best thing about the internet is that anyone can publish, unfortunately the worst thing about the internet is that anyone can publish.  Another fitting cliché is that opinions are like buttholes, I propose that the blog sphere has become a public restroom, reeking from the stench of personal opinion backed by analogies, anecdotes, etc… and little fact (much like the opinions expressed in this blog).  By no means am I absolving myself from the claims made in this post, I am as guilty as the next guy when it comes to writing valueless content, but I do feel like I mix in some valuable content that is based on empirical data and facts.  I am all for some good rhetoric but let’s face it people, we all like to hear ourselves talk regardless of how little value the commentary actually has.  When your platform is largely opinions based loosely on facts as defined in the “Marchitecture” documentation you have to be willing to accept the 50% of the people who will agree with your perspective and 50% of the people who won’t.  Why does anyone even care what influences the authors perspective, why does it matter to the content consumer?  The assumption we all should make is that the content author is motivated by something, this motivation can be pure or corrupt in nature.  The great thing is you can decide to either agree or disagree, offer up some additional conjecture or not, that is the beauty of free will.


Every person has a predisposition to one perspective or another thus the concept of a non-biased view of the world, policy, product, etc… is made impossible by this little thing we call human nature.  But wait it's worse, beyond just human nature we have what I believe to be the two additional key aspects that influence behavior:

Indoctrination:  The belief system in which we participate (e.g. – WAFL vs CoW).  There is no doubt that a long time NetApp employee, user, etc… who has been indoctrinated in to the culture, ideology, thought process, etc… will believe that WAFL is a superior technology when compared to CoFW.  In contrast a person indoctrinated into the EMC culture could likely argue why CoFW is a superior technology.  The problem is that both of these perspectives speak to the technology and not the use case.

Personal Gain (monetary or otherwise):  My favorite because it has a huge impact because the so called independent analysts in the technology community (who will remain unnamed due to the litigious nature of the world we live in) are really marketing mercenaries or blackmail artists depending on your perspective.  This is not to say that analysts do not initiate coverage on technologies that they are no being paid to follow but let’s just say that the coverage of technologies that are being paid to follow is a bit more substantial.  It is funny how as human beings our opinions tend to align with our goals.

So my word of caution is as follows, trust ONLY yourself (and yes you can trust yourself, it is true that you likely have an agenda but is also true that this agenda is likely in your best interest), read lots of differing opinions and formulate your own.  Realize that reading information found on the web can be a dangerous thing if you don’t take what you have learned, internalize it and think for yourself.  My favorite example here is researching symptoms on WebMD, use the WebMD Symptom Checker and enter the req’d info (e.g. – male, 35-44 years), click submit, then drill down on the head, once you get to the symptom picker choose Headache (worst ever) now take note of the only possible condition.  Enough said about the dangers of the internet and not thinking for yourself.  So PLEASE apply some modicum of logic, reason and realism when digesting opinionated content.

So what prompted this seemingly common sense cautionary tale.  My opinionated colleague over at posted an OPINION entitled “More FUD busting: Deduplication – is variable-block better than fixed-block, and should you care?” on his blog that received criticism from other opinionated ministers of public enlightenment and propaganda.  I have read through the posts and the short answer is everyone is correct and equally adept at the art of FUD slinging, what a tragedy.   IMHO the market today (especially among the big boys) has parity +/- 1% (exclusive of the features that don’t work or no one cares about, and yes they exist in all products), the 1% differentiation is often littered with caveats, the blogs outlining these caveats, workarounds, use cases, etc… are the valuable ones, spend more time consuming this content and less time reading content that reminds more of TMZ than a technical blog.

It should be fairly easy when consuming content to determine what is valuable and what is not, just read Scott Lowe’s Blog to see what good content looks like.

One final thought, was the FTC warning really necessary, really????  See what I mean about the litigious nature of our society.  It all starts with nationalizing health care, the next thing you know the FTC is commandeering your blog, where does it end.

Data Profiling with Windows PowerShell

A customer asked me the other day about a method to do some data profiling against a file system.  So I thought I would share the request, my suggestion and little PowerShell script I crafted to do some data profiling.  The request read as follows:  “Do you know of any tools that will list all files in a directory, and all subs and provide attributes like filename, path, owner, create date, modify date, last access date and maybe some other attributes in report format?”

My recommendation was a commercial product called TreeSize Professional by JAM Software the product license cost is ~ $50 and worth every penny, scan speeds are good, supports UNC paths and reporting is intuitive.  Overall an excellent product.

As an alternative below is a quick PowerShell script (also attached to post as data_profile.ps1) that will create a CSV file with data profiling information, once the CSV file is create the CSV can be opened in Excel (or your spreadsheet tool of choice) or imported into a DB and manipulated.

The above script scans all files recursively starting at c:\files and outputs the results to results.csv.  One thing to note is that the scan stores all data in an array in memory, the is because the PowerShell Export-Csv function does not support appending to a CSV file (you gotta wonder what Microsoft talks about in design meetings).  I will likely create a version of the script that uses the out-file function to write each row to the csv file as the scan happens rather then storing in memory until the scan is completes and then writing the entire array to the report.csv file, goal here is to reduce the memory footprint during large scans.

The output of this file script will be similar to the following:

Blackberry Issues

So about a week ago my Blackberry (8900) booted to a screen stating “Error [some number I can’t remember]:  Reload OS”, obviously not good.  So I broke out JLcmder (a must have for all hard core Blackberry hackers), and proceeded to wipe and OS and reload my BB.  Yesterday afternoon I am sitting at my desk and I look down at my Blackberry and it is sitting there with a white screen, nothing but a white screen.  I try a soft reboot, and back to the white screen, I try a battery pull, back to the white screen.  Then in not my finest moment I some how rationalize that running over to the T-Mobile store will be the easiest/quickest fix, they waste a solid 30 mins of my life pulling the battery repeatedly and praying that it will boot, I will never get that 30 mins back.  As usual I returned to the office hooked the BB up to a laptop to see if I could connect from JLcmder, no luck.  I removed the battery, sim card and my MicroSD memory card, replaced the battery and rebooted, my BB returned.  I then stated scouring the forums, turns out that a few others had seen an issue with a corrupted SD card that caused the white screen of death.  Last night I formatted my SD card (fat32) and placed it back into my BB and the phone booted fine (happy about that).  When will I realize to never call my carrier for technical support (I have been with Verizon, AT&T and now T-Mobile and they are all the same. When you go to a BB specialist and the first thing they tell you do is pull your batter, you have to wonder how special he or she is.)  Hope this helps someone.

What have I been up to… Project Hive…

Obviously my post frequency has dramatically decreased this is due to a couple of factors.  First I am busy so I have less time to turn my experiences into easy to digest blog posts and second myself and a few of my comrades have been developing something we call “Project Hive” .  As you can probably tell from many of my blog posts most of my work in recent years has been associated with EMC technologies.  Throughout the years we realized that while there are some good framework tools out there they are costly, require significant customization and often don’t solve the common day-to-day operational issues that system administrators face.  The goal of “Project Hive” is to dramatically simplify the common tasks associated with managing EMC technologies.  Being intimately familiar with these tasks we have developed a platform that is based on a distributed collection, aggregation and presentation, we call this the “Honeycomb”, each Honeycomb contains modules, we call these “Workers” which are responsible for the collection, aggregation and analysis of data from discrete infrastructure components, all workers are centrally managed on the Honeycomb and use standard based methods to collect data (i.e. – WMI, SSH, SNMP, APIs, etc…).  “Project Hive” is a very active project and we are continually adding functionality to existing workers and building new workers as time permits or requirements dictate.

Any EMC customer who has been through an upgrade is familiar with the EMCGrab process (the process of running the EMCGrab utility on each individual SAN attach host within the environment and providing the output to EMC so they can validate the host environment prior to the upgrade).

In a reasonably sized environment this process can be tedious and time consuming, one of our released workers centralizes and automates the EMCGrab process.  I recently created a video which contrasts the process of running an EMCGrab manually on an individual host vs. using the Hive Worker.  My hope is to publish more of these videos in the future but as you can imagine they take a bit of time to produce.  If you are looking for more information contact the Project Hive team at

A hi-resolution video is available here .

New Navi Look-and-Feel

Navispphere emerging from the dark ages with a new look-and-feel
Key takeaways from Navisphere Birds-of-a-Feather session:
  • New Navisphere will adhere to the EMC Common Management Initiative
  • Task focus UI vs old object based UI
    • Improved Navigation, multiple entry points, drill down
    • Improved scalability
  • Summary pages with aggregated data
  • Hardware diagrams with exploded views
  • Tables
    • Costomizable
    • Exportable
  • NaviAnalyzer will provide the ability to scope the logging (e.g. Only log NAR data for a specific LUN, RG, etc…)

In the first Navisphere release NaviAnalyzer will not conform the Common Management Interface.

Classic NaviCLI is going away completely in the next release of Navi so UPGRADE YOUR SCRIPTS to NaviSecCLI if you have not already.

Overall the screenshots look pretty good.


Symm V-Max Overview with Enginuity Overview Session

Below are my notes from the Symm V-Max Overview with Enginuity Overview session, please excuse any typographical errors as I posted this as quickly as possible.  If you see any content errors or missing information please comment.

  • Symm V-Max and V-Max SE
    • Distributed Global Memory
  • Symm V-Max Configuration Overview
    • 1 – 8 V-Max Engines
    • Up to 128 FC FE Ports
    • Up to 64 FiCON FE Ports
    • Up to 64 gigE FE Ports
    • Up to 1 TB of Mem
    • Up to 10 Storage Bays
    • 96-2400 Drives
    • Up to 2 PB of total capacity
  • Director Boards Populated from 1 – 16 bottom to top
  • Engines are populated from inside out
  • Symm V-Max SE
    • Singe V-Max Engine (Engine 4)
    • Up to 16 FC FE Ports
    • Up to 8 FiCON FE Ports
    • Up to 8 gigE FE Ports
    • Up to 128 GB of Mem
    • Upto 120 Drives
  • V-Max Engine Overview
    • 2 Director Boards
    • Redundant PS, battery, fans, etc…
    • FA and DA on V-Max Engine
    • Backend I/O module supports up to 4 DAEs
      • 4 Backend I/O modules will support up to 8 DAEs in a redundant configuration
    • EFD support for  200 and 400 GB drives
  • FC I/O modules, FiCON I/O module, iSCSI/gigE I/O module
  • Memory Config
    • 32GB, 64GB, 128GB options
    • Memory can be configured by adding memory to a existing V-Max engine or adding additional V-Max engines
    • Memory is mirrored across V-Max engines for improved availability
  • Hard to show the picture I am looking at but the rear of the V-Max engine chassis has the following ports
    • Virtual Matrix interfaces
    • Backend I/O modules interfaces
    • Front End I/O modules interfaces
  • Symm V-Max Matrix Interface Board Enclosure (MIBE)
  • Each V-Max Engines can be directly connected to 8 DAEs
    • Depending on the configuration up to two additional DAEs can be daisy chained beyond the primary DAE
  • V-Max provides 2x the ability to connect directly to drive bays over the DMX, from 64 drive loops to 128 drive loops
  • I/O Flow
    • Director board has 2 quad core procs
    • CPUs are mapped as A-H slices

The market is depressed it’s not the “Depression”

The market had a huge day today with the Dow rising 936 points!  Following the biggest rise in the markets history I thought I would post a few figures from Friday for all the people who pulled thier money out the market last week in a panic and who most likely are questioning the move after today.

As of Friday’s closing bell the US martkets were down ~ 18%, EU markets ~ 25% and the Asian markets ~ 30%.  While an 18% slump sucks is it enough to make me panic?  Well let’s look at a few additional indicators today vs. the "Depression"

  Today "Depression"
Industrial Production

Down ~ 1% Down ~ 44%
Unemployment ~6% ~ 25%
Housing payments in the rears ~ 4% ~ 40%

The market conditions today are a long way from the conditions in 1929.  Things look much closer to the market crash of 1973/74 where the Dow lost ~ 45% of its value.

Now let’s look at an investmetment made in 1972 and present day value.  This would be representative of a thirty somethings investment today with a retirement target cashout.

I looked at an investment in XOM made 01/70 and sold today, even following last week the gain is ~ +843%.  The Dow inthe same time perios is ~ +1060%.

I also looked at the past 10 years from Oct 1998 til today which shows a ~ +107% for XOM and ~ +19% for the Dow.

Conclusion, the market is not a place to put your cash if you need liquidity.  As a long term investment the market is the place to be!

A rant on the economic situation…

Let me start by saying this post has little to do with IT at least not directly.

Anyone who knows me knows that today is Friday which is “Dadddy & Maddy date night”, for those of you who don’t know me on Fridays my wife gets to take a break as Mom and enjoy some adult interaction while I get to enjoy my daughters company, usually at Johnny Rockets.  I usually get a hamburger and Maddy gets a hot dog or chicken fingers and we talk about how her week was.  Lately she talks a lot about her good friend Ashna who she seems very fond of.  Tonight we had a really nice dinner and now we are home to “Party Like Rock Stars” (Her words not mine, this is how she articulates staying up and watching a movie with daddy, tonight’s movie selection is Narnia.  With some time on my hands while she enjoys the movie it gives me an opportunity to express my anxieties using the written word.  Anyway this brings me to the point of this post.

For the past 2 weeks I have been listening to people complain about the market.  Today’s train ride into NYC was especially irritating and it seems with each passing hour I get more irritated.  What kills me is in a typical American, spoiled brat like, I want it all fashion every comment places blame on “Wall Street”, corporate CEOs, predatory banks, yada, yada, yada!!!!!  I have yet to hear someone say man I f’d up, I am under a mortgage payment I really can’t afford, I should have known this when I singed up for that Power Option ARM.  I have two BMW lease payments that are nearing their 36 month term, I should have just bought a couple of used Honda Civics, etc… what was I thinking.  My only option is to pull my money out of the market now at a massive loss.  I can’t blame the market for this only my own stupidity, this would not be as big of a deal had if I just exercised better judgment and some prudence.  The following are examples of what I am hearing when listening to people talk about their perception of the economic situation.

  • People seemed to think that the market was a high yield savings plan made available to the upper middle class and the wealthy that would continue paying ridiculous returns forever therefore they took a unhealthy position.
  • I make 35k (example) a year, 2K a month net, I have no savings and a $500 a month car payment but this nice man in blue pinstripe suit is telling me I can afford a 300K mortgage.  The guy in the suit may be a predator but you let yourself be his prey.  His greed and your greed screwed all of us!
  • I sunk all my money into real estate as an investment vehicle assuming the real estate market would continue to endlessly climb.  To compound the issue I used the financial programs developed by the predatory bankers, physicists and mathematicians and completely disregard historical real estate market volatility.  BTW – I have no savings or other investment vehicles and I was relying on the historically documented volatile real estate market defying history this time around.
  • I did not find it odd that my monthly mortgage payment was less than the loan amount divided by 360 (BTW – since I have little faith in human beings these days 360 months is a standard 30 year mortgage term).
  • etc…..  The list goes on and on.

How depressing is the stupidity depicted above.  To me it is far more depressing than the money I am losing in the market right now.  To quote Ron White, “You can’t fix stupid”.

The bottom line is that if you are one of these people you either hoped to rent a life above your means forever or maybe you thought you would rent for a while with an option to buy in the future.  I am pissed off, the idea or re-capitalizing mortgages so people can stay in homes they could not afford in the first place makes my blood boil!  The Fed is now developing widespread social welfare programs to attempt to resurrect our Stupocracy.  Regulation, social welfare programs and whatever other rhetoric the Fed comes up with will not be an answer for the epidemic stupidity plaguing our countries citizens.

Economies are based on the simple concept of supply and demand.  Sometimes supply creates demand and sometimes demand has a way of finding a supply.  In the case of the financial programs of the past few years let’s not forget that the demand never slowed.  The banks were in a state of economic euphoria so they were not thinking straight, but the vast number of consumers consuming these obviously dangerous products like power option arms, interest only loans, no income verification loans, 50 year fixed rate mortgages, etc… consciously disregarded logic to satisfy greed in many cases.

If we continue on our current path of having our cake and eating it too we are doomed as a country!  We are in a downward spiral because the prophecy is self fulfilling, too many people have leveraged positions in stock market and real estate when they should not, this is forcing massive sell offs because people need cash.  There is no quick fix, politicians are in full bullshit mode with the election 30 days away they want the uneducated and impressionable listening to feel good about their rhetoric when in reality they have no idea what to do about our financial crisis.  The fix here is rational and prudent behavior, yes it is going to take time to undo the irrational behavior of the past but the only way to fix the problem is to sleep in the bed we made.  This is not a domestic problem, the world economy is suffering as bad and in many cases worse than we are.  Only we can fix this mess we made, take some damn ownership and change your lifestyle!

I have carved out a decent life for me and my family, managed to intelligently accrue a decent nest egg which is loosing double digit figures almost daily these days but I am hoping I am intelligent enough and liquid enough to try to capitalize on this.  I am amazed by the 30 somethings in a full panic, taking major losses when they should be viewing themselves as having 30 years of runway to recoup their losses, pushing more money into the market today and lowering their cost basis and decreasing the time required to recoup their loses.  Instead unhealthy positions have many 30 somethings in a situation where absorbing the loss and taking the cash is the only option.

The market is not much different than the blackjack table, but in the market counting cards is welcomed, encouraged and expected.  It is all about being cool, waiting for the shoe to turn and having the bank roll to capitalize when it does.  If you panic, become greedy or spend your bank roll to early you may leave injured!

Tighten your belts, double down on your investments and there may be something at the end of the rainbow.  This is my advice and I am taking it, I am cutting back on personal expenses to double my monthly cash into the market.  This is time for measured investment, dollar cost averaging, reducing your cost basis and buying and holding as the market stabilizes.  Never forget the bottom is 0, but if that happens the cash you pull out of the market will be worth 0 as well, remember the tender you are walking around with is only as valuable as our capital markets, there is no gold bullion backing that paper!

Keep a positive perspective, the markets will turn and hopefully you will be there to benefit in some positive way.

Thin Provisioning 101

I was asked by someone for a quick overview of Thin Provisioning and when I view this as an valuable or applicable technology.

So let me start with a quick simplified visual overview of both Thin Provisioning and Traditional (aka Thick)Provisioning.


So now that you understand the concepts of Thin Provisioning and Traditional (aka Thick) Provisioning let me quick talk about the only where I see Thin Provisioning as a valuable technology.

I look at Thin Provisioning in the same way that a Disaster Recovery (DR) provider looks at taking on new customers.  If storage is your business (i.e. – You are offering a shared storage model to customer co-located in your data center) then Thin Provisioning may be a key ingredient to your business model.  Let me expand on this, DR providers like SunGuard oversubscribe their data centers hedging that 100% of their customer will not have a disaster at the same time (BTW this has happened and put the provider out of business).  Thin Provisioning works in the same way by providing the user with the belief that they have 100% of the capacity while in fact the capacity may be over provisioned and the storage service provider (SSP) is hedging that 100% of the user co-located on the storage array will not demand 100% of the resources at the same time.

There are some very minor management benefits that I outlined in the pictorial above but IMO given some of the pitfalls associated with Thin Provisioning these do not provide a compelling reason consider Thin Provisioning.  Read and interesting article here that outlines one very real issue encountered with Thin Provisioning and NTFS.

So in conclusion if you are SSP of some sort consider Thin Provisioning otherwise go thick or go home 🙂