Those who know me, know I like to write, they also know I like to use the phrase “operational elegance”. Being elegant is not as easy as it may seem, after all, we are human, and our human nature generally does not allow us to persistently conduct ourselves like Claire Underwood. This is a good thing. As I was thinking about the value of being operational elegant, Claire Underwood popped into my head because let’s face it, she is about as mechanical as it gets, but most human beings can’t rely on themselves to function like Claire Underwood. Generally speaking, manual execution is usually accompanied by mistakes.
Last week I had the opportunity and pleasure to present to our sales team at their national sales meeting, and operational elegance was one of the topics I touched on.
Sometimes this concept can be hard to visualize, luckily while on my way home from Phoenix, I received a Slack message from a project manager on our team which provided the fuel for this musing. Her message was to share some operational elegance, operational elegance that I thought was worth writing about and sharing. In addition to liking to write, disliking brute force solutions and obfuscation, I also value DNA over resume and innovation over imitation. What was great about this Slack message was how it spoke to me about how far we have come, how it is now part of our DNA to innovate and elegantly solve problems. Innovation is happening organically, team members are looking at problems and working to address them elegantly, they are not satisfied with just getting things done, they are incubating new solutions and iterating on them until they achieve elegance. If I were the sort of person to ever claim success, it would be because of changes to the organizational genome that look like this.
We are about a month away from our 2019 team kickoff, last year the theme of our kickoff was #PUSH. The idea was to challenge everything and push our boundaries to become more operationally elegant, accept failure as a means to push ourselves further and faster. The output from last years kickoff was inspirational, many team members who had never written a line of code showed up with impressive projects showcasing innovations that combined their mechanical, electrical, computer and software engineering skills to elegantly solve real-world problems, that were meaningful to them, like how to deliver a treat to your puppy while not at home. Engineering an operational elegant solution that addresses empathy, how do you not love this!
This year for our 2019 kickoff challenge we will have two teams building 1/16th scale autonomous vehicles. This project is about teamwork, creativity, and developing technical skills that extend beyond marchitecture in areas that are driving our industry, like AI/ML/DL, Keras, Tensorflow, etc. Everyone from project managers to engineers has to participate because everyone has the ability and desire to develop skills because they possess the DNA that says the more I know the better I can be; knowledge gives way to perspective, perspective and know-how give way to elegance. This is why DNA reigns supreme over resume. As a team, we have a responsibility to push each other and to help each other grow.
Innovation and operational elegance are not things you do; they are a way of life. Looking at a problem, being frustrated by inefficiencies, brute force tactical solutions, and taking ownership of developing a more elegant solution and continuously improving that solution requires a mindset.
When we think about project management we think about organizing a work effort, managing budget, risk, resourcing, stakeholders, expectations, escalations, communication, etc. But I believe there is room for operational elegance here, where the process becomes automated and measurable. Not only do I believe this, but I also have examples that prove when you have someone with a broader perspective, frustrated by a lack of elegance, with the know-how and freedom to innovate they can quickly convert vision to reality and deliver operational elegance.
This brings me to the latest example of operational elegance and the catalyst for this blog, our very own code loving project manager. Two years ago when I interviewed this individual we had a really good conversation. Having spent most of her career in academia she was a bit frustrated by the politics and ready to move into the private sector and take on new challenges. This person would be working remotely so being self-driven was important, and I certainly got that vibe during our conversations. What I really liked about her was that she was a programmer who moved into project management because she had lost her zest for programming, much of this was due to how so many legacy institutions apply rigid constraints on developers, sucking all the fun out of the creative process. The great news was she possessed the skill, and we possessed the environment to reignite something she once loved. Our culture is one that focuses on outcomes, documentation, reuse, and elegance but polyglots are welcome, even those who chose to speak PHP. 🙂 The focus is always on solving problems, maximizing reuse, removing constraints like solve this complex problem inside these fifty parameters. The world has changed and it favors the innovator over the imitator, to innovate effectively the innovator requires creative freedom. Fast forward to today and this individual is thriving on the freedom to innovate, she is looking at problems, innovating and solving them elegantly on her own. She is detailed, she values the process, she is convicted, she is willing to change and adapt, but not without a battle. 🙂 She will expect you to be as committed as she is.
Last year we took on a project that required us to build a process and systems to support a customer with 300 sites in 52 countries in 41 different languages. We had never done anything like this, we identified a call center partner and began to build a process, but the year was filled with challenges, the provider was not executing very well and worse they could not provide is detailed metrics so we could identify where the failures were occurring, this really put us in a bad position. Our approach was to be transparent with our customer, we started a search for a new provider, once we decided on a new provider we worked on a transition plan and focused on addressing some of the issues we could identify using our year of experience. We developed and introduced IVR (Interactive Voice Response) to reduce call abandonment, we focused on CDRs (Call Detail Records) so we could provide detailed reporting to the customer, see where we were having issues with hold times, call abandonment, etc. While our delivery in 2018 was less than stellar due to third-party provider constraints and failures our transparency and desire to invest in building a better service for the customer is what makes us a good partner and created the opportunity for us to improve our quality of service and grow our business 2019. We are not solving problems that have been solved thousands of times before, we are innovating solutions for our customers that provide a competitive advantage; innovation requires a partnership where we plan for failure, where we take the necessary precautions to limit the blast radius, where we iterate and incrementally improve our execution.
So today we have entirely rebuilt the delivery process, we have detailed metrics being provided by our third-party call center and language translation provider, but it doesn’t stop there, we want to ensure that the path to transparency is efficient, automated, consistent, auditable, and elegant. We should not be downloading data, pulling it into Excel, manipulating it, and emailing weekly reports to our customers because that is inefficient and suspect AF! APIs (Application Programming Interfaces) to the rescue! The automation of collection, aggregation, and presentation of information with the right skills and perspective is easy, the line between inefficient and suspect AF, and elegant and transparent AF depends on DNA that compels people to recognize that there has to be a better way to do something, the knowledge to possess this perspective, the desire to challenge yourself and execute, and the wisdom to recognize that Charles Babbage built the difference engine to automate a repetitive task.
We should strive to deliver EaC (Everything as Code), because elegance is rooted in our ability to remove humans from executing repetitive tasks, allowing the human mind to focus on reasoning through complex problems which are not binary and highly repetitive.
The heavy lifting might look something like this:
<?php
/*
The purpose of this program is to populate the following table in ServiceNow
with data from the IVR Call Log.
Table Name: Call Center Data (u_call_center_data)
u_call_center_uuid String (empty) 50
u_call_incident Reference Incident sys_id
u_call_language String 50
u_call_time Date/Time 40
u_caller_did String 25
u_did_country Reference Country sys_id
u_duration_in_seconds Integer 40
u_ivr_path String 25
u_source_number String 25
*/
/*
config.php contains the access information to the call center and servicenow
instance
CALL_CENTER = call center url
CALL_CENTER_KEY = call center key
SERVICE_NOW = servicenow url
SERVICE_NOW_USR = servicenow user
SERVICE_NOW_PWD = servicenow password
*/
$config= (include_once('config.php'));
$records= getCallCenter();
foreach($records['rows'] as $record){
$parsedRecord = recordParsing((array)($record));
$parsed_uuid = ($parsedRecord['u_call_center_uuid']);
/* Going out to Servicenow to verify that the record does not exist already.
The u_call_center_uuid should be the unique identifier
if it does not find any records, we will add it.
*/
$verify = verifyUniqueRecord($parsed_uuid);
var_dump($verify);
if ($verify == false){
/* Unique record was found so processing will continue by
obtaining the country that matches the DID number. The country and
DID are contained in the cmn_location table.
*/
$country = findCountryFromDid($parsedRecord['u_caller_did']);
if($country > ''){
$parsedRecord["u_did_country"]= $country;
}else{
/* DID Number without a country.
will add an email process to have it verified
as a valid DID in our list. Continue to add
the record without it though.
*/
postError("Country could not be located for DID. Data includes: \n".$parsedRecord['u_caller_did']);
}
// Matching call center record not found so one will be created
if($parsedRecord['u_duration_in_seconds'] > 120){
/*
If the call reached OpsGenie, it creates an incident.
We want to attempt to match the call with the incident.
*/
$incident = lookForIncident($parsedRecord['u_call_time']);
if($incident > ' '){
// if incident was found, attach the incident number to the record.
$parsedRecord["u_call_incident"] = $incident;
}
}
// post to our ServiceNow
$post = postToSNow($parsedRecord);
}
}
function getCallCenter(){
global $config;
// Basic curl call to the call center. Return's all records.
$curl = curl_init();
$url = $config['CALL_CENTER'].'apikey='.$config['CALL_CENTER_KEY'];
curl_setopt($curl, CURLOPT_HTTPGET, 1);
curl_setopt_array($curl, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_FRESH_CONNECT => TRUE,
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_HTTPHEADER => array(
"accept: application/json",
"authorization: Basic ############################",
"content-type: application/json"
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if (($err) or $response === '') {
echo "cURL Error #:" . $err;
postError("Error obtaining data from call center: \n".$err);
return 'Error';
} else {
$records = (array)json_decode($response);
return($records);
}
}
function recordParsing($record){
// parse the record from the IVR log to build the ServiceNow record.
$parsedRecord = array(
"u_call_center_uuid"=>$record['uniqueid'],
"u_call_language"=>$record["lang"],
"u_caller_did"=>$record["did"],
"u_duration_in_seconds"=>$record["duration"],
"u_ivr_path"=>$record["path"],
"u_call_time"=>$record['calldate'],
"u_source_number"=>$record["src"],
"u_domain"=>'############################'
// Getting the country based on the DID Number
);
return $parsedRecord;
}
function verifyUniqueRecord($parsed_uuid){
/* Verify that the record does not exist in ServiceNow
by looking for the record's unique id.
*/
$query = '?sysparm_limit=1&u_call_center_uuid='.$parsed_uuid;
$return = service_now_call('u_call_center_data','',$query,'GET');
if($return['Error'] > ''){
postError("Error verifying duplicates in ServiceNow. Data includes: \n".$parsed_uuid.'\n'.$return);
return false;
}
$record = (array)$return['result'][0];
// if there was a return, send in an 'a'
if($record["sys_id"]){
return true;
}
return false;
}
function findCountryFromDID($did){
/*
$did = string
We will send in the did number from the call log and find the
country it is assigned to in ServiceNow.
*/
$query = '?sysparm_limit=1&u_country.u_msa_country_did_number='.$did;
$return = service_now_call('cmn_location','',$query,'GET');
if($return['Error'] > ''){
postError("Error finding country in ServiceNow. Data includes: \n".$did.'\n'.$return);
return false;
}else {
$result = (array)$return['result'];
$records = (array)$result;
foreach($records as $location){
$tlocation = (array)$location;
$country = (array)$tlocation['u_country'];
return $country['value'];
}
}
return '';
}
function lookForIncident($calldate){
/* Attempt to match the call to an incident based on the date/time.
Can adjust the number of ours to go back based on when the update last
occured by modifying $time
*/
$time = '3600';
$query = "?sysparm_query=openedRELATIVEGE%40hour%40ago%40".$time."&company=MSA%20-%20FedEx";
$return = service_now_call('incident','',$query,'GET');
if($return['Error'] > ''){
postError("Error querying for duplicates. Data includes: \n".$calldate.'\n'.$return);
} else {
// We found at least one possible record
foreach($return['result'] as $record){
$record = (array)($record);
$record = (array)($record);
// calldate needs to be greater than the incident record
$dateTime = new DateTime($record["opened_at"]);
if($calldate < $dateTime){
$calldate2 = new DateTime($calldate);
$dteDiff = $calldate2->diff($dateTime);
$minutes = $dteDiff->days * 24 * 60;
$minutes += $dteDiff->h * 60;
$minutes += $dteDiff->i;
if($minutes <= 3){
/*
Set for 3 minutes because there is a delay from the
call's start time until it reaches opsgenie/servicenow.
We are estimating 3 minute delay to find possible records.
If found, return the incident sys_id to the record.
*/
return $record["sys_id"];
}
}
}
}
return '';
}
function postToSNow($data){
// Post the record ($data) to ServiceNow
$return = service_now_call('u_call_center_data',$data,'','POST');
if($return['Error'] > ''){
postError("Error posting call log item to ServiceNow. Data includes: \n".$data.'\n'.$return);
return 'Error';
} else {
return $return;
}
}
function postError($data){
// If any error is detected, we want to create an incident
// $data contains the description of the error
// Building the Incident Record
$incident = array(
"short_description"=>"Error from Call Record procedure",
"category"=>"ServiceNow",
"description"=>$data,
"company"=>"############################", // us
"customer"=>"############################", // them
"caller"=>"api.user",
"opened_by"=>"api.user"
);
$return = service_now_call('incident',$incident,'','POST');
if($return['Error'] > ''){
// if we had an error on the posting of the error, send email to servicenow
$to = "foobar@service-now.com";
$subject = "Error posting error in call log process";
$message = "Original error: \n";
$message = $return."\n";
$message .= $data;
$header = "From:foo@bar.com \r\n";
$header .= "MIME-Version: 1.0\r\n";
$header .= "Content-type: text/html\r\n";
$retval = mail($to,$subject,$message,$header);
if( $retval == true ) {
echo "Message sent successfully...";
}else {
echo "Message could not be sent...";
}
return 'Error';
} else {
return $return;
}
}
function service_now_call($table,$data,$query,$type){
global $config;
$url = $config['SERVICE_NOW'].$table.$query;
$curl = curl_init();
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($curl, CURLOPT_USERPWD, $config['SERVICE_NOW_USR'].':'.$config['SERVICE_NOW_PWD']);
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_HTTPHEADER, array("Content-Type: application/json"));
if($type === 'POST'){
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, json_encode($data));
}
$response = curl_exec($curl);
$return = (array)json_decode($response);
if ($err) {
return array('Error' =>$err);
} else {
$return['Error']='';
// var_dump($return);
return $return;
}
}
?>
And deliver an elegant outcome like this:
Bravo to this individual for innovating, even if code is PHP. 🙂 But freedom is the fuel for innovation so great work and keep pushing!