#27 – Distributed tracing with Istio on AWS

Ever wondered how you troubleshoot among ephemeral microservices in a container environment? Well, by tracing service interaction, that’s how! And I’m not just referring to outages here, this is also the secret to tracking down performance issues and inefficiencies among microservices.

In this episode I’m joined by Neeraj Poddar from F5 Innovation, Aspen Mesh! If you haven’t caught our previous episodes from the Aspen Mesh, Shawn Worke (their fearless leader) explains what its all about in #23 – Aspen Mesh w/ Shawn Wormke

But if you’re pressed for time and ou just want to know how to trace microservice interactions with Istio on AWS, then waste not another minute and watch this episode:

Thanks for joining us, Neeraj! Youy can find his great blog titled Distributed tracing with Istio in AWS <= there. #26: WWT on why Super-NetOps is different

Some great content from friends at World Wide Technology (WWT) explaining how the Super-NetOps Training Program differs from traditional infrastructure automation training.

“Its a very interesting and very exciting program”
– Mark Wall


In this podcast, live from the WWT Global Sales Conference in Las Vegas, Principle Solutions Architect, Joel King, and Practice Lead for Application Delivery, Mark Wall, explain what the Super-NetOps training program is all about and why its so important.

From the podcast
On the challenges organizations are facing, Mark Wall shares, “What it really boils down to is the infrastructure, the networking infrastructure, traditional IT Operations folks really have a very hard time keeping up with the development side of the organization.”

Wall continuous, “Theres a big gap in how the infrastructure side is able to deliver services that are required by the development organization. So, it almost boils down to ‘I can’t keep up‘, ‘I cant deliver those services fast enough‘ or ‘I don’t really understand how my piece, my networking component, my application delivery component, fits into the public cloud or into this automated process’ and there’s kind of a disconnect in there.”

“That skillset gap is something that all of our customers ask about, and they struggle with. Traditional network engineers don’t have the programming background. They don’t have the understanding of some of the technologies around different data structures.” adds Joel King, “How do I train those engineers to be able to have the skills that they need to do the type of automated deployments that Mark’s talking about. That’s a big key area for many of our customers is, what skills do I need, what training, what education do I need, and thats one of the things that the Super-NetOps program is trying to address. To enable those engineers with new skills to be able to be successful”.

WWT is supporting the Super-NetOps movement by putting together enterprise environments that support multiple technologies, beyond just F5’s BIG-IP, to help customers build end-to-end solutions.

As I stated in my DevOps Enterprise Summit talk last year, “Give me a Swiss Army Knife, and MacGyver I do not become”. So, its great to see Mark and Joel building upon the initial Super-NetOps program and adding huge value to their customers.

Enough from me! Make time to listen to this great podcast here: WWT on Super-NetOps #25: MiniKube for Dev’s

Another great episode from our F5 Incubator friends, In this episode I’m joined by Sr Architect, Andrew Jenkins, who explains how, in mere seconds, he spins up Kubernetes clusters on his laptop in a docker container.

An awesome solution for rapid dev/test environments that I will be adopting as soon as I’ve hit post on this article.

Here’s the video with Andrew:


And here’s a blog Andrew put together with the technical details: Building Istio with Minikube-in-a-Container and Jenkins

Many thanks for joining us on, Andrew. I look forward to hearing from you again! #24: Becoming Super-NetOps: Day 1

If you’re right at the beginning of your Super-NetOps journey, then this article is for you!

A common question from our NetOps pals is, “How do you get started?” Well, let me take a stab at that in todays article. Before we begin, a few things to keep in mind.

  1. EVERYONE is starting from a different level. Focusing on who’s ahead of you in the learning path will only distract you from your success. This journey is about you. Go at YOUR pace.
  2. Making mistakes is good. Mistakes are evidence that you are learning new things and have the confidence in yourself to evolve. Learn more about a great culture for innovation from John Allspaw, here. <- A good one to share with the team!
  3. Beyond the tools and scripts, pay close attention to how you change your approach to troubleshooting and moving forward with a solution. Often overlooked, learning to automate without adopting and nurturing new practices and culture is almost pointless. Take the time to see how process changes and how you begin to look at challenges differently.

Ok, keeping these points in mind, here’s some new concepts to look into:

1. Understand RESTful interfaces

If this is your first time venturing away from the GUI/CLI, or maybe you just want a refresh, I recommend you watch this great REST API introduction video posted by WebConcepts. In this video you’ll see how you can communicate with popular on-line services including Facebook, Google Maps, and Instagram via their REST APIs:


2. Interact with a RESTful Interface

There are many tutorials on the internet that show how to communicate with a RESTful interface from a scripting language, like Python or Javascript. But what if you don’t know the scripting languages they refer to? Sometimes its best to avoid an overload of too many new concepts to learn at once.

For this very reason, I tend to direct people to the awesome, multi-platform REST client, POSTMAN:

While their messaging does target ‘API Development teams’, its fantastic for API beginners, too. With POSTMAN installed, I recommend you watch the great tutorial “How to use the POSTAN API Response Viewer”:


Once you’ve worked through the basics, I recommend going through the POSTMAN video tutorials to learn some of the time-saving features you’ll come to depend on:

3. Troubleshooting JSON

Now that you’ve had some interaction with a RESTful interface, you’ve probably had some experience with how a small error can break things. Fear not, while you’re on your path to becoming JSON-fluent there’s always the great on-line JSON validator,

Simply ‘paste’ your misbehaving JSON data into the text field and click ‘Validate JSON’. Below its showing me that I missed a comma on the end of the second line:


We’ll look at more data formats, like YAML, in future posts.

4. Conclusion

If you’ve worked through these exercises then congratulations is in order. You have already begun your journey towards becoming a Super-NetOps engineer! If you’re still feeling a little overwhelmed with these concepts, there is no shame is working through them again from the beginning. Repetition builds expertise and being comfortable with change is all part of the journey!

Next in the series we’ll look at some more advanced POSTMAN features and then take what you’ve learned in POSTMAN and apply it to scripting languages. #23 – Aspen Mesh w/ Shawn Wormke

Want to learn about Service Mesh? Well, you’re in luck! Today I am joined by Shawn Wormke, Sr. Director of Aspen Mesh – – an F5 Networks Incubator innovation.

Aspen Mesh is an ‘enterprise service mesh’ built on Istio. In her recent blog, industry thought leader, Lori MacVittie (@lmacvittie), announced the drivers behind launching Aspen Mesh with:

…we believe a robust microservice communication fabric is the best possible path to scaling containerized apps whether in the data center or in the cloud (or both). But we also understand the needs and complexity of enterprise production environments. A service-mesh needs to do more than just scale apps; it also needs to monitor and secure them.

Watch this video to understand the future of Aspen Mesh: the microservices architecture its destined for, and the use cases it will address regarding the application of dynamic policy and service visibility through continuous feedback.

Thanks for joining us, Shawn!

REDtalks #22 – IBM Cloud with Simon Kofkin-Hansen

In this episode I’m joined by Simon Kofkin-Hansen of IBM Cloud.

As Distinguished Engineer of Cloud Automation, Simon travels the world enabling organizations a smooth transition towards their hybrid cloud goals. Watch this episode to understand some of the realities of a journey to cloud, what to look at for, and what you might want to do differently to make the most out of your newly adopted operating model. HINT: automation is key!

Thanks for joining us, Simon! Great to have you on

REDtalks #21 – Super-NetOps Live Q&A

UPDATE: incase you missed it, see a recording of the “What is Super-NetOps Live Q&A” here:

On October 24th, at 9am PST (UTC -7) we will be hosting a Live Q&A forum on Super-NetOps (NetOps for DevOps) – click on the video below for details, or open directly on YouTube here.

Whether you’re a total beginner to infrastructure automation, or well on your way with Programmable Infrastructure, bring your questions for our Super-NetOps champions to answer. Simply enter your question into the ‘live chat’ window now, or during the live broadcast.

For some background, we’ve covered Super-NetOps in previous articles including Does DevOps need a “Super-NetOps”? and REDtalks #19 – Super-NetOps Explained. Due to the overwhelming interest we’re hosting this Live Q&A session to help accelerate your journey.

Live Q&A will commence October 24th, 9am PST (UTC -7).


REDtalks #20 – Programmable Infrastructure 101 – be Declarative

While the payoff of a successful implementation can be HUGE, the beginning of your Programmable Infrastructure journey may be difficult and fraught with danger. There are decisions you need to make right from the start that will dictate the experience you and your customers, external or internal, encounter. To avoid ‘kicking an operational nightmare can further down the road’ (or into the orchestrator as it may be), ensure you are employing declarative API interfaces and appropriate levels of abstraction in your automation tooling.

In this episode I explain the importance of declarative API interfaces. Trying to move all of the knobs and buttons of programmable infrastructure into your deployment tooling WILL come back to bight you later. It will be costly to support, it will create an integration too rigid to adapt to new systems and services, and it will increasingly inhibit business agility as time goes on.


Thanks for joining us on the 20th episode of

REDtalks #19 – Super-NetOps Explained

Due to popular demand, here is a short (16 minute) video explanation of the Super-NetOps persona.

I frequently give talks on the need for Super-NetOps engineers and architects to support modern development/DevOps processes. However, that talk (especially with enthusiastic Q&A sessions) takes about 90 minutes. Hence, below is a quick ‘why you need Super-NetOps’ version I put together for those in a hurry.

Brilliantly summarized by my colleague, Joe Cassidy, “Great short video by Nathan Pearce explaining the “Super-NetOps” concept wherein NetOps embraces principles and practices of DevOps to reduce the skills gap between Infrastructure and Developer silos. Like DevOps, “Super-NetOps” is not a technology, tool or person but rather a culture change where infrastructure teams adopt “systems thinking”. This shifting focus towards “systems thinking” empowers the traditional infrastructure silos to collaborate, continuously improve delivery, enhance communication, enable self-service and speed development. F5 is helping enterprises embrace this concept by (among other things) enabling APIs, exposing Declarative Interfaces and executing on Self-Service Models and Templates.

Thanks for listening!


REDtalks #18 – Enabling the docker TCP API in AWS

Not a traditional REDtalks post today (no interview/demo), but this took me a while to work out so I thought I’d share.

What’s this about?

It all started with me building REST extensibility solutions for F5 Networks in AWS. I created (Launched) a new AWS AMI Linux instance – yep, the very first one on the list: “Amazon Linux AMI 2017.03.0 (HVM), SSD Volume Type“.

Next, I followed the AWS instructions to install docker:

sudo yum update -y

sudo yum install -y docker

sudo service docker start

sudo usermod -a -G docker ec2-user

docker info

NOTE: Full docs here:

This is where I got stuck!

As part of the solution I needed to issue a docker command on the docker host, from inside a container… Ok, Batman, to the Google-copter…

There’s loads of suggestions out there to map /var/run/docker.sock into the container using -v. For example:

$ docker run -it -v /var/run/docker.sock:/var/run/docker.sock my_container sh

With this you can execute:

$ curl --unix-socket /var/run/docker.sock http:/containers/json

HOWEVER, there are loads of forum posts saying to be real careful about mapping /var/run/docker.sock to all you containers…

What to do?

Enable the API over TCP! 

Back to the Google-copter, there are a few posts out there about getting it running on Ubuntu but none for the Linux AMI distro…

A solution (hours later…)

1. Change some startup options:

The default ‘OPTIONS’ in /etc/init.d/docker is:


we need to change this to:

OPTIONS="${OPTIONS:-${other_args}} -H tcp:// -H unix:///var/run/docker.sock"

So, you need to go ahead and edit that to something link this:

$ sudo vi /etc/init.d/docker

OPTIONS="${OPTIONS:-${other_args}} -H tcp:// -H unix:///var/run/docker.sock"

2. Restart docker for these options to take effect:

sudo service docker restart

Now you have enabled the docker API over TCP! #w00t

Test the API

Lets get the API version:

curl http://<ip_address>:2375/version

NOTE: Replace <ip_address> with the IP address of the docker host, or its hostname!

The response will look something like this:


Note the:


Now add that version number to the beginning of the URI, slap json on the end of it, and presto:

curl http://<ip_address>:2375/v1.24/images/json




Now you can go read this:


CAUTION: One last step, and this is REALLY important! Don’t leave your Docker API open on the Internet!