Engineering Small


Stay focused on the engineering reality defined by your capabilities and requirements. Line up the engineering tasks with the business value. A shift in the engineering tasks should be a conscious decision as a result of conversations between the technical and business leaders in order to better deliver on the business value. Technical debt can be discarded when the engineering objectives change. Remember all technology is transient and that pride cometh before a fall.

it’s never happened here


its never happened

There are somethings that I dislike. The image above is a screen capture from a conversation I was part of recently and said by someone who I respect deeply. On this point, we disagree deeply however.

Modern DevOps culture demands a great deal of pragmatism. In particular it demands that you don’t solve problems before they become problems. It’s the benefit of working in a non-critical environment. We’re told that ‘just-in-time’ is the best fit for fast iteration and product development and who can argue with success?  I wonder however if we’re losing a key component of good Engineering, that of forethought.

The conversation was around GitHub protected branches and whether they were a good idea for some/all our repo’s. I think these are a good idea for all repo’s and selected branches, especially master, and doesn’t take long to setup nor much in the way of maintenance or documentation. I can also understand my colleague’s position that there’s no need to go enable this everywhere as nothing bad has happened yet that would suggest this would be a good protective measure.

To me that’s a bit like putting the safety on after pulling the trigger. The thought’s there but the execution is suspect. It’s too Dev and not enough Ops for me.

In operations/SRE or whatever you call it these days, the paramount responsibility is safety. Safety leads to uptime. Uptime leads to sales. Sales lead to money. Money leads to wages. Wages lead to whiskey. Operations engineers are there to make sure that the business is supported and protected at all times. Operations engineers are there to ensure that engineering is able to deliver but that the what’s delivered is supportable and operable on an on-going basis – even if the entire engineering team changes. There is I think a fine distinction here between enabling the business and enabling engineering teams. Those two enablements often require subtle differences in approach, tact and execution. It’s the fine line that sometimes trips you up.

On this occasion I left this well alone, no one needs this kind of debate on a Friday and it’s a small thing. But small things have undone great enterprises before so in the back of my head a warning bell will sound all weekend! I like to think I allow teams I manage a great deal of autonomy and the operations team haven’t expressed any concerns. Whether that’s because it’s easier not to is up for debate – but not today.

senior platform engineer


I am an Engineer

I work for TES Global. TES Global’s story is an extraordinary one: its digital community on TES Connect is one of the fastest growing of any profession globally, and it boasts a 100-year heritage at the centre of the teaching and education community, with offices in London, San Francisco and Sydney. Today, TES Global has over 6.9 million teacher members in 197 countries across the globe and connects more than 72 million teachers and students. Up to 6.3 million resources are downloaded from the site a week, 13 a second. Home to more than 800,000 individually crafted teaching resources developed by teachers for teachers, this unparalleled collection helps to guide, inform and inspire educators around the world.

The operations team at TES plays a big part in ensuring our platform is kept alive and healthy as well as working very closely with a range of developers to deliver one of the smoothest delivery pipelines I’ve had the privilege to work with. I’m now looking to expand my team and am looking for an individual who has current permission to work in the UK and will not require VISA sponsorship to join us as a Senior Platform Engineer (SPE).

SPEs are responsible not only for the underlying hybrid cloud infrastructure but also for networking, security, performance, automation, integration and education of the wider internal developer community in the best practices of using our infrastructure and services. They are the last but one line of support and assistance and as such are expected to understand their subject matter deeply and thoroughly. It can be an incredibly challenging role but one that we all feel is worth doing and worth doing well. We all work in an Agile manner and believe in effective communication as the single most important tool at our disposal.

The TES platform lives within the AWS public cloud and an internal Xen based stack. SPEs are expected to be deeply familiar with both these stacks and to provide interfaces for developer operations that are seamless between the two.

We make extensive use of Docker, Ansible and Python and our SPEs are expected to be familiar with these technologies and to become extremely proficient with them. You’ll be supported each step of the way on that journey.

We work closely with the wider engineering teams to share our experiences and knowledge to continuously improve the technical operations procedures, tools and approaches throughout the business.

What’s it like to work here?

We release regularly and frequently here at TES and that means our systems and our infrastructure has to be as dynamic and robust as the engineers creating the features. This isn’t a 9 to 5 job and we work the job and not the clock. TES values it’s people highly so there are various social events regularly scheduled to help maintain a healthy atmosphere and a sense of community. We try to hire clever people to do simple things and so make the complex seem mundane.

We constantly look at ways we can improve how we work to cooperate and tune the process to the people rather than the other way around. We don’t believe in blindly following doctrine but encourage teams to arrive quickly and effectively on ways of working that help deliver the quality our customers have become accustomed to.

We’re no strangers to trying edge things and our CTO, Clifton Cunningham, allows us all the freedom to experiment responsibly in order to drive our innovation with many of us having worked with him a number of times before. There’s a reason for that.

Responsibilities

  • Providing input into any new solutions and for any enhancements of existing ones.
  • Working with Agile development teams in a web facing environment and for mentoring junior developers in the DevOps process.
  • Driving and owning relationships with multiple suppliers and internal teams.
  • Driving continuous improvement across multi-disciplined teams.
  • Help establish, renew and run processes for on-going maintenance and monitoring operations within the platform.
  • Providing out of hours on-call services on a rota basis.

Candidates should have experience of working in large scale, highly available enterprise environments and have demonstrable capabilities of contributing to a large scale python project. Experience should include some or all of the following technologies;

  • Ansible
  • Python
  • NodeJS
  • Cassandra
  • ElasticSearch
  • RabbitMQ
  • Redis
  • Logstash
  • Kibana
  • Bash
  • Linux Networking
  • Virtualization
  • Hadoop / RDS / MapReduce
  • Git / npm / dpkg / apt
  • Docker / Docker Swarm
  • Rancher / Messos / Marathon / Kube

Application Process

If you’re looking for somewhere to grow your skills and career and not just another job where you can hack then we’re probably going to get along fine. We’re very interested in any female engineers who might want to consider this role to help bring balance to the force.

There will be a technical test to pass as well as some tests that you won’t think a technical role would require. We’re looking for the whole of you and not just your technical kudos so come with an open mind.

If you’d like to talk about this role then please get in touch with me at khushil.dep@tesglobal.com with a current CV and GitHub account.

NO AGENCIES.

Hold for authentication on OSX with a network printer


Well this baffled be for a few minutes and I’m really just capturing this for my own knowledge later. If this happens to you it’s most likely your network password has changed or expired or something. Get it reset and then use your Keychain Access app and locate the printer name. Re-enter the password and you’re good to go.

information is everything


WIRED reports this morning that

ALL UNITED AIRLINES flights in the US were grounded this morning for nearly an hour, over “dispatching information.”

There’s much speculation around the possibility of erroneous flight plans being generated by the flight planning system and even a tweet about a possible hack of the system.

Tweet of message from Ted Benson

Whatever the cause, there exists today a silent but growing problem upon which little thought is being spent. Ours is a world of increasing reliance on electronic systems that orchestrate more and more of our lives. These electronic systems govern many safety critical functions and are driven to automatic and autonomic operation by data. While there is much made of Data Science and it’s disciplined application to drive greater revenue and safer interactions between man and machine, we are forgetting the much bigger and deeper field of Information Science.

The thing is that data is only part of the picture and in our ever increasing dependance on automatic and autonomic systems we’re going to have to make sure we take a holistic look at how we and our machines compose the ever growing amount of data into usable and trustable  information. Data Science is the discipline of extracting knowledge from data. But, there is a difference between having knowledge and knowing it. Knowledge is the accepted inferences made from data. Knowing is the ability to understand the full context and veracity of the knowledge in relation to the wider system within which the knowledge is embedded. It’s the ability to trust the knowledge that you or your machines have inferred.

This knowing begins with being able to trust your data. To be able to unequivocally assure all components of the system that the data it is being provided is what it say’s it is and is from where it says it’s from – at least. Each use-case will have more to add to the definition of trusted data but all use-cases need to be able to verify these two basic facts. Note I said verify. I didn’t say trust. Trust is almost always assumed.

It comes from within it’s own system so surely it can be trusted – right?

There’s firewalls, ACLs, tripwires, log parsing, alerting and a plethora of other defences – right?

Those defences are all well managed, well understood and well maintained – right?

You understand the provenance of your information – right?

provenance

ˈprɒv(ə)nəns/

noun

the place of origin or earliest known history of something.

a record of ownership of a work of art or an antique, used as a guide to authenticity or quality.

A much abused term in the modern parlance of Engineering. Often used by companies and individuals who wish to bolster the credibility of their ‘inventions’ which are often no more than the regurgitation of broken technology patterns and modes of man-machine behaviours.

The truth of knowing is the purity found in the absence of assumption.

Within our discipline of Engineering it’s a tough order to fill. Engineers are so used to patterns of assumed trust and behaviour.

We trust out key stores.

We trust our CAs.

We trust those that we are unable to validate the veracity of us because it makes our lives easier. Then we are surprised when that trust is proven false. We are surprised when our implied modes of trust is circumvented from within and without, with ease.

The problem is the knowing the provenance of your information. We as a discipline have to become more aware of the challenges of understanding the knowing of our information. It is not enough to be able to trust our data any longer.

I work for a company called Guardtime. We believe we have a robust and provable solution to this challenge of information provenance. We believe we can provide your data and information handling with a leading edge technology which will allow you to know your information; allow your machines to know your information. It’s called KSI. There’s a great paper on what this technology is – go read it.

Over the coming weeks I’ll be expanding on some use-cases within my own disciplines of cloud solutions and devops – so, stay tuned but in the meantime try some light reading:

Can’t Touch This

Stopping The Next Snowden