AWS configuration with Terraform

 

Recently I had a contract which required me to immerse myself in AWS configuration for a LAMP stack, which led me to Terraform, amongst other things. I thought I’d publish here the list of resources that helped me. It’s a lot of links!

One tip I found was to use Jenkins to do automatic validation of your Terraform scripts. I think that’s a good tip.

Talk by Nicki Watt of Hashicorp: https://www.youtube.com/watch?v=wgzgVm7Sqlk

Good tutorial: https://www.youtube.com/watch?v=LVgP63BkhKQ

AWS

Cloud Best Practices: https://d1.awsstatic.com/whitepapers/AWS_Cloud_Best_Practices.pdf

Elastic beanstalk might be a way to go: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts-webserver.html

These days I tend to put everything behind an Nginx reverse proxy: https://github.com/awslabs/ecs-nginx-reverse-proxy

Some AWS Webinars: https://aws.amazon.com/architecture/

Drupal. Which was quite close to what I’d be needing: https://aws.amazon.com/quickstart/architecture/drupal/ and https://aws-quickstart.s3.amazonaws.com/quickstart-drupal/doc/drupal-on-the-aws-cloud.pdf and https://aws.amazon.com/quickstart/architecture/drupal/ and https://github.com/aws-samples/aws-refarch-drupal

Doing your Git stuff at Amazon appears to have some advantages. CodeCommit: https://aws.amazon.com/codecommit/

AWS cost Best Practices: https://aws.amazon.com/premiumsupport/trustedadvisor/best-practices/

Web Hosting general best practices: https://d1.awsstatic.com/whitepapers/aws-web-hosting-best-practices.pdf

A reference architecture for Drupal hosting:

Akamai

If you’re on AWS it makes sense to use Amazon’s own CDN service. If however, you already have an Akamai account, it’s supported by terraform.

https://github.com/Comcast/terraform-provider-akamai

Terraform

Interestingly, Terraform works across multiple cloud providers.

An introduction from Gruntwork, a Terraform consultancy: https://blog.gruntwork.io/an-introduction-to-terraform-f17df9c6d180

Terraform with AWS: https://dzone.com/articles/terraform-with-aws

Terraform getting started: https://www.pluralsight.com/courses/terraform-getting-started

Hashicorp training: https://www.hashicorp.com/training

Udemy have courses if you want to pay money: https://www.udemy.com/learn-devops-infrastructure-automation-with-terraform/

Gruntwork comprehensive guide: https://blog.gruntwork.io/a-comprehensive-guide-to-terraform-b3d32832baca

Gruntwork slideshare: https://www.slideshare.net/brikis98/comprehensive-terraform-training

Another tutorial: https://gist.github.com/p0bailey/3a34689f49b075ed058373dd73a7bce6

All about provisioners: https://www.terraform.io/docs/provisioners/index.html

Ansible

If Ansible is your configurator of choice, you can shoehorn it in to Terraform.

https://alex.dzyoba.com/blog/terraform-ansible/

https://github.com/express42/terraform-ansible-example/blob/master/ansible/terraform.py

https://github.com/radekg/terraform-provisioner-ansible

https://github.com/jonmorehouse/terraform-provisioner-ansible

Chef

Chef is natively supported.

https://sdbrett.com/BrettsITBlog/2017/08/using-the-chef-provisioner-with-terraform/

Chef and AWS: https://www.chef.io/implementations/aws/

Jenkins

Jenkins on AWS: https://d0.awsstatic.com/whitepapers/DevOps/Jenkins_on_AWS.pdf

https://aws.amazon.com/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/

https://aws.amazon.com/blogs/devops/set-up-a-build-pipeline-with-jenkins-and-amazon-ecs/

https://docs.aws.amazon.com/aws-technical-content/latest/jenkins-on-aws/best-practices.html

https://wiki.jenkins.io/display/JENKINS/Jenkins+Best+Practices

https://docs.aws.amazon.com/aws-technical-content/latest/jenkins-on-aws/jenkins-on-aws.html

And there’s the link dump. It should be enough to keep you going for a couple of days! Overall, I think if you’re going to use AWS, embrace it fully. Except Cloudflare. They’re awesome.

WordPress spice with plugins

Wordpress logoSo, PHP and MySQL, two slightly suboptimal technologies run a fairly large chunk of the internet in the form of WordPress. You have the idea for a blog or maybe want to knock up a quick corporate web site. What’s your first step?

Themes

  • Choose a WordPress theme. There are loads out there, some free some paid for. My site of choice for finding themes free or otherwise is Themeforest. A fair number of the themes are free, and you can choose 2 or 3 column, responsive and so on.

Having chosen your host (we use bluehost.com for www.pandaandpolarbear.com and this site), then it’s time to flesh out the functionality of your site with plugins.

Plugins

  • Akismet – a pretty good comment spam filter plugin. It will mark spam for you so you can you through and trash it. Not sure I’ve ever had a spam comment go through.
  • Cloudflare – These guys are making the internet better. A DDoS, CDN and free SSL solution. 128 data centres. Who is to argue with that?
  • Cookie Consent – Everyone needs this, right?
  • XML sitemaps. Does what is says on the can!
  • Jetpack – Even more themes, stats, SEO tools, Security stuff.
  • Loading Page – while the page is loading, shows a pretty graphic. Given the stats on site abandonment, any distraction is worth it.
  • NextScripts: Social Networks Auto-Poster – lets you spam nearly 30 social media channels.
  • P3 (Plugin Performance Profiler) – Really useful to see where the CPU time is going and if a plugin is taking the time. In my experience, plugins take about 50% of the page render time.
  • W3 Total Cache – caching is good. Most site are not that dynamic so caching is relly good to have.
  • Wordfence Security – useful to have. We’ve had someone uploading rogue JavaScript to WordPress and this spotted it.
  • WP Smush – optimise graphics for the size you’re rendering them at. This is a cool speedup. When you’ve got four years of art, it’s a big win.
  • Yoast SEO – If you’re wordy like me, it’s good to have something reminding you of the good stuff to put in your posts to get the attention of the search engines.
  • Amazon Associates Link Builder  – nice integration with Amazon associates.
  • Finally, Link checker – useful to check for broken links, or destination pages that have gone away.

Other stuff

Don’t forget to sign up to the Google suite, Google analytics, Google webmaster tools, and Google Lighthouse.

Conclusion

That’s a small selection of the plugins we use. There are a whole bunch of Woocommerce related WordPress ones and others related to selling stuff.

I did a site for Dusty Knuckle Pizza, which was working great until they foolishly decided to spend money and get something worse. IMHO.

So that’s that. Your site is now standing on the shoulders of giants.

Remind me, why do people still build web sites manually?

Please employ the bear to do something interesting!

So yet again, I spent time battling a legacy perl code base with no tests, no Jenkins/Bamboo, no deployment pipeline and half an agile process.

Now I get to do battle with recruiters again, something that fills my life with joy and purpose.

I thought I’d put my thoughts down as to what I’m looking for in a job.

First up, contract or permanent? That’s easy. I’ve been contracting for 18 years and I don’t see that changing UNLESS you have a really juicy CTO role on offer. More of that later. I think it’s just largely temperament. I like to have an independent, outside view, trying not to get absorbed in the local cargo cult. So there are two things I do.

Senior Perl developer.

My career can be best described as “careering from one thing to another”. If I’d had any sense, or career direction, or a mentor, I’d have stayed much more firmly in the CTO field. I’ve flirted with many startups over the years, but none have actually stuck. So what am I looking for in a perl gig? Here goes:

  • A modern framework. Give me Catalyst preferably, a framework standing on the shoulders of giants. Dancer or Mojolicious would work as well. Template Toolkit is the ideal templater.
  • Tests. It should be obvious, but often isn’t. If you write code without tests your code is immediately legacy.
  • A sane database schema. One that MySQL Workbench can reverse engineer into a pretty diagram. An ORM. There’s little point these days hard-coding SQL. That’s so passé. Give me DBIx::Class.
  • A well-run Agile process. I got my Scrum master certification and now “doing agile” as opposed to “being agile” brings me out in a rash. One purpose of agile is to get better and unless you do that, you’re not agile. Just standups and sprint planning don’t cut it.
  • Javascript I can take or leave, but it’s a given these days. I can do it but I’ll hate myself afterwards.
  • Don’t talk to me about web servers. Not my problem any more.
  • I want support infrastructure that’ been there since the beginning. That means Perl::Critic and perltidy. Pretty, clean code please.
  • Please let me please talk to REST APIs, none of that SOAP rubbish.

CTO

I’ve been a CTO. And interim a few times. Obviously I’d do it all completely differently this time, knowing what I know now.

  • Let me grow the team. I’ve had amazing luck in the past picking great teams. Indeed, a team that largely didn’t know perl and then became experts. I’ve also been involved in a firing. We’re still friends.
  • Let’s have all the tools we need: Atlassian (or equivalent) stack or integrated equivalent.
  • I want to buy in a good Agile coach for a few months to get us on the right track.
  • I want to manage upwards well. Demo the important stuff to the other directors and management at the end of every sprint. Respond to the business.
  • If you’re good, you can work from home. This is the 21st Century. Being forced to turn up to an office is one of my bugbears. You don’t need my physical presence. Skype and Slack will do the job.
  • Give me something exciting to lead. Not sure I could cope with another publisher web site.
  • Let me speak at conferences. Yes, I know I’m a straight, white male. It’s a burden. But I AM left handed! I’m a minority! It’s good for the company visibility.

And probably stacks more.

As an aside, any good personal projects worth chipping in to right now?

Recruitment. Broken industry.

Dodgy Dave
How I imagine working in recruitment

Introduction

Looking back over my LinkedIn, I have had the following gripes about recruiters and recruitment. As a contractor, I do the interview round two or three times a year, so this stuff is becoming second nature.

Recruitment gripes

  • Diction. I’ve had calls from English people who either mumble incoherently, have incoherent regional accents or are Indian with the same. Really guys, listen to Radio 4 and copy their accent.
  • Please don’t call me from a speakerphone in a room full of echos. Also, your phone system may well be crackly, out of date junk. Please get a modern system where I can hear what you’re saying.
  • I’m not Gordon.
  • If you call and I don’t pick up, LEAVE a succinct, clear message.
  • Have caller ID so if you fail at leaving a message, I can call you back.
    If you do leave me a message, leave words. I really don’t want five minutes of your office background noise and you flirting with a co-worker.
  • If I contact you via LinkedIn or even by phone, PLEASE RESPOND. I’m trying to make money for you.
  • If you call me and the line drops, CALL ME BACK. Don’t just wander off and make out with a fellow recruiter.
  • LinkedIn, I *hate* being stalked by anonymous recruiters. Why do you allow this? Why are recruiters afraid to reveal themselves?
  • Recruiters, and job sites too. Why do you send me Java, Scala or Go jobs? I don’t do that. You’re wasting your time.
  • Please write literate adverts. I’m not a “principle” developer. Writing advertisement littered with spelling or grammar errors is doing you no favours.
  • Please make sure that the salary mentioned in the job headline matches what’s mentioned in the body. Come to that, mentioning a salary or day rate in the advertisement is a Good Thing.
  • Really research your client. Even yesterday I spoke to a recruiter who barely knew the client, didn’t know what their vertical sector was, and had a really sketchy job spec. At the very least, check out their profile on Glassdoor.
  • It’s REALLY nice to know where in the UK the job/contract is. Come to that, knowing where in LONDON the contract is would be nice. Docklands and Shepherd’s Bush are worlds apart.
  • Just because my CV isn’t littered with Jenkins, JIRA, Confluence and Git, doesn’t mean I don’t use them every day! They’re fundamental.
  • Recruiters, keep *your* web site jobs listings up to date. Get a job in, put it on your site. Fill a job, take it down. Otherwise, you’re wasting my time and yours.
  • Don’t spam me on my work email address. Straight to the trash, this one.
  • Don’t email me with barely a spec. I’ve just had two. What’s the point? “Must be good with computers.” Well, *duh*.
  • Don’t demand references up front. You’re just trawling for business.
  • Don’t demand ridiculous documentation like passport copies, A-level certificates, utility bills up front. That all comes AFTER the offer. I’ve worked for banks where that process took a day when I was being “onboarded”.
  • Don’t expect me to fill in a long, convoluted web form that replicates pretty much everything that’s in my CV. At least that’s optional with Jobserve.

Finale

And now, the big one. Back in the day, when I was CTO, I was hiring for a team of Linux, Perl developers. Not one of the people I hired had all of the things I was looking for. Some people were Java or PHP. One guy was a Windows dude. I hired them because they were GOOD PEOPLE and as it turned out, we built a great team, of expert Linux/Perl people.

Asking for 10 years of Python is essentially meaningless. These days we have Google and Stackoverflow. I’ve done Python and Ruby on this basis. It’s not hard. Anything can be learned. Unlike twenty years ago, there’s barely a question that hasn’t been asked, answered and blogged. I want the right PEOPLE, not your in-depth Node.js skills. Asking for particular knowledge is essentially meaningless.

So there you have it. The entire recruitment industry is based on a false premise and staffed by a proportion of cowboys and dodgy companies. Where have we heard that before?

David Hodgkinson’s blog

So here’s a new blog after nearly a year of internet silence. My life mostly exists on Facebook and a few retweets. I have a friend who retreated from Twitter to his blog. Maybe that’s what I might do. And set up IFTTT to post from here to Facebook and Twitter.

If you care about what I did previously, there’s the Wayback Machine.

I have a backup of the old site and if I can overcome previous borkage, I’ll load it. We’ll see. I like the sparseness. It’s not like I did much significant before.