At the recent YOW! Night in Brisbane (as well and Sydney and Melbourne), Lynn Langit, and independent software consultant and cloud expert presented “Google Cloud Data Pipeline Patterns”. It was great to meet her, here are my notes from the event:
Storage is so simple you don’t need to think about it – data lake architecture
Simply select how much memory and how many cores and it computes the cost
Google is quick because they are laying their own fibre and using their own infrastructure – less than a minute to spin up a VM, with no local data centre
Google philosophy is you shouldn’t have to hire someone to manage the pricing
Bioinformatics is new to the cloud and they are in great need of engineers, Google is having a stab at a genomics API
BigQuery is NoOps, has been evolved to be a data warehouse, with no servers,
Big Relational if you just want SQL, Amazon Aurora is the fastest growing cloud database, Google has released Cloud Spanner, first database to truly meet CAP theory
BigQuery is column store, Cloud Spanner is OLTP
MQTT seems to be coming the defacto standard for IoT
IoT uses BigTable first and BigQuery second in Google reference architectures, because it is cheaper
Python is the emerging ML language for some reason
At the recent YOW! Night in Brisbane (as well and Sydney and Melbourne), Lindsay Holmwood (the Head of Technology at the DTA) presented “Breaking the Cylinders of Excellence”. It was a rare experience to hear the story of how the DTA is using cutting edge development practices to help the government catch up with, and even exceed, the public sector.
DTA – aid transformation in government, small agency
Delivery hubs in Sydney and Canberra – help identify and plug capability gaps in teams
Prototype of how government services could work gov.au/alpha
Digital Service Standard – 13 characteristics on what good looks like in government, useful in organisations as well
Cloud.gov.au – government cloud service, usage growing, continuous delivery pipeline (which is a major change for government who are used to 2 changes per year)
The unit of delivery is the team – not about individuals, but the team – borrowed from GDS
Government is slow, but government is designed to be stable, they cannot fail, they have characteristics that are resistant to change
Myth that organisations must choose between speed and reliability, high performing organisations deploy more frequently, have shorter lead times, fewer failures and recover faster, but they also have a greater profit
Want to deliver like a startup but be stable like a government
Not a lot of cross pollination between departments currently
Read the policy! – quite often the process is not mandated
Document what works and doesn’t so it becomes a repeatable pattern – ie. running a meetup inhouse, don’t tell me I can’t do it, tell me how I can run it without being thrown in jail!
Stick with technologies the government is comfortable with if you are changing the delivery engine
Security matters – prevention is a battle you will always lose, detection is your best defence – aggregate and log in one place, identify threat signatures, etc
Embed security people on big services so it is part of the architecture
Proactive testing between different governments around the world on similar platforms
Simplest security breaches make the most mess – infected excel macros, leaving free USB keys in the foyer that are malware infected
Need to put user needs first – alpha mockup using tools like Jeckyll, then beta then live
Lots of people strictly interpret the design and delivery guides – they are guides not rules!
Create a longer runway by pulling tech forward – turn down the volume of design, turn up the volume of tech
If it hurts, do it more often!
Fixed cost delivery with agile is a thing, agile is a way to de-risk in the government
Don’t put manual testing on the critical deployment path – have special skills on hand for accessibility, performance and security
At the recent YOW! Night in Melbourne (as well and Sydney and Brisbane), Daryl Wilding-McBride (the CTO of DiUS) presented “What I Learned while Teaching Kids at Flying Robot School”. It was an interesting story on the importance of social good for those of us in the technical space.
Waking hours capacity – families, hobbies, paid work, unpaid work
Not all work has equal social impact – pays the bills > interesting > impactful > worthwhile
Worthwhile work creates a legacy and passes the BBQ test (something you are proud to convey and recognised as value by the other person)
80000hours.org – the average hours you have from university to retirement, help you decide how to spend that time and be effective
It was great to have Martin Fowler back in Australia and host him in conjunction with ThoughtWorks for a YOW! Night in Sydney (he was also in Melbourne but I was unfortunately not able to attend).
Martin followed his usual approach of breaking his talk into three mini talks on Microservices, Event Sourcing and Infrastructure as Code. Here are the videos I shot from the session for YOW!
if you need to give the audience bad news, give it first
instantly does not mean effortlessly
1. Talk about your passion
to feel more confident, you need competence – talk about subjects you genuinely understand
seeing someone who is excited… is exciting
energy, enthusiasm and passion through your actions and speech will translate to your audience
find something in the required topic that gives you passion – even if you loathe the topic or have been forced by your boss to present it
2. Tell them a story
our memory is very volatile – stays for 8-10 seconds unless we do something with it
7+/- 2 is horribly optimistic and not backed by research, real number is 4 +/- 1
stories are our oldest information processing tool
stories have a flow to assist acquisition and memorisation (all our memories are reconstructed from a storyline), have a hierarchy to assist comprehension and recollection
tell the historical story or the story of what happened, process or funny anecdotes
story is for your benefit to get the sequence and content right – audience don’t necessarily need to know
stories make complexity comprehensible, structure recognisable, information easy to remember, make audiences feel more comfortable
3. Don’t search for content, select it
what should I say is the wrong question, question you should start with is what could I say
humans are good at recognising important stuff rather than recalling important stuff
start with a blank sheet and write down everything you know about the topic that you might want to say – stream of consciousness
whittle down to 3-5 most relevant and important topics to talk about
these 5 points becomes the chapters, so go looking for the narrative that connects them – they may not connect so look for a couple of lesser topics that better connect the 5 important things
competency – think about the questions you were asking when you were learning
4. Simplify your slides
tools encourage a bad job
content matters but not as much as style
content is your payload to explode the audiences brains
style – the stuff the audience doesn’t see that prevents them seeing what they should see
bad style – anything that prevents the audience seeing what they should see
a wall of text – technical audience will read everything, regardless of whether it is relevant or not
Apple is good at presentations – simple but effective
big words – people at the back can still read them
slide numbers turn your presentation into a death march – get rid of background, name and title on every slide, get rid of the logos (audience sees salesperson)
slide deck is to focus audience on the presentation – if they need context give them a separate PDF or notes
each message is a different slide
cluttered is overwhelming and as a result they switch off the attention channel as they are trying to read everything
show less on more slides
5. Manage the questions
a presentation should always be for the benefit of the audience – give them what they need
have an explicit questions policy – hold to the end of each topic, end of the talk, or interactive through the talk (can however affect the flow)
always be keen to take questions – shows you care
make the questions fit in with your question – “that’s a really good question” makes others more comfortable to ask question
6. Animate code simulations
explain code temporally, not spatially
use animations to reveal information one thing at a time
walk through code as an animation and highlighting
low tech animations – use the same slide over and over – cell animation
don’t export your slides – notes
live coding – synthesise, automate or have a partner – need to keep contact with the audience
7. Deliver your message fearlessly
use your nervousness – turn fear into energy
never give a presentation for the first time – practice it live at least 3 times
At the recent YOW! Night in Brisbane (as well and Sydney and Melbourne), Lynn Langit, and independent software consultant and cloud expert presented “Google Cloud Data Pipeline Patterns”. It was great to meet her, here are my notes from the event:
