Modalities of the Staff and Principal Engineer with Ken Collins (2/2)

Welcome to episode 242 of the Nerd Journey Podcast [@NerdJourney]! We’re John White (@vJourneyman) and Nick Korte (@NetworkNerd_) – two technology professionals with backgrounds in IT Operations and Sales Engineering on a mission to help others accelerate career progression and increase job satisfaction by bringing listeners the advice we wish we’d been given earlier in our careers. In today’s episode we share part 2 of a discussion with Ken Collins, detailing Ken’s contributions to open source software and how he got into it, his experience as an AWS Hero, and thoughts on the staff and principal engineer roles and what they really entail.

Original Recording Date: 08-12-2023

Ken Collins is a principal engineer at Custom Ink and an AWS Serverless Hero. Find him on Twitter @metaskills. If you missed part 1 of our discussion with Ken, check out Episode 241.

Topics – The Open Source Habit, Learning in Public and Staff Engineers, Far Transfer and Guidance on the Staff / Principal Engineer’s Path, Advocacy and Being an AWS Hero, Managers and Principal Engineers

2:56 – The Open Source Habit

  • What are some of the things Ken sees people missing in realizing that they need a portfolio of work and some ways we an better communicate what is in our portfolio of work to show the impact we’re making and our expertise?
    • Ken mentions you might be working for a company and doing amazing things, but what you’re doing might not be publicly available (i.e. not something you can point to on GitHub). Unfortunately some people have a perception that if you don’t have a certain status on GitHub it’s a strike against you.
    • Ken feels there are likely engineers out there who can communicate their non-public facing work in an interview when changing jobs perhaps better than he ever could.
    • “I just happened to find out that for me, one of the things I’m really good at is separating core business logic and sort of core open-sourceable code. So I’ve always ended up, in a weird way, working from that…continental railroad perspective on the business’ needs but in a way that’s very…abstract and…componentized….I’ve always had this amazing open source portfolio that I’ve just always built, and it’s just my default way to work.” – Ken Collins, on open source contributions as a way of working
    • Ken mentions that the above is a habit that was forced upon him. When he joined as a first time software engineer, Ken knew he wanted to program in Ruby on Rails because he loved the language so much.
      • The company Ken worked for served the trucking industry in that that made the software dealers used. Everything used SQL Server as its database back end. When a specific version of Rails came out they dropped support for the SQL Server adapter.
      • Because Ken loved the language he was using and enjoyed the company, to keep his job enjoyable he decided to take on the task of developing the SQL Server adapter needed to maintain functionality.
      • “That was that specific moment that was very specific to me that was very specific to a point in time that was very specific to the market at that time where it just really shaped my career.” – Ken Collins, on getting into open source
    • Nick says the above is a great job interview story! Ken saw a gap and decided to fill that gap, which allowed him to develop greater expertise as a developer and started him down an open source path.
      • Ken says he was able to be a better partner and contributor to the work at hand.
  • We’ve heard some people say contributing to open source isn’t really relatable experience if you want to go into software development while others have said it definitely is. What is the perception from Ken’s view among software engineering leaders?
    • Ken has worked at a company that has grown its number of software engineers to less than a dozen to many dozens.
    • “There’s always a focus on being able to grow people and help them and nurture them through their career and recognizing the underlying talent. I would say it is unusual to find a candidate or a way of working inside a company that places emphasis on open source.” – Ken Collins, on open source as a way into software development
    • Ken says many times companies want soft skills in candidates, how well they collaborate with others on a team, etc.
    • It definitely has worked well for Ken and shortcut some interview processes, but open source contributions are certainly not required for most people to work in software development.
    • “I was forced into open source from trauma. And ten that trauma keeps playing out in just the way that I work….I like showing my work, and I like seeing how other people sort of critique it….I do very much like to learn from my mistakes and success at the same time, and the only way I know of doing that is broadly sharing it with people.” – Ken Collins, on showing his work via open source
      • Some call this learning in public.

9:01 – Learning in Public and Staff Engineers

  • Was the learning in public what accelerated his programming career to become more senior?
    • There are different roles / personas / archetypes of principal and staff engineers.
      • Sometimes they are problem solvers for a specific need or around a specific technology.
      • It’s hard to be a leader within a company and it not be public. The open source work and community engagement has helped Ken, but he’s still working to improve to be a better speaker, have more empathy, and be a better active listener.
      • Ken shares a story about himself as a young child and not realizing there were grades for the work he did in school. He tells us he went in and tried to be himself every day, realizing only in about 6th grade that there were numbers by which we are judged in school. This has carried over into Ken’s career.
      • “Every day I just try to show up and be excellent and be amazing to myself and other people. And that goes all the way back apparently to grade school.” – Ken Collins, on his attitude each day at work
    • What is a staff engineer, and why pursue it?
      • Ken mentioned Custom Ink has been a growing organization over the years. They have different teams which need healthy levels of autonomy. Ken provides a nice analogy that you don’t want all the rooms in a house decorated differently.
      • “A company can only go so far on purpose-based alignment. Sometimes you need that sort of strategic understanding of how to get there.” – Ken Collins
      • There’s nothing wrong with being an individual contributor, but companies will need help setting strategic direction for technology and understanding how to get there.
      • A specific archetype of a staff engineer is focusing on architecture (which fits the strategy focus above).
      • Ken was sold on the staff / principal path because he had always been curious about the why behind things.
      • At Custom Ink, Ken was the first remote employee 10 years ago when he started. The people who worked in an office were in the same building. Those people didn’t really get up and walk to the next room to talk with others about their work.
      • “There was no pull request culture. You didn’t ask somebody why they pushed something into main.” – Ken Collins, on the lack of collaboration amongst different engineering teams when he started at Custom Ink
      • The engineering level to which Ken had progressed was more around coordination, vision, and strategy.
      • We recently spoke to business psychologist Leanne Elliott in Episode 237 and Episode 238 about how leaders that wanted to encourage more collaboration for people in the office didn’t design the office to create more collisions of people on different teams.
      • Ken references Conway’s Law and how people not talking to each other in the office likely impacted the way they built software at the time.

15:04 – Far Transfer and Guidance on the Staff / Principal Engineer’s Path

  • Ken is an AWS Serverless Hero, and he has a lot of content on Lambda, architectures built, and often shares stories of problems solved at Custom Ink.
    • “I’m going to show you something that I’ve done and a way that we’ve solved a problem. But I want you to understand that the context I’m giving you this in is very much wrapped up in something that you may not be experiencing. Even if it looks technically interesting…don’t think it’s for you because it’s solving our problem, and our needs are very specific in where we are.” – Ken Collins, on the disclaimer he shares when giving tech talks
    • Ken likes to ensure he gives the context around technical things to help people truly understand.
    • Nick says this mirrors the advice Ken gave about people’s career (i.e. something that worked for Ken may not work for others due to a different context).
      • Ken says this likely goes back to open source versus who you hire. You want to hire problem solvers and thinkers and not just a programmer to fill a seat.
      • “It’s the thinking that’s the problem solving.” – Ken Collins
      • John says this goes back to putting ping pong tables in offices and the lack of context that caused it to catch on as a way to encourage collaboration for many companies. It might not be the right thing for other companies.
      • Ken mentioned being at the AWS Hero Summit in Seattle recently.
  • Nick says the meta lessons align with the idea of far transfer mentioned in David Epstein’s book, Range: Why Generalists Triumph in a Specialized World. It’s about transferring knowledge across domains into a different context by using the deep structural similarities in each.
  • What should someone really think about before pursuing a role as a staff / principal engineer?
    • Ken would talk to the person about learning to grow past technical into people, which he tells us is difficult when your background is doing technically amazing work.
    • It might be easier to make the shift for those who were more people focused than technically focused.
    • “Be ready more for personal development and non-technical development, and / or trying to figure out the right balance for you as you navigate those two spaces….I think technical things are easier to move along. People are not.” – Ken Collins, on pursuing staff / principal engineer
    • Ken tells us that architecting something (like a system design) on paper doesn’t automatically mean it aligns with the way people do work and follow processes. One would need knowledge of these aspects to form a well-architected system. It takes both empathy for people and technical knowledge. Ken reiterates the difficulty of navigating this space well and that he continues to seek to improve himself.
    • For many in the role it may mean being in a lot of meetings, and for others, it may mean you won’t code any longer. Be honest with yourself about what you want to do.
    • Ken cites Staff Engineer: Leadership beyond the Management Track by Will Larson as an excellent resource. This book is something Ken and other principal engineers at Custom Ink have discussed quite a bit.
    • For a long time Ken was the only principal engineer at Custom Ink.
      • “Do the research. Look at how the role operates for different companies and how you can be successful in that role. And pick one, tack to it, evolve it, and be different. Do you want to be the right hand? Do you want to be technical? Do you want to be more the architect? What percentage of technical knowledge versus high level technical do you want? Find a persona. Find an archetype. And pick it out and start tacking toward it.” – Ken Collins, on advice for the would be staff / principal engineer
    • John says the different modalities of being a staff or principal engineer are interesting and that very few of them are “super senior” in that they tend to go along with force multiplication, depth and breadth, and context at multiple levels. Climbing up the career ladder in a role and gaining seniority and that up means only one thing seems to be a fallacy to which many fall victim.
      • This reminds Nick of the guests we’ve spoken to who have progressed into a seemingly less technical role and the fear of shedding the technical. Perhaps people don’t realize we can be technical in bands or spectrums in different roles, and it sounds like they can do that inside the staff or principal engineer roles. Nick hadn’t thought about this before having the conversation with Ken.
      • Ken remembers hearing about tech lead, architect, solver, and right hand as the different staff engineer archetypes from Will Larson’s book.
      • “There’s always an area where you can add value as a principal engineer at an org….One of the things I love doing in my role within the commercial organization is gluing together marketing, merchandising, and sales. I’m a conversion junkie at the end of the day. Helping those people be excellent is just always fun for me because you can watch the output so easily.” – Ken Collins, on what he loves about being a principal engineer

25:14 – Advocacy and Being an AWS Hero

  • John wonders if the advocacy program in which Ken participates is an extension of community involvement?
    • Ken is an AWS Serverless Hero, but AWS Heroes covers categories like compute, serverless, containers, data, IoT, etc.
    • AWS is a huge company, and the Heroes program recognizes people doing amazing work using different AWS technologies.
    • Ken works for a company that doesn’t do external recognition or developer relations. He is impressed with how well AWS does as a company of speaking to and reaching the technical community.
    • Ken says he received an e-mail one day asking if he wanted to be a hero (not something he signed up for). With the Heroes program there are different levels of engagement with the community.
    • Ken’s focus is on Lambda and compute. He does a lot of work on Ruby on Rails and helps people get applications running inside Lambda.
      • Ken maintains a site called which is sort of like an open source organizational unit with the technologies that help people use Ruby on Rails and AWS Lambda together. It allows people to get up and running with these technologies quickly and with the ability to scale.
      • Ken continues to do this work and also gets to work with AWS product teams closely on the Lambda product, feeling that when he talks and gives directional feedback, AWS teams listen. This proximity to product teams at AWS is very much in line with how principal engineers like to function and work with internal product teams at their companies.
      • Ken calls being part of this group “an amazing gift” and really enjoys bouncing ideas off other Heroes. He was able to do a lot of this at the recent AWS Hero Summit.
      • “Some of the coolest things I use at AWS were actually recommended by other Heroes. And that is so cool!” – Ken Collins
    • Nick says the transferability of the staff / principal engineer mindset to being part of an advocacy program like the AWS Heroes program is once again evidence of far transfer.

29:17 – Managers and Principal Engineers

  • What are some of the differences between a staff / principal engineer and people managers / people leaders?

    • Ken does not have any direct reports as a principal engineer. No one is asking him for raises, and he doesn’t do regular 1-1 meetings with direct reports (since there are none).
    • Organizationally, Ken is adjacent to engineering managers and people that do have direct reports. He is also at a leadership level in the company as an individual contributor.
    • Ken says he lacks some line of sight into people management from where he sits. He has seen principal engineers on Twitter who do have direct reports.
    • “It’s a way for you to be an influence on the organization or the communities that you’re in without having to really be responsible for people. That’s great and bad at the same time.” – Ken Collins, on differences in the principal engineer’s role compared to that of a people leader
  • Did Ken’s early experience as a manager influence the decision to stay individual contributor?

    • Ken didn’t come out with a negative feeling / perception from earlier experience as a manager. He was young, focused on execution, and lacking empathy / emotional intelligence.
    • Ken feels he naturally (not necessarily intentionally) stayed at the technical level and feels he has been afforded the ability to be himself.
  • Would Ken rule out management?

    • Ken thinks it might be needed, but he’s not sure, thinking he will have to figure it out when the time comes. It seems like there is a natural tendency of having people report to you.
    • Nick mentioned we’ve interviewed people who went up to management, back to individual contributor, and back to management again. We’re busting stereotypes that say once you pursue management you are stuck there forever. It could be an option for a time, forever, or for never.
    • Ken says at Custom Ink 30-40% of principal engineers used to be managers.
    • John likes the fact that there is a path for people who may not want to manage a team but still want to impact the technical culture in the same way a manager or director would (or have the same impact radius).
      • Some may be better at their impact if they don’t have to lead people. These people may be more effective as influencers of people than leaders of people.
    • Ken thinks people who went from manager to principal engineer may have had growth burnout, some effect from the pandemic and the remote culture, etc.
    • “The natural tendency toward growth is to push people to managing….I think it’s really amazing that recognizes that you can go from a manager to some sort of individual contributor and that’s amazing and celebrated. We’ve certainly done that at Custom Ink. It’s never looked down upon.” – Ken Collins
      • Maybe if someone couldn’t make the impact they wanted as managers (with people) might be able do it in other ways (moving things forward, etc.).
  • Mentioned in the outro

    • We’ve explored the principal title and role with some other guests whose perspective you might want to learn from in these episodes:
      • David Babbit
        • Episode 197 – Connected to the Customer Base with David Babbitt (3/3)
        • Episode 195 – Perceptions and Realities of Startup Life with David Babbitt (1/3)
      • Phil Monk
        • Episode 185 – Make Accommodations for Success with Phil Monk (1/2)
        • Episode 186 – The Unassuming Architect with Phil Monk (2/2)
      • Joe Chenevey
        • Episode 170 – Signal What You Want with Joe Chenevey (1/2)
        • Episode 171 – Coaches as Mindset Curators with Joe Chenevey (2/2)
      • Josh Duffney
        • Episode 233 – Leaps of Faith That Fail with Josh Duffney (1/2)
        • Episode 234 – Sunk Costs and Pivots with Josh Duffney (2/2)
      • Scott Lowe
        • Episode 152 – The Theme of Your Career with Scott Lowe (1/2)
        • Episode 153 – Creating a Full Stack Career with Scott Lowe (2/2)
      • Andrew Miller
        • Episode 167 – Pause and Step Outside with Andrew Miller (3/3)
    • Staff and principal engineer type roles are different at different companies. We need to be looking at job descriptions to understand what that means at a specific company.
      • You can adapt our way of working and your style to roles like this, and it may look different from the way others do it.
    • Be sure to check out Will Laron’s book Staff Engineer: Leadership beyond the Management Track!
    • Ken is a principal engineer inside the realm of software development and brings a unique perspective as a result.
    • Nick thought back to Episode 123 and Josh Duffney’s comment of just adding value being synonymous with a principal engineer’s ability to add value and an attitude we can all adopt.
    • There is nothing wrong with making a move from manager to individual contributor. Check out Episode 202 with special guest Yvette Edwards discussing some career zigzags and moves between individual contributor and manager, back again, and then back the other way.

Contact us if you need help on the journey, and be sure to check out the Nerd Journey Podcast Knowledge Graph.

Leave a Reply

Your email address will not be published. Required fields are marked *