Whilst I personally don't have first hand experience with GitHub's security team - your experiences and suppositions perfectly match my own general observations with our sector.
I've had the pleasure of working with absolutely outstanding security professionals, doing exactly what you see - relatively thankless work.
Due to the alignment of incentives, it can in fact be worse than merely thankless and face internal opposition - so the perverse reality of Microsoft/GitHub needing to feel more pain is sadly likely true.
This article's author is wonderfully honest about having not worked at a larger company and is right to call out the issues that are all about organisational alignment/ownership.. or lack thereof.
You could never pay me enough money to be responsible for security at GitHub though, that's for sure.
Strongly inclined to agree here: Having recently joined a small applied AI startup and we were discussing the need for E2E tests. My initial gut reaction (which I kept quiet) was that such things turn into unmaintainable messes which delay releases and increasingly reduce in value.
I recognised this was grounded in an entirely different world of software engineering and organisation size though. I followed a path of thinking about what went wrong historically and how might they be solved: Better structure, discipline, resources - all of the things which agentic AI facilitates.
You are right about most skipping this part: But I view it as being like a sewerage and sanitation system - largely invisible and not thought about but critical for long-term health.
Also this ties in very nicely with Netflix's approach to Chaos Engineering and enabling it at broader scale.
Thanks for sharing this - it looks very promising in terms of capabilities to address many of my own issues with building out agents. Effectively you've implemented a lot of things I planned to do but in a robust manner and so have saved me a great deal of effort.
Edit to add: This might spur on a few more to start doing that, but people are quick to forget/prioritise other areas. If this keeps happening then it will change.
It's an awesome project and I imagine it has saved countless production incidents. The amount of times I've said "it was probably certificate expiry" and been correct is reasonably high.
In my own cases of responsibility, Caddy would have eliminated them had it been around. Instead I've learned to be paranoid, though having things like this are far better in terms of easing cognitive burden.
Cheers for all of the hard work by you and other maintainers.
My 1st thought: The comment to which you are replying is why I'm not sure I'd have the patience to maintain an OS project. Though the older I get, the better I get at ignoring certain things.
My 2nd thought: Actually, this is very likely a culture/communication difference whereby both people care (I'm a big fan of Erin Meyer's work here)
My 4th though: Perhaps I can make 'enemies' become friends if they both have disdain for AI ;)
Note: I would absolutely not submit this as-is. Caddy's an amazing project though I am not very familiar with its implementation and I'd seek to understand it, conventions etc. and make some obvious improvements to the code which has been generated - but this was a minor bit of fun. I created 4 separate versions and only in one of them did anything with TLS related get amended.
I ought to caveat that from a KYC perspective, I likely would have been 'easy' since my career history involves many well-known companies and the requested account was for a UK company that has been running for nearly 20 years.
Though I'm no longer personally in the UK. Microsoft on the other hand: It was impossible for me to open up an Azure account even with their Customer Support. Suspect I will have to use Tailscale to bypass overly rigid geographical controls.
I'm not sure if they've changed the process since your issues (hopefully) - I was using their trial and when it expired had to go through the KYC process. They under-promised by saying two working days to reach out, but everything was wrapped up within an hour and done over email.
I nearly moved away from Twilio having read the negative feedback, but my personal experience so far was very prompt support.
Also with you (both) on this one in pretty much every way too. I'd justify hard mode that I loved challenges (and I did, just not always in the right areas).
I try not to look/think back too much - I had (sort of still have) a very successful career but the costs associated with getting there were and are still being paid for.
Getting treatment and therapy has really helped improve my ability to be present, though still such a battle.
This is really cool of you to openly share the prompt - props for that.
Really cool product, as someone currently attempting to build a somewhat similar internal tool I have an understanding of some of the pain points involved.
Please don't allow yourselves to be bought out by Apple in the way Buddy Build were back in 2018 though! (and then shut down)
Oh wow, clicking on their "Join Team" button is absolutely amazing. Credit to them.. almost makes me wish I was a Mac developer to apply for a job. Almost.
Well put and you're correct: There IS a lot of hype/BS still sadly - as companies seek to jump on the hype train without effectively adapting. My karma took a serious hit for my last post - but yesterday I met with someone whose life has been profoundly impacted by AI:
- An extremely dedicated and high achieving professional, at the very top of her game with deep industry/sectoral knowledge: Successful and with outstanding connections.
- Mother of a young child.
- Tradition/requirement for success within the sector was/is working extremely long hours: 80-hour weeks are common.
She's implemented AI to automate many of her previous laborious tasks and literally cut down her required hours by 90%. She's now able to spend more time with her family, but also - able to now focus on growing/scaling in ways previously impossible.
Knowing how to use it, what to rely upon, what to verify and building in effective processes is the key. But today AI is at its worst and it already exceeds human performance in many areas.. it's only going in one direction.
Hopefully the spotlight becomes humanity being able to focus on what makes us human and our values, not mundane/routine tasks and allows us to better focus on higher-value/relationships.
"all that's left to jump on at the moment is AI" -> No, it's the effective applications of AI. It's unprecedented.
I was in the VC space for a while previously, most pitch decks claimed to be using AI: But doing even the briefest of DD - it was generally BS. Now it's real.
With respect to everything being awful: One might say that's always been the case. However, now there's a chance (and requirement) to build in place safeguards/checks/evals and massively improve both speed and quality of services through AI.
Don't judge for the problems: Look at the exponential curve, think about how to solve the problems. Otherwise, you will get left behind.
They had an opportunity to actually adapt, to embrace getting rapid feedback/iterating: But they are not equipped for it culturally. Major lost opportunity as it could have been a driver of internal change.
I'm certain they'll get it right soon enough though. People were writing off Google in terms of AI until this year.. and oh how attitudes have changed.
Exactly. Negativity in headlines/attitudes gets higher engagement - whilst the challenges are real, the opportunities are also truly astounding. Distance yourself from the naysayers.
Focus on the positive, the possibilities and what you can control and do. It's always time to be optimistic.
Pulsar vs Kafka was a significant lesson to me: The "best" technology isn't always the winner.
I put it in quotes because I'm a massive fan of Pulsar and addressing the shortcomings of Kafka. However, with regards to some choices at a former workplace: The broader existing support/integration ecosystem along with Confluent's commercial capabilities won out with regards to technology choices and I was forced to acquiesce.
A bit like Betamax vs VHS, albeit that one pre-dates me significantly.
I've had the pleasure of working with absolutely outstanding security professionals, doing exactly what you see - relatively thankless work.
Due to the alignment of incentives, it can in fact be worse than merely thankless and face internal opposition - so the perverse reality of Microsoft/GitHub needing to feel more pain is sadly likely true.
This article's author is wonderfully honest about having not worked at a larger company and is right to call out the issues that are all about organisational alignment/ownership.. or lack thereof.
You could never pay me enough money to be responsible for security at GitHub though, that's for sure.