TomOwens · HackerTrans

قبل 4 أيامdiscuss

And what happens when the agent modifies or skips tests or finds other ways to circumvent the constraints? There are already examples of agents trying anything they can to achieve the objective. The code is the one source of truth for the system's behavior, so beyond rapid prototypes that will be thrown away, I don't see how a human can set aside accountability for changes made by an agent.

TomOwens

قبل 22 يومًاdiscuss

Dave Farley's book is really good, and I'd highly recommend it.

However, I do have an issue with his focus on applying science in the definition, even though it doesn't come across in his final conclusions. In the history of engineering, that's a relatively new development. It started, to some extent, in Europe in the 1700s, but it really took off in the 1920s and exploded in the United States after World War II (1940s-1950s). It culminated with the Report of the Committee on Evaluation of Engineering Education (the Grinter Report) in the mid-1950s.

This isn't to say that science isn't important to engineering, since it absolutely is. Science provides knowledge used to better understand the world being changed by engineering. But there are also plenty of examples of engineering going ahead of science - the steam engine, the airplane, generative AI. We didn't fully understand the governing rules before the technology existed.

Any definition of engineering needs to be broader. Ferguson (Engineering and the Mind's Eye), Florman (several works, but primarily The Existential Pleasures of Engineering), and Vincenti (What Engineers Know and How They Know It) all explore ways in which engineering can't rely on science alone. I think Koen (Discussion of the Method) puts it best, where the application of science is one heuristic that engineers may choose to draw upon.

TomOwens

قبل 26 يومًاdiscuss

Having a bottleneck with AI-generated code is probably a good thing.

Consider before AI. In cases where code was written by a trusted human, you had confidence that they were doing the right stuff - the people understood the context and requirements, the changes were based on those requirements, and tests were updated. You could rely on some basic automated checks to run a linter and static analyzer, execute the tests, and check test coverage. You can defer longer-running tests to a later stage, but people can merge their work in and collaborate. You can have blocking reviews for untrusted outside contributors or new employees and review the rest later.

After AI, the input is a mess. I haven't personally seen the worst of it, but I've read about people generating code before any requirements development or analysis. Code is generated without understanding what it is supposed to do, which also prevents good testing. Even with good requirements, people generate solutions that aren't right, and the generation step may remove failing tests or make unintended changes that aren't covered by tests. So much code is generated from untrusted sources and rubber-stamped. Introducing a bottleneck where a human can slow the process down is a good thing here.

Maybe if we can get people to understand how to use AI tools responsibly and have humans take full accountability for the output and changes, we can get back to the old way. Trusted humans and an AI agent assistant can make changes that are solid enough to merge. Independent control over the pipeline and basic automated checks would give just enough confidence in the change, and delayed review before deploying or enabling the feature would work. But not until trusted humans actually maintain oversight over their tools.

TomOwens

الشهر الماضيdiscuss

In practice, the frameworks don't work out of the box. Take Scrum, for example. The framework described today has had all the original technical practices used by the first Scrum Teams stripped out, with no new technical practices added, and it's been generalized to fit many organizations so it's marketable. There are many questions to ask. Why do I need to have a Daily Scrum every day? If I'm mobbing, do I need a Daily Scrum at all? Why a maximum timebox of a month for a Sprint instead of 5 or 6 weeks? What technical practices do I need to deliver software every day or week? All of these are important questions that would likely result in changes.

The culture is the big thing.

I've only worked in very large enterprises, but I've never had to deal with a dickhead CEO because they push decisions down. I haven't even had to deal with a dickhead business unit VP or director. Decisions are actively made 3 or 4 levels below the CEO. That's almost always at the product level, which could be one level up from a team if multiple teams work on a single product. I've also rarely had to deal with people being shoehorned in because we strive to eliminate single points of failure. That means having at least two people with (or working toward) a given skill set, cross-training, and being willing to work outside their comfort zone. People who stay in their niche and don't work with others often don't last long, by their choice.

So, yes. These things can work. I've done it. But it means you can't follow someone else's script or a canned framework and set of tools. It means having a deep understanding of why a framework has made certain choices around roles and events and what to change to build something that works for you. But it's also cultural change in the organization because agile ways of working are inherently different than predictive methods - they need different relationships and inputs and create different outputs.

TomOwens

الشهر الماضيdiscuss

A better title would be "Dark Agile is a scourge on the planet". There are plenty of people who've written about this, but Ron Jeffries' posts "Dark Scrum" (https://ronjeffries.com/articles/016-09ff/defense/) and "We Tried Baseball and It Didn't Work" (https://ronjeffries.com/xprog/articles/jatbaseball/) cover this well.

Retros: Retros are about being more effective. That usually means talking about and solving problems. If too many problems are beyond the team's control, that's indicative of a "faux agile" or "dark agile" environment. If your "CEO being a dickhead" is the biggest problem in anything but the smallest organizations, your CEO is not pushing decisions down to the lowest possible level where they can be made and isn't trusting and respecting the people with the knowledge and skills to get the work done. This is the kind of cultural shift needed to make agility work. If most decisions are made two or three (or more) levels above the team, then agility falls apart. When it comes to successes, it's not about patting yourselves on the back, but about finding ways to repeat those successes again and again in the future.

Standups: Even a "quick morning briefing'" isn't the intent of a standup. It's a planning session. If you have a bunch of people who barely work together, you won't get value from a daily standup. But when you have highly collaborative teams and are planning things like who is going to pair or mob today, what work will be integrated and would be ready for review, what unexpected problems came up, how to account for someone's unplanned time off, or how to plan for someone's upcoming time off, then you're getting value. In some cases, daily is even a bit much. Two or three times a week could work, especially if the team communicates regularly and well outside of this meeting.

Agile coaches: Unfortunately, companies hiring unqualified coaches has turned this role into nothing more than a team secretary or admin. Looking at Lyssa Adkins and Michael Spayd's Agile Coach Competency Framework shows that an effective agile coach needs many skills: lean and agile practice, teaching and mentoring, coaching and facilitating, and mastery in at least one of technical, business/product, or organizational development and transformation. Instead of hiring people who could actually teach and coach their teams and organizations on how to reorganize themselves and apply new practices, they hired people who took a two-day course without any practical experience in a lean or agile organization and who can't teach the people doing the work. Even if you do get someone with the right skills, the organization needs to trust and respect this person and be willing to make the changes needed to support agility.

Sprint / cycle planning: There are many models that emphasize continuous flow. However, it's not so much about the planning but the continuous heartbeat of the entire iteration. Sometimes, it's easier for key stakeholders to interact with the team on a regular, pre-defined cadence. When you establish that interaction, regular planning comes naturally since you want to plan what you'll do between the end of one interaction and the start of the next. If you're getting constant feedback, though, the value of this decreases. I would say that estimation, especially at the unit-of-work level, is unnecessary, and there are plenty of resources on using historical data on work completion to plan upcoming work.

Grooming: No single person can make the right work on their own. Investing a little time from the whole team to review the work and develop a shared understanding greatly reduces the time needed to plan the work. You can also think about many perspectives - what various external stakeholders need or want, the architecture of the system, how you'll verify and validate the work, how to make sure it's deployable and operatable (and monitorable) in production. You'll find ways to reorder the work because doing X before Y will reduce the risk of Y or you'll find that the team needs to prototype and explore. Without some kind of refinement activity, I'm not sure how work can be clearly defined for the team to execute.

Trusting people to do their jobs is the heart of lean and agile methods. But, at the same time, you need structures and patterns to coordinate people, both internally on a team and externally with stakeholders. The problems arise when there is no trust, and people are forced to work in ways that simply aren't appropriate for their context.

TomOwens

قبل شهرينdiscuss

I graduated from a software engineering program accredited by ABET as engineering, rather than as a computing program, which would include CS, data science, information systems, and IT. Even with that, I didn't have much, if any, coursework covering the topics needed to pass the FE exam.

The other issue is when your state has requirements for work experience under a PE. At the time, I was working in aerospace/defense, and we had almost no PEs in any discipline. I knew of two or three PEs, one of whom was a project manager and no longer doing technical work. None of them were in the software organization. Even if people could take and pass the FE exam, it would be hard to get the experience. This one is on the states, though, since they create the requirements for becoming a PE.

What makes it worse is that my undergrad degree almost exactly mapped to the SE PE exam, which was very heavily based on SWEBOK. Before I graduated and before the PE exam was a thing, I started down the path of the IEEE CS certifications, which covered much of the same topics. I stopped that when they revamped their certification pathways.

TomOwens

قبل 4 أشهرdiscuss

The AGPLv3 doesn't protect against cleanroom engineering.

The full passage from Clause 13 is: "...if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software".

The output of cleanroom engineering is not considered a modified version of the Program. Therefore, users do not need an opportunity to receive the Corresponding Source. The wording about a "standard or customary means of facilitating copying software" is designed to be inclusive of various delivery mechanisms, such as sending a self-addressed stamped envelope to the developer to receive a flash drive or CD, or hosting the source code on GitHub, or a website that allows users to download compressed files.

All that said, there are interesting questions about whether using AI tools to create specifications and then build from those specifications is a form of cleanroom engineering. That's probably a worthwhile avenue to pursue, especially demonstrating that AI tools offer the provable isolation of developing the specification and developing the implementation.

I also checked out the Malus website, and at least one of the claims is a bit questionable. A service like Malus doesn't really solve license compliance overhead. Organizations that require extensive legal reviews and audits to use components will likely also have robust vendor management processes in place. The burden would shift from reviewing and auditing the component to reviewing and auditing Malus. These organizations also tend to be rather risk-averse. Claims about "full legal indemnification" in an "offshore subsidiary in a jurisdiction that doesn't recognize software copyright" would raise legal flags.

I'm also interested in what models tools like Malus use, especially since many are trained on open-source projects to begin with. So far, the training piece has been considered fair use in a handful of US cases. The output, though, is still to be determined. This is why GitHub has integrated a public code search for the output of its AI agents and organizations are investing in static code analysis that searches public code repositories for similar code. AI tools that reproduce open-source code may trigger those licenses, and I don't see any mention of Malus assessing the output against open-source projects.

There are still plenty of open questions, but we're far from the death of open-source software.

TomOwens

قبل 6 أشهرdiscuss

> You absolutely can; the model is quite capable of reproducing works it was trained on, if not perfectly then at least close enough to infringe copyright. The only thing stopping it from doing so is filters put in place by services to attempt to dodge the question.

The model doesn't reproduce anything. It's a mathematical representation of the training data. Software that uses the model generates the output. The same model can be used across multiple software applications for different purposes. If I were to go to https://huggingface.co/deepseek-ai/DeepSeek-V3.2/tree/main (for example) and download those files, I wouldn't be able to reverse-engineer the training data without building more software.

Compare that to a search database, which needs the full text in an indexable format, directly associated with the document it came from. Although you can encrypt the database, at some point, it needs to have the text mapped to documents, which would make it much easier to reconstruct the complete original documents.

> That's not something you can handle via guardrails. If you read a piece of code, and then produce something substantially similar in expression (not just in algorithm and comparable functional details), you've still created a derivative work. There is no well-defined threshold for "how similar", the fundamental question is whether you derived from the other code or not.

The threshold of originality defines whether something can be protected by copyright. There are plenty of small snippets of code that can't be protected. But there are still questions about these small snippets that were consumed in the context of a larger, protected work, especially when there are only so many ways to express the same concept in a given language. It's definitely easier in written text than code to reason about.

TomOwens

قبل 6 أشهرdiscuss

Point 2 misses the distinction between AI models and their outputs.

Let's assume for a moment that training AI (or, in other words, creating an AI model) is not fair use. That means that all of the license restrictions must be adhered to.

For the MIT license, the requirement is to include the copyright notice and permission notice "in all copies or substantial portions of the Software". If we're going to argue that the model is a substantial portion of the software, then only the model would need to carry the notices. And we've already settled on accessing over a server doesn't trigger these clauses.

Something like the AGPL is more interesting. Again, if we accept that the model is a derivative work of the content it was trained on, then the AGPL's viral nature would require that the model be released under an appropriate license. However, it still says nothing about the output. In fact, the GPL family licenses don't require the output of software under one of those licenses to be open, so I suspect that would also be true for content.

So far, though, in the US, it seems courts are beginning to recognize AI model training as fair use. Honestly, I'm not surprised, given that it was seen as fair use to build a searchable database of copyright-protected text. The AI model is an even more transformative use, since (from my understanding) you can't reverse engineer the training data out of a model.

But there is still the ethical question of disclosing the training material. Plagiarism still exists, even for content in the public domain. So attributing the complete set of training material would probably fall into this form of ethical question, rather than the legal questions around intellectual property and licensing agreements. How you go about obtaining the training material is also a relevant discussion, since even fair use doesn't allow you to pirate material, and you must still legally obtain it - fair use only allows you to use it once you've obtained it.

There are still questions for output, but those are, in my opinion, less interesting. If you have a searchable copy of your training material, you can do a fuzzy search of that material to return potential cases where the model returned something close to the original content. GitHub already does something similar with GitHub Copilot and finding public code that matches AI responses, but there are still questions there, too. It's more around matches that may not be in the training data or how much duplicated code needs to be attributed. But once you find the original content, working with licensing becomes easier. There are also questions about guardrails and how much is necessary to prevent exact reproduction of copyright protected material that, even if licensed for training, isn't licensed for redistribution.

TomOwens

قبل 8 أشهرdiscuss

The premise of this whole post is incorrect. If an organization is building an AI product or offering an AI service, then a SOC 2 report, or at least a SOC 2 Type 2 report, should answer these questions.

"What happens if someone tries to extract training data?" CC6.7 covers data loss and data transfer restrictions. I've typically included controls related to monitoring data transfer, including flagging and highlighting potential breaches. Documented procedures on what happens if data loss or unauthorized data transfer occurs. These can be reviewed, but may be hard for the auditor to test unless they were executed and there's evidence that they were executed as written.

"Can this agent be manipulated into accessing data it shouldn't? How do you test for adversarial attacks?" I'm struggling to understand the difference between these questions. It seems like part of the answer likely overlaps with controls to address CC6.7 and data loss or data transfer restrictions. CC8.1 discusses testing the product or service.

"How do you prevent prompt injection?" This may be a bit specific for a SOC 2 Type 2 report, since it really gets into requirements, architecture, and design decisions rather than controls over the requirements, architecture, and design. That is, you can essentially not require preventing prompt injection and follow all of your controls related to, for example, CC8.1. CC8.1 talks about managing, authorizing, executing, and documenting changes. You can do all of these things well without that requirement in place.

"What guardrails are in place, and have they been validated?" This is the entire SOC 2 Type 2 report. It lists all evaluated criteria, describes the organization's controls, and provides an audit of those controls. It's up to the organization being audited, however, to think about what controls are necessary for their context. The controls that should be in scope of the audit will differ for an AI product or service than for something else. The recipient of the SOC 2 report can review the controls and ask questions.

Part of the burden is on the organization getting the SOC 2 audit report to think about what controls they need. But there's also a burden on the organization reviewing the audit report not just to see that there are no exceptions, but to review the controls described to make sure the controls are in place for the given product or service. And this detailed information about the controls is what makes something like the SOC 2 audit report a whole lot more useful than something like an ISO 27001 certificate, which says that whatever policies and procedures are in place meet the requirements of the standard and doesn't offer details on how those requirements are met.

TomOwens

قبل سنتينdiscuss

I'm looking through what I have access to quickly.

I started with the CSA's Cloud Controls Matrix, just because they trace to a bunch of other standards. They have a control - IAM-07 - that is to "de-provision or respectively modify access of movers / leavers or system identify changes in a timely manner in order to effectively adopt and communicate identity and access management policies". This control points to other sources.

One standard that I have access to, CIS Critical Security Controls v8.1, calls for a process that disables or deletes accounts and revoking access "immediately upon termination, rights revocation, or role change of a user". I believe v8.1 is the latest version.

The Trust Services Criteria mapping is to CC5.3 and CC6.3. This defers to defined organizational policies and procedures and doesn't specify any timelines.

ISO/IEC 27001:2022 and ISO/IEC 27002:2022 mapping is A.5.15 and A.5.18. This is identified as a gap in earlier versions of these standards. I don't have ready access to these standards, so I can't tell you if they give any timelines.

The NIST 800-53 rev 5 mappings are to AC-2 (1, 2, 6, 8), AC-3 (8), AC-6 (7), AU-10 (4), AU-16 (1), and CM-7 (1). All of these appear to defer to organizationally-defined timing and frequency for review.

The NIST CSF v2.0 mapping is to GV.RR-04, GV.SC-10, PR.AA-01, and PR.AA-05. The most relevant ones are the PR.AA controls and they both defer to organizational policies for definitions.

As far as I can tell, most standards simply require that a company defines their policies and procedures and then certification or audit against that standard would only ensure that the documented policies are being followed. If you wanted to implement the "immediate", one way to do it would be to document that as your process (optionally by adopting the CIS Critical Security Controls, but you may or may not want to adopt the whole set) and then have it in a SOC 2 Type 2 audit where the auditor would sample people who have left the organization and when their access was revoked.

TomOwens

قبل 3 سنواتdiscuss

If the copyrighted code was uploaded to GitHub by the owner, there's no problem with this. When you upload code to GitHub, one of the rights that you grant to GitHub is the right to use your content for "improving the Service over time". See D.4. License Grant to Us in the GitHub Terms of Service. Once it is up there, you also grant other users certain rights, like viewing public repos and forking repos into their own copies. See D.5. License Grant to Other Users. Even with the most restrictive protections in place, using GitHub requires you to give up certain rights.

A question would be if creating and training Copilot is "improving the Service over time". I would suspect that it would be, though.

There are still some open questions around what happens when Copilot suggests code verbatim, but these are mostly for the users of Copilot. Although I would hope that GitHub is thinking about offering information to ensure that users understand the source of code they use, if it may be protected, and what licenses it may be offered under. There are still some interesting legal questions here, but I don't think that the training of Copilot is one of them.

A more interesting question would be what GitHub does if someone uploads someone else's copyright-protected code to GitHub and it is used for training Copilot before it is removed. If you don't own the copyright, you can't grant GitHub the rights needed to use that code for anything, including improving the service.

TomOwens

قبل 5 سنواتdiscuss

Most of the comments on that app as well as here are probably wrong. I'd suspect that everyone who had the app "installed without their permission" opted into the Android COVID-19 Exposure Notification program. This was deployed by Google as part of an update to Google Play Services.

When you go to your phone's settings with this update, there's an option to enable COVID-19 Exposure Notifications. When you turn it on, it prompts you for your location and will download your region's app that uses your phone's new capabilities to connect to the appropriate health authorities.

Massachusetts just opted into this program in the last couple of weeks. I'm honestly not sure why they did it so late - this would have been helpful earlier. Apple iPhones also have this capability, including interoperability with Android phones, and iPhone users in Massachusetts are also able to turn on this setting.

Now, if someone can actually prove that they didn't opt into the COVID-19 Exposure Notifications, then I'd be concerned. But my guess is they opted in when it came out, but there was no app for their region, so nothing was downloaded and the feature did nothing. Then, Massachusetts rolled out the app now and lots of people who configured their phones earlier in the pandemic got a new app. They granted permission for it, perhaps months ago.

TomOwens

قبل 5 سنواتdiscuss

I think this is why the PDF is actually the "Guide to the Software Engineering Body of Knowledge". It's not a representation of the complete body of knowledge itself, but extracting key concepts and terms and provides pointers to things that are most relevant. If things are irrelevant or disproven over time, the guide to the body of knowledge would remove those terms, concepts, or references and point to something else.

TomOwens

قبل 5 سنواتdiscuss

I wonder if the people involved in approving and conducting this research are aware of the ACM's Code of Ethics. I can see pretty clear links to at least two or three of the code's ethical principles. This seems to be a pretty serious breakdown of the researchers understanding their ethical responsibilities, but also the review and approval of research projects.

TomOwens

قبل 5 سنواتdiscuss

Yeah. That's a better way of putting it. The author didn't opt to change the license. He corrected a licensing error.

I do agree that making the downstream users aware is important, I just don't agree that immediately yanking is the right solution. Putting out a new version would have been nice. Adding a post-install message to the new versions would have been good to start to get the word out. Not sure how far to take it, but opening issues with dependencies (RubyGems provides this information) would have also been nice, giving the major dependencies a good notice before yanking.

TomOwens

قبل 5 سنواتdiscuss

Vendoring is a good first step, too. As long as you have a local copy of all the dependencies, you're better off than needing to go pull them from the Internet every time you want them and risk having them gone. Potentially worse is having the same version but with modifications.

TomOwens

قبل 5 سنواتdiscuss

This just bit me.

The first thing that I noticed was that some people are not understanding the GPL. It's far more impactful to Rails than the vast majority of web applications built using Rails. The use of GPL'd files means that the gem itself has to be released under the GPL. Since the gem is now under the GPL, dependencies are also under the GPL. That would include Rails. However, even if Rails was under the GPL, organizations could still build closed-source web applications using Rails since network access is not distribution. That's the whole point of the AGPL.

However, it does raise a lot of questions about when someone is allowed to yank a gem (or any library, really). It's been a while since I took a deep dive, but I was under the general impression that there was some leeway around not breaking the world when rectifying license issues. I would think that releasing new versions under the correct license and giving everyone notice and time (30 days?) to update would be fine for most copyright holders. I'd suspect that most open source developers wouldn't want to break the world. The sudden yanking with no warning caused builds to fail everywhere.

The absolute worst thing, though, was that changing a license should not be a minor (or a major) version number increase. It should be a patch. The breaking was simply because Rails is pinned to 0.3.x, but the first release under the new license was 0.4.x. Fortunately, the author released a 0.3.6 patch with the correct license, so it's just a matter of a bundle update to get the latest version. But if he hadn't, Rails would have had to release a new version and anyone on legacy/unsupported Rails versions would be hosed if they had to rebuild and redeploy.

This is a really good reason to stand up your own artifact repository and put all of your third-party dependencies in it, especially if you're a business.