Congratulations on the launch and work so far! This is indeed very interesting. I believe a language like this is necessary for future progress. I have also been impressed with Julia and its progress. I would have liked to see more progress on Ahead of Time compilation and using it to write Python extension modules.
For Mojo, I'm interested in seeing how the language can be used as a path forward for the Cython community. This could be a stepping-stone towards reimplementation of Python in Mojo. For the past 3 year, I have been talking about the need for a Python-Steering-Council-recommended extension language for Python. This will be particularly important as WebAssembly keeps progressing and potentially redefining what we mean by virtualization and containers.
We have already been using LLVM extensively in Numba and there have been several explorations around MLIR and related technologies. There are several potential paths forward and I'm looking forward to finding ways to cooperate.
Understanding what will be open-source is of course, critical for that.
This is a very good point. You must support AOT compilation if you are going to have a general purpose system. This was clear when Julia came out and I interacted with core developers communicating our experience with creating SciPy and the critical reliance on the AOT features of Fortran/C/C++ as well as the bindings to Python. I believe simpler spellings can be achieved (i.e. more unification between scripting, dynamic JIT, and foundational AOT use-case), but the ecosystem is not close to a one-language to rule them all scenario.
There is some work to bring it up to 3.12 and some resistance to merge it into 3.X because of the impact on extension modules (they all have to be recompiled and in some cases changed a bit).
If you are interested in it, reach out to Sam. He has done a pretty impressive piece of engineering work.
I'm sorry about triggering you. I can see your point. I did not mean to focus attention away from what happened and how to avoid it.
I am sincerely interested in seeing if the HackerRank leadership will reach out and discuss. I was not intending to promote anything I'm doing right now. I suspect they won't but I will talk to them respectfully if they do.
Let's promote the SymPy maintainers, though. Let's also promote NumFOCUS, because they do a valuable service to community-driven projects and could help SymPy respond to this sensibly. Let's also promote NumFOCUS because they efficiently and tirelessly work to help the projects they fiscally sponsor (like SymPy) -- providing legal support: https://numfocus.org/.
Perhaps something good can come out of this, still.
Thanks for showing up here and replying. I can understand the net being cast too wide for a real problem you are trying to solve. It does have real consequences for already under-resourced communities, though.
I appreciate the "fix-it-twice" attitude implied by the RCA promise (Root Cause Analysis for those who also had to look it up). Also, consider recognition and restitution for the unnecessary work you created for the NumFOCUS director, a NumFOCUS lawyer, and the SymPy maintainers.
A $25k donation to NumFOCUS would be a good start.
If you are willing to talk about what happened publicly, I'm starting a podcast/video series to discuss business and open-source. This would make an interesting conversation. Perhaps we can turn this into a positive to raise awareness and inspire better behavior in the industry? Ping me.
This author misses the point that the first problem a business faces is how to get customers and how to please them. Getting customers is the number one priority -- perfect architecture is only important to the degree that helps get customers.
Business owners and managers do need to understand that they may have to rewrite the software from the ground up because the first version will have all the problems the author talks about.
Most business owners and managers actually do understand this already. The goal of the first version is not to be architecturally perfect. The goal of the first version is to see if anyone will care if it exists. This is a good argument for not always building on the first version of a startup idea -- but actually build it again once you know people will buy.
I think this is true. I have used the Python C-API heavily having started SciPy and NumPy and Numba. I have a pre-alpha plan for addressing the C-API by introducing EPython (a typed subset of Python for extending it). It is not usable and in idea stage only, but I welcome collaborators and funders: https://github.com/epython-dev/epython. Here is a talk that describes a bit more the vision: https://morioh.com/p/6db365736476
I didn’t read the entire thread of previous discussion. The referenced article is consistent with the work of Thomas Metzinger who wrote a very accessible book called “The Ego Tunnel” as a follow on to his more detailed and scholarly treatise “Being No One”
He introduced me to the ideas in this paper that he induced from his own observations of his “mystical” experiences.
It is fascinating to realize these self models exist in a wide variety of animals and the modeling of others is what makes some animals different.
Yuval Harari argues that Homo sapiens took over all other Homo species because of the development of “myth” which may be related to the Hyperactive attribution of consciousness referenced in the article. Humans cooperate at scale much better than any other species (but also fight at scale more aggressively) because we can communicate about abstractions.
This is about creating a standard for array computing in Python which has been around for more than 20 years. NumPy (which has been a de facto standard) has been around for 14 years and was based on Numeric which was around for 10 years before that. Pandas has been around almost 10 years now. The parts of the standard that are clear and will emerge are the parts that have been around and used by sufficient numbers for at least a decade.
I agree that interfaces like Datasette are a fantastic idea. But, it's not the same thing. Think of this more like the SQL standard. What would you do if SQL did not exist and there were no common ways to discuss querying databases?
Bias warning --- I'm part of the group doing this work. For an end-user, your arguments are fine. A single developer or team can indeed use whatever tool they want and adapt their code without too much difficulty.
The real challenge comes, however, as you try to build an ecosystem of libraries on top of multiple different APIs. You end up reinventing the wheel or struggling to maintain multiple versions of the same library.
Some cooperation is immensely helpful at the lowest level, and we hope ultimately to be helpful to the different tool builders in providing information and guidance about what is already a standard and agreed upon in the space.
This is not about establishing or forcing behavior. It is about documenting and clarifying best practices that have already emerged as well as potentially giving libraries a way to signal to downstream developers of additional libraries that they adhere to the standard.
I didn't perceive it to be an argument for anything specific. I did not get the sense the author is suggesting either lifting restrictions or not. I saw it as more of a piece trying to get us all to think harder and better and expose the difficult trade-offs and the conundrum we face as humans of the morality vs utility argument of saving someone hurting in front of us at any cost versus saving more people at a lower cost potentially at the risk of not saving the person in front of us. I appreciated the author's parable of the river.
My own thoughts go in the direction of maybe the right thing to do is different at different "scales". The right thing to do in my family is different than the right thing to do in my job which is probably different than the right thing to do at the city level, the county level, the state level, the nation level, and the world level (to use labels appropriate for the USA). Leaders at each level have a different decision to make and things work better the more people we have making "good" decisions at each level. The process of figuring out what "good" decisions at each level is the really hard part that is not easily wrapped up with platitudes, memes, or other projections into a single one-dimensional axis (i.e. left vs. right) of a high-dimensional, multi-attribute space.
Travis Oliphant, here. OpenTeams is much bigger than me, but I have been part of the early team putting energy into the effort. Henry has captured some of the personal motivations for me. They are sincere. There are plenty of ways for me to make money more easily than wrestling with the economics of open-source software. This is my calling and something I care about a lot. You can watch a video of me talking something about this here: https://www.youtube.com/watch?v=MCEUPosJuIY if you want to get to know me better.
I have had the privilege of watching open-source projects I started or participated in get used by millions. I want to see a world where more people are paid a great living to work on open-source software. There are several approaches to improving the funding fabric for open-source. OpenTeams is one of them. Quansight Initiate (http://www.qi.ventures) is another one. Quansight Labs (http://labs.quansight.org) and Quansight (http://www.quansight.com) are others. A group of us are also working on a few other ideas including FairOSS (not going to give the website because it is a behind-the scenes operation for now --- ping me if you are interested in learning more).
I have spent many years writing open-source software, improving open-source ecosystems, hiring open-source developers, and paying people to work on open-source software. I have also spent the past 12 years in particular working in industry watching open-source get used and seeing the disconnect between open-source usage and open-source communities.
There are several business models that can work to improve things. With OpenTeams we are building out a platform and ecosystem to help open-source communities thrive by directing money from industries that use them to the communities that exist while also learning from the basic laws of incentive alignment and local choice making that are at the heart of every economy.
One of my core believes is that not all software will be or should be open source. I don't make this conclusion lightly. However all software will be based on and use open source foundations. Those communities that produce these foundations should be able to thrive and yet retain their freedom and self direction.
OpenTeams is a platform to make that happen by connecting companies that use open-source and want to fund specific features and improvements with organizations that hire open-source developers to work on proposals that have input from community participants and leaders. Some initiatives will be new projects, some will be usability improvements and feature proposals for existing projects. An effect of OpenTeams is also to enable a place for all open-source contributors to build their reputation (not just contributions that lead to pull-requests).
We are in a pre-alpha stage at the moment, but will be continuing to add features and are open to suggestions and ideas.
I was also there on Thursday and agree. The talks were actually observing that cloud giants do alter the business landscape for open-source, but there are always innovations and possibilities. There were one or two talks on alternative licensing schemes, but the most interesting talks to me were alternative virtuous cycles that are available for supporting open-source while allowing any company to benefit.
Thank you for doing this. This is a great idea. If every company that used open source committed to supporting the projects they depend on with this kind of approach, it would solve a lot of the current sustainability problems. I look forward to several initiatives over the next few years to encourage this kind of behavior from all companies.
I believe it is true that you can make more money selling non-free software (some of the profits from which should always be used to make free software).
You can generate money only through free software, however. Here are three approaches I have found: 1) Consult on projects that use the free software and use some of the profits to support that free software, 2) Sell enterprise-grade support on the free software to big companies. This is much more than just help-desk and answer the phone. All commercial software comes with a big contract. You provide the same kind of contract just no license restrictions. Others can do the same and so you have to distinguish yourself by either having all or most of the experts on the software or just really good marketing. 3) Dual license using GPL3/AGPL for the free version and a commercial license and then aggressively go after people for GPL violations if they don't get the commercial license.
I don't like the relationships created by the third model ---
your sales processes become aggressive and counter-service minded by definition. I don't see it scaling and really providing value.
The other two are really hard to impossible to get investors exited about and therefore you struggle to get the capital together you need to prime the customer pump.
There is a another general model with many corollaries where you basically "do something else" that uses the software as a critical part of the business and let the profits of that activity fund open-source development.
A lot of open-source these days is actually funded by this kind of activity (or from VC's hoping to profit from a promise of great wealth from this kind of activity).
I knew what I was getting into when I wrote NumPy. I knew there was not a clear way to support my family by releasing open source software, and I knew I was risking my academic career.
I did it because I believed in the wider benefit of ideas that can be infinitely shared once created and the need for software infrastructure to be open-source --- especially to empower the brightest minds to create. I did it because others had done it before me and I loved using the tools they created. I hoped I would inspire others to share what they could.
There have been a lot of people who have helped over the years. From employers willing to allow a few hours here and there to go to the project, to community members willing to spend nights and weekends with you making things work, to investors (at Continuum) willing to help you build a business centered on Open Source.
There are many people who are helping to fix the problem. In 2012, I had two ideas as to how to help. Those who know me will not be surprised to learn that I pursued both of them. One was the creation of NumFOCUS that is working as a non-profit to improve things. The second was the creation of Continuum (http://www.continuum.io) to be a company that would work to find a way to pay people to work on Open Source full-time.
We have explored several business models and actually found three that work pretty well for us. One we are growing with investors, a second we are continuing with, and another we are actually in the process of helping others get started with and ramping down on ourselves.
Along the way, I've learned that open source is best described in the business world as "shared R&D". To really take advantage of that R&D you need to particpate in it.
We call our group that does that our "Community Innovation" group. We have about 35 people in that group now all building open-source software funded via several mechanisms.
We are looking for people to help us continue this journey of growing a company that resonantly contributes significantly to Open Source as part of its mission. If you are interested, contact me --- I am easy to track down via email.
For Mojo, I'm interested in seeing how the language can be used as a path forward for the Cython community. This could be a stepping-stone towards reimplementation of Python in Mojo. For the past 3 year, I have been talking about the need for a Python-Steering-Council-recommended extension language for Python. This will be particularly important as WebAssembly keeps progressing and potentially redefining what we mean by virtualization and containers.
We have already been using LLVM extensively in Numba and there have been several explorations around MLIR and related technologies. There are several potential paths forward and I'm looking forward to finding ways to cooperate.
Understanding what will be open-source is of course, critical for that.