Ask HN: Alternatives to organizing code in files and folders?
13 comments
I actually like the filesystem and its hierarchical structure. What is missing is speedy access via alternative ways similar to views in SQL where you can narrow down your criteria based on filenames, dates, and contents. Finding something in a huge collection of source files used to be very slow until I discovered ripgrep which is insanely fast.
Hierarchical structure has been criticized a lot in recent years, and rightly so as for the naive view which insists that there is One Right Way to categorize things. But the not-so naive view acknowledges that any hierarchy is artificial and ultimately arbitrary, while it at the same time asserts that it can also be useful. Think of a library: which one is more useful to your most frequent use cases, one that keeps books ordered in divisions and subdivisions of subjects on the shelves, or one that keeps books in random order? I think it is the first. You'll still need one or more catalogs and one can still debate where that book on maths for the engineer goes (maths, engineering, or text books); that doesn't mean the hierarchy of the books on the shelves is useless.
Hierarchical structure has been criticized a lot in recent years, and rightly so as for the naive view which insists that there is One Right Way to categorize things. But the not-so naive view acknowledges that any hierarchy is artificial and ultimately arbitrary, while it at the same time asserts that it can also be useful. Think of a library: which one is more useful to your most frequent use cases, one that keeps books ordered in divisions and subdivisions of subjects on the shelves, or one that keeps books in random order? I think it is the first. You'll still need one or more catalogs and one can still debate where that book on maths for the engineer goes (maths, engineering, or text books); that doesn't mean the hierarchy of the books on the shelves is useless.
I find I often rely on a bunch of third party packages. They are essentially part of my code but I don't care about their folder structure, only their API. These are stored flat in npm/cargo/pip. Lot's of my code ends up not being business-specific and could be extracted into similar small packages.
When I need one of these packages, I search Google, and typically read their API on Github. I never navigate a hierarchy to find them.
When we make our own code flat, we have problems because we don't have an easy way to search. Another argument would be that hierarchies can help contain dependencies - i.e. don't depend on code outside of a directory.
> that doesn't mean the hierarchy of the books on the shelves is useless
I just think the hierarchy should be virtual/automatic, and the focus should be on tracking dependencies between components. I think very few people would look at dependency graphs of their codebase...there's just not good tools.
When I need one of these packages, I search Google, and typically read their API on Github. I never navigate a hierarchy to find them.
When we make our own code flat, we have problems because we don't have an easy way to search. Another argument would be that hierarchies can help contain dependencies - i.e. don't depend on code outside of a directory.
> that doesn't mean the hierarchy of the books on the shelves is useless
I just think the hierarchy should be virtual/automatic, and the focus should be on tracking dependencies between components. I think very few people would look at dependency graphs of their codebase...there's just not good tools.
A huge drawback to changing this is all of our computing resources are based on a 100+ year old concept of a file cabinet with folders and files. It has a lot of historical traction which you would be attempting to go up against with a different concept.
As long as the code is easy to find and doesn't have similar naming as other code you should be good to organize however you want. If you work on a team then make some decisions as a team on how to organize files. I work with Rails which has an opinionated view on folder/file structure which is great because I don't have to spend time thinking about it and can focus on getting things done.
As long as the code is easy to find and doesn't have similar naming as other code you should be good to organize however you want. If you work on a team then make some decisions as a team on how to organize files. I work with Rails which has an opinionated view on folder/file structure which is great because I don't have to spend time thinking about it and can focus on getting things done.
> all of our computing resources are based on a 100+ year old concept of a file cabinet with folders and files.
While they are based on the same concept, putting things inside of things (in hierarchies) is a pretty solid way of sorting stuff.
There are other ways as well, such as labels/tags. But regardless of technology I feel like folders (in one way or another) is one of the tools you would reach for.
While they are based on the same concept, putting things inside of things (in hierarchies) is a pretty solid way of sorting stuff.
There are other ways as well, such as labels/tags. But regardless of technology I feel like folders (in one way or another) is one of the tools you would reach for.
> putting things inside of things
Were you thinking of this Monty Python skit when you wrote that?
https://youtu.be/LFrdqQZ8FFc
Were you thinking of this Monty Python skit when you wrote that?
https://youtu.be/LFrdqQZ8FFc
Revamp an IDE to abstract away from the file system. Let the IDE organize the code according to functions and subsystems instead of files. Add tools to the IDE for better search, and the ability to hide sections of code and comments when you don't want to see them. Add an automatic map builder that outlines control flow and indicates which sections of code are closely related. Add an AI to suggest code and to analyze and summarize the structure of the code. The code all gets compiled into one binary or app so it is logical to treat all of it as one monolithic thing, but people aren't capable of even seeing 300 pages of code at the same time, they can only keep small sections visible and work with them at one time. Design the IDE to handle that complexity and make it easy to zoom in on the code that is relevant to what you want to do.
Smalltalk. And yes, mapping code components to filesystem objects is a kind of an impedance mismatch.
Just downloaded https://squeak.org/ to play around with this concept.
I wonder if there is already a modern tool/suite for Node/Python inspired by Smalltalk...
I wonder if there is already a modern tool/suite for Node/Python inspired by Smalltalk...
Consider playing with Pharo [1], it shows how it can still use Git to store sources in background.
There is LivelyKernel [2] but some versions are more file-oriented (like Lively 4 [3])
[1] https://pharo.org/
[2] https://lively-kernel.org/
[3] https://lively-kernel.org/lively4/lively4-jens/start.html
There is LivelyKernel [2] but some versions are more file-oriented (like Lively 4 [3])
[1] https://pharo.org/
[2] https://lively-kernel.org/
[3] https://lively-kernel.org/lively4/lively4-jens/start.html
Very nice. Squeak is also my first choice for Smalltalk environments. I'm not much of a Smalltalker but I know several who were real involved.
For the current reality that is directories and files, you can get pretty far with sed, grep, git grep, shell scripting, find -type f -name 'foo' -exec ... '{}' ';', etc, etc...
If some alternate reality existed, based on some other data structure / schema... I guess some other set of tools and workflows would be needed to find and move whatever the new name for blobs of bytes (files) would be...
If some alternate reality existed, based on some other data structure / schema... I guess some other set of tools and workflows would be needed to find and move whatever the new name for blobs of bytes (files) would be...
I've always liked the Lifestreams[0] idea of organising documents chronologically, at least for personal files. Not sure how practical it is for code.
0. http://cs-www.cs.yale.edu/homes/freeman/lifestreams.html
0. http://cs-www.cs.yale.edu/homes/freeman/lifestreams.html
As an experiment try putting all the code in a single file (I am not recommending this). It then becomes impossible to hide complexity without decreasing it, and you naturaly come up with all sorts of utility functions. The code becomes denser.
It is an interesting exercise, and with a good typesystem it can go a long way.
It is an interesting exercise, and with a good typesystem it can go a long way.
What if you had one giant folder with every file available? The obvious problem is polluting the namespace with multiple same named files, naming things and enforcing UUID’s becomes annoying.
I think folders and sub folders makes sense for run times and IDE’s to “know where to look”.
I think folders and sub folders makes sense for run times and IDE’s to “know where to look”.
> polluting the namespace
Yeh this seems to be the primary problem. You would basically use uuids and end up with your files in a sort of database.
I think this often scares people because they don't know where something definitively lives anymore...and you rely on a tool to access your stuff.
Some people actually prefer a definitive location for things...in order to build a mental map to come back to things. When I am coding I know where to find things quickly based on the shape of the folder structures I browse. Folders just seem natural with many analogies to the physical world...whereas graph clouds such is with tagging are rare.
But I think we must remember that the file system is essentially a database with a tool that one relies on to access the underlying bytes on disk.
Yeh this seems to be the primary problem. You would basically use uuids and end up with your files in a sort of database.
I think this often scares people because they don't know where something definitively lives anymore...and you rely on a tool to access your stuff.
Some people actually prefer a definitive location for things...in order to build a mental map to come back to things. When I am coding I know where to find things quickly based on the shape of the folder structures I browse. Folders just seem natural with many analogies to the physical world...whereas graph clouds such is with tagging are rare.
But I think we must remember that the file system is essentially a database with a tool that one relies on to access the underlying bytes on disk.
The file system works and is a common denominator. Even if you could come up with a better scheme for organizing software projects, what use would it be if no tools except for your own supports it? Using ls for listing modules/classes in a project is useful, as is removing such items with rm. So is grepping, or using some other editor than your preferred one to interact with the project. Maybe something better will show up in the future - like asking ChatGPT where your stuff is - but I'm skeptical.
> Using ls for listing modules/classes...
Interesting. Just made me think of using a custom filesystem to navigate a codebase. Similar to: https://github.com/osnr/TabFS. I wonder if anyone has done this.
Interesting. Just made me think of using a custom filesystem to navigate a codebase. Similar to: https://github.com/osnr/TabFS. I wonder if anyone has done this.
Hierarchies (folders) and tags are the only way to organize things.
I’ve been curious about this topic for a while primarily because I’ve found that issues with circular dependencies/imports in my projects are simply a consequence of their organization into files.
If every exported variable or function was in a separate file, that would eliminate most of my problems. But it’s just a pain to write code that way…
If every exported variable or function was in a separate file, that would eliminate most of my problems. But it’s just a pain to write code that way…
> if every exported variable or function was in a separate file...But it’s just a pain to write code that way…
I wonder if something that automatically organises your code into separate files would help...and fixes all the imports for you. A cli tool that would _lint/autofix_ your code organization. And then you would just need:
- IDE plugin (VSCode, IntelliJ) to show an organized IDE file tree
- cli tool to allow browsing organized code from terminal
- perhaps a custom fs for viewing your project in Finder/terminal and easily browsing in other tools
- git wrapper to show correct file names in diff/status
- chrome extension for Github to show flat files in a structured way
I wonder if something that automatically organises your code into separate files would help...and fixes all the imports for you. A cli tool that would _lint/autofix_ your code organization. And then you would just need:
- IDE plugin (VSCode, IntelliJ) to show an organized IDE file tree
- cli tool to allow browsing organized code from terminal
- perhaps a custom fs for viewing your project in Finder/terminal and easily browsing in other tools
- git wrapper to show correct file names in diff/status
- chrome extension for Github to show flat files in a structured way
That would be close, but you’ll end up with more annoyances if you remain focused on the FileSystem.
- auto-generated file names are gross. - 50+ imports in each file with bogus file names is gross
The FileSystem itself is a gross way to organize code after all.
I may want to organize my code in multiple different overlapping views. I may want to organize by behavior (state, actions, etc) or by feature (login, comment, etc).
My suggestion is an IDE that hides files from the user entirely. Something a little more akin to SmallTalk.
- auto-generated file names are gross. - 50+ imports in each file with bogus file names is gross
The FileSystem itself is a gross way to organize code after all.
I may want to organize my code in multiple different overlapping views. I may want to organize by behavior (state, actions, etc) or by feature (login, comment, etc).
My suggestion is an IDE that hides files from the user entirely. Something a little more akin to SmallTalk.
Is the filesystem interface itself the pain point? Would you rather be organizing URLs? Or is it the compound nature of source versus build machinery? Would you rather lodge your source and build machinery in polyglot?
Pycharm help you to refactor variables/functions/class as long as they have unique names in the first place. Otherwise everything messed up together.
They do it by auto renaming and something else.
They do it by auto renaming and something else.
A barrier to change is our current tools such as IDEs and VCS/Github are tightly tied to the files and folders paradigm.
Though, every tool that doesn't map 1-1 with the filesystem I have always hated the experience. Think: XCode, Visual Studio. But maybe it just hasn't been done right.
What are your thoughts on alternatives? Drawbacks to change?