Organizing Large C Programs(cse.shirazu.ac.ir)
cse.shirazu.ac.ir
Organizing Large C Programs
http://www.cse.shirazu.ac.ir/~azimi/prog88/lectures/15large_programs.pdf
16 comments
> How you organize million line projects is an interesting topic. Does anybody have any actual links?
Large Scale C++ Software Design by John Lakos is about that topic. Of course, C++ is its own special hell when it comes to physical coupling because of what tends to go (and sometimes has to go) in header files.
It's not a terrible book when considering the lack of alternative sources for the material covered, but a lot of its suggestions are showing their advanced age and were dubious and controversial even at the time of writing. Nowadays you have to take into account precompiled headers, unity builds, distributed build clusters, etc, when considering trade-offs in physical design for large code bases.
Free startup idea: Incredibuild in the cloud! Tell us how how fast you want your code base to build and you got it! You'd have to design and implement finer-grained distributed compiling and linking to overcome Amdahl's Law speed-up barriers and make it really competitive with something like Incredibuild for larger companies. But you could bootstrap it with coarser-grained distribution (distributed compiling at the file granularity but with non-distributed monolithic linking) a la earlier versions of Incredibuild while targeting smaller development shops without a lot of spare CPU cycles floating around the office. Making it dead simple to drop in and use with existing code bases would be key.
Large Scale C++ Software Design by John Lakos is about that topic. Of course, C++ is its own special hell when it comes to physical coupling because of what tends to go (and sometimes has to go) in header files.
It's not a terrible book when considering the lack of alternative sources for the material covered, but a lot of its suggestions are showing their advanced age and were dubious and controversial even at the time of writing. Nowadays you have to take into account precompiled headers, unity builds, distributed build clusters, etc, when considering trade-offs in physical design for large code bases.
Free startup idea: Incredibuild in the cloud! Tell us how how fast you want your code base to build and you got it! You'd have to design and implement finer-grained distributed compiling and linking to overcome Amdahl's Law speed-up barriers and make it really competitive with something like Incredibuild for larger companies. But you could bootstrap it with coarser-grained distribution (distributed compiling at the file granularity but with non-distributed monolithic linking) a la earlier versions of Incredibuild while targeting smaller development shops without a lot of spare CPU cycles floating around the office. Making it dead simple to drop in and use with existing code bases would be key.
C is not C++. They have different issues, namespaces, abi linkage, libraries, various other things. They also have different application domains.
That is true, but the concepts in the Lakos book can be applied to both C and C++. Splitting units of code based upon physical dependencies and having a strong naming system are the basic underlying points. (Disclaimer: I work with John Lakos)
Thank you. The other guy appears to have been thinking I was stating that flippantly, when really a book that is about C++ file organization has to be trying to also be about C file organization.
C++ techniques don't apply to C only projects that often.
C++ techniques don't apply to C only projects that often.
Oh, hey, you're _that_ guy! Every thread needs one of you. Thank you so much for putting me in my place by stating the obvious!
Had you said "C or C++" instead of C++, I might not have said that.
Many many many C++ users act like we just don't know C++ or something when use C in systems programming instead of C++, and assume "We just need to upgrade to C++" or the whatever.
If a book only had C++ techniques, it wouldn't be speaking to the topic of this post (which is large scale C programming)
And the comment at the same level at the guy who knows the author directly speaks to how it ALSO applies to C, showing the relevance of your post to the topic
I wasn't being flippant
Many many many C++ users act like we just don't know C++ or something when use C in systems programming instead of C++, and assume "We just need to upgrade to C++" or the whatever.
If a book only had C++ techniques, it wouldn't be speaking to the topic of this post (which is large scale C programming)
And the comment at the same level at the guy who knows the author directly speaks to how it ALSO applies to C, showing the relevance of your post to the topic
I wasn't being flippant
What's this "we"? In the past I've worked on C++ code bases with millions of lines of code. But at RAD we almost exclusively work in C (and assembly, for dozens of platforms). C++ inherited C's broken model for separate compilation with all that implies about physical design for large code bases. As I suggested in the second sentence of my original post, C++ aggravates that problem and therefore calls for special solutions, but there is still plenty of overlap.
None of the context of your experience was in your original comment that talked about C++ organization in a thread about C organization. Sorry there was a misunderstanding.
Could you elaborate more on what trade-offs you see? Have you hit specific problems within your own code base?
That is a big question that deserves a big answer beyond the scope of Hacker News threads. But a few points taken at random:
- The handle-body idiom loosens physical coupling but adds immense overhead to both time and space. For many applications in C++'s domain this is simply unacceptable, full stop. Applying the idiom ubiquitously would mean throwing out most of C and C++'s advantages for programmers in my kind of work.
- For real-time applications (soft or hard), it is important that the performance delta between debug and release builds is kept to a minimum, so you can use effective tools to debug a reasonable facsimile of the application in its final shipping form. This has numerous corollaries. For example, if member function definitions are generally put in .cpp files rather than header files to reduce physical coupling, then you get no cross-module inlining except when expensive, debugging unfriendly and incremental rebuilding unfriendly optimizations like link-time code generation are enabled. In principle, C has the same problem. But in practice, C++ programmers tend to use lots of small function definitions that each do a small amount of work and then defer to other functions. That style of coding depends on aggressive inlining for good performance; it's common to see a much, much larger performance gap between debug and release mode performance with C++ code bases than with C++ code bases for video games (which is what I know best).
- The handle-body idiom loosens physical coupling but adds immense overhead to both time and space. For many applications in C++'s domain this is simply unacceptable, full stop. Applying the idiom ubiquitously would mean throwing out most of C and C++'s advantages for programmers in my kind of work.
- For real-time applications (soft or hard), it is important that the performance delta between debug and release builds is kept to a minimum, so you can use effective tools to debug a reasonable facsimile of the application in its final shipping form. This has numerous corollaries. For example, if member function definitions are generally put in .cpp files rather than header files to reduce physical coupling, then you get no cross-module inlining except when expensive, debugging unfriendly and incremental rebuilding unfriendly optimizations like link-time code generation are enabled. In principle, C has the same problem. But in practice, C++ programmers tend to use lots of small function definitions that each do a small amount of work and then defer to other functions. That style of coding depends on aggressive inlining for good performance; it's common to see a much, much larger performance gap between debug and release mode performance with C++ code bases than with C++ code bases for video games (which is what I know best).
I agree. I was looking forward to some useful information, but this is really the first thing you learn after "hello world" in C. I'm somewhat baffled as to why this was even submitted to HN much less how it made it to the front page.
In my experience, managing C projects becomes frustrating pretty fast but maybe that's a problem with all languages (most of my limited experience is in C).
In my experience, managing C projects becomes frustrating pretty fast but maybe that's a problem with all languages (most of my limited experience is in C).
[deleted]
Thanks for the warning: I too was hoping that this was about organizing large C programs. Makes me glad that I skim the comments first!
I agree that this is kind of mistitled for HN - this is an intro to C - but it is actually a nice intro to C headers and linking if you have never seen that before.
[deleted]
How you organize million line projects is an interesting topic. Does anybody have any actual links?