Is there a software-engineering methodology for functional programming?(stackoverflow.com)
stackoverflow.com
Is there a software-engineering methodology for functional programming?
http://stackoverflow.com/q/4852251/89806
7 comments
From the UML standard:
Using UML with Functional Programming is like using a hammer to drive screws into walls.
The Unified Modeling Language is a visual language
for specifying, constructing, and documenting the
artifacts of systems. It is a general-purpose modeling
language that can be used with all major object and
component methods, and that can be applied to all
application domains (e.g., health, finance, telecom,
aerospace) and implementation platforms (e.g., J2EE, .NET).
Using UML with Functional Programming is like using a hammer to drive screws into walls.
I agree with the general direction that you are going, but consider at least one exception: http://en.wikipedia.org/wiki/Sequence_diagram. I find these sequence diagrams useful in whatever kind of systems that I am working with.
I feel that the disconnect is the presumption (which at one point in my career I was fully on board with) that Everything Is An Object.
Additionally, I have grown resistant to "methodology" more than anything. In my harsher moments, I argue that methodologies are insisted on by PHB who don't really understand programming and want something to insulate them from the reality of software development.
I feel that the disconnect is the presumption (which at one point in my career I was fully on board with) that Everything Is An Object.
Additionally, I have grown resistant to "methodology" more than anything. In my harsher moments, I argue that methodologies are insisted on by PHB who don't really understand programming and want something to insulate them from the reality of software development.
Sequence is only really useful when dealing with computations with side effects. So I have found the value of these diagrams much less as I try to keep the majority of the system referentially transparent.
The other huge disadvantage to sequence diagrams is that they are very poor at describing concurrency, and only work real well for simple single threaded systems.
The other huge disadvantage to sequence diagrams is that they are very poor at describing concurrency, and only work real well for simple single threaded systems.
Sequence is only really useful when dealing with computations with side effects
I disagree. If, for example, you are dealing with cooperating processes, then sequence diagrams are quite useful.
I disagree. If, for example, you are dealing with cooperating processes, then sequence diagrams are quite useful.
Draw a box. Write something in the box. Draw an arrow.
There. You're using UML.
Yes, the guys who spec out UML -- and many of the practitioners -- think of it as a do-all, end-all language of goodness and OOP.
But read your quote again. It's a visual language. A purpose of a language is to communicate. Who are you communicating with? Some folks think you're communicating with the computer itself, and their UML looks that way. But the language communicates to other people. You can be as loose or as detailed as you like, as long as you are communicating. The language spec simply provides you a range of depth in formality to use depending on your circumstances.
I teach this, I use this, and I've coded both OOP and FP for years. I think I know what the hell I'm talking about. Yes, according to the authors it can be used with all major OO systems, but that's a far cry from meaning the two are inseparable. It can also be used with railroad design, or industrial lighting systems.
I get really tired of dogmatism in programming. The idea that you can quote an authority as somebody who somehow controls the everyday practical uses of a thing. It's almost like we think of standards bodies as the programmers and people as the little machines that run the standards. UML is a tool. Pick it up. Use it. Understand what the tool is used for -- in a practical sense, not in a technical one. The definition of the tool is in the hands of the wielder, not some standards group. They just write specs. No one tool can do everything, and the directions that came with the tool were probably written by folks who see the entire world in terms of their tool.
If you're not designing your functional structures in some fashion -- including just in your head -- then I don't want to be maintaining your code. Whether or not you draw pretty pictures to do that or not is only important if you need to communicate that design to somebody else or sort it out in your head. And that's not even getting into all the other uses of UML -- component modeling, network diagramming, etc -- that have nothing at all to do with programming. If you think of UML artifacts as directly relating to OOP, go talk to a business domain modeler who uses UML, or a network analyst, or a BA.
It's a modeling language, not a coding language. You use it when you need to model. Modeling can help you in all kinds of situations. Or not. Draw a box. The rest of the discussion is around whether you need to draw the box or not, and it's so dependent on particular situations it's not germane to this conversation. Simple as that.
There. You're using UML.
Yes, the guys who spec out UML -- and many of the practitioners -- think of it as a do-all, end-all language of goodness and OOP.
But read your quote again. It's a visual language. A purpose of a language is to communicate. Who are you communicating with? Some folks think you're communicating with the computer itself, and their UML looks that way. But the language communicates to other people. You can be as loose or as detailed as you like, as long as you are communicating. The language spec simply provides you a range of depth in formality to use depending on your circumstances.
I teach this, I use this, and I've coded both OOP and FP for years. I think I know what the hell I'm talking about. Yes, according to the authors it can be used with all major OO systems, but that's a far cry from meaning the two are inseparable. It can also be used with railroad design, or industrial lighting systems.
I get really tired of dogmatism in programming. The idea that you can quote an authority as somebody who somehow controls the everyday practical uses of a thing. It's almost like we think of standards bodies as the programmers and people as the little machines that run the standards. UML is a tool. Pick it up. Use it. Understand what the tool is used for -- in a practical sense, not in a technical one. The definition of the tool is in the hands of the wielder, not some standards group. They just write specs. No one tool can do everything, and the directions that came with the tool were probably written by folks who see the entire world in terms of their tool.
If you're not designing your functional structures in some fashion -- including just in your head -- then I don't want to be maintaining your code. Whether or not you draw pretty pictures to do that or not is only important if you need to communicate that design to somebody else or sort it out in your head. And that's not even getting into all the other uses of UML -- component modeling, network diagramming, etc -- that have nothing at all to do with programming. If you think of UML artifacts as directly relating to OOP, go talk to a business domain modeler who uses UML, or a network analyst, or a BA.
It's a modeling language, not a coding language. You use it when you need to model. Modeling can help you in all kinds of situations. Or not. Draw a box. The rest of the discussion is around whether you need to draw the box or not, and it's so dependent on particular situations it's not germane to this conversation. Simple as that.
If you draw a box with an arrow, then you use a drawing program and not UML.
There is a reason, while in addition to OOP (object-oriented Programming) we are talking about OOA (object-oriented analysis) and OOD (object-oriented design). It's wrong to believe that OO is only for programming - it goes right into modelling and design. UML is originally created to support OO modelling.
There is a reason, while in addition to OOP (object-oriented Programming) we are talking about OOA (object-oriented analysis) and OOD (object-oriented design). It's wrong to believe that OO is only for programming - it goes right into modelling and design. UML is originally created to support OO modelling.
Um, there are actually quite a few boxes and arrow modelling specifications besides UML.
It depends on the level of abstraction and the view of the system that should be described. A model of the structure of a system with components, subcomponents and their relationship with each other does not include any information about the actual implementation of the components and the programming paradigm being used.
Also, a data model of the business domain is not concerned with any technical detail such as the components or subcomponents the system. You can model all these artifacts using UML.
Of course, using an UML class diagram for a detailed design describing the implementation of a component does not make sense when using functional programming.
Of course, using an UML class diagram for a detailed design describing the implementation of a component does not make sense when using functional programming.
While the standard is obviously written with OOP in mind UML is just as well a usable tool for desribing e.g. Erlang actors (sequence diagram) or the functionality buried in lisp packages. It helps drawing boundaries and responsibilities which might be just as important in FP.
The bulk of code generation tools and diagram "helpers" might be pointless in FP, but when you sketch on a napkin boxes and arrows just do their work in a way that even OO people can understand what you are talking about.
The bulk of code generation tools and diagram "helpers" might be pointless in FP, but when you sketch on a napkin boxes and arrows just do their work in a way that even OO people can understand what you are talking about.
Not necessarily: lisp has CLOS and components (if I am correctly understanding both your and their definition).
I do agree with the premise that FP feels a lot more agile to me, and thus benefits less from a big design-up-front effort.
I do agree with the premise that FP feels a lot more agile to me, and thus benefits less from a big design-up-front effort.
Lisp with CLOS is in no way Functional Programming. CLOS is the Common Lisp Object System. But even that is already a poor fit with UML, since CLOS is sufficiently different from the message-passing OO paradigm.
mm I don't know about UML but I take two approaches. When going through concepts I use boxes with arrows and texts - I bet UML people would laugh derisively if I claimed what I drew was UML, my shapes arent even consistent from diagram to diagram.
On the next level I use type diagrams ala category theory of functions to figure out what will be needed in code. The diagrams help in terms of thinking out the concepts, also about composition (how FP manages code reuse and extensibility) and setting up guard rails to guide my thinking. The diagrams are also underpinned by a mathematical language and I can do some basic algebra with the concepts before writing code. I use only the most basic parts of category theory though.
Through out this process, and most heavily at the start, I am usually also at the interactive testing out ideas.
On the next level I use type diagrams ala category theory of functions to figure out what will be needed in code. The diagrams help in terms of thinking out the concepts, also about composition (how FP manages code reuse and extensibility) and setting up guard rails to guide my thinking. The diagrams are also underpinned by a mathematical language and I can do some basic algebra with the concepts before writing code. I use only the most basic parts of category theory though.
Through out this process, and most heavily at the start, I am usually also at the interactive testing out ideas.
I bet UML people would laugh derisively
I think we have a disconnect here in most folk's opinion of UML -- what it's supposed to be used for, what kinds of people use it, how to do it "right" or not -- and actual reality.
Are you communicating effectively with others, with yourself? Do you diagram enough to help solve problems? Sounds to me like you're doing just fine. Sometimes my UML is so scratchy and rough that it looks like something from an impressionist painting. That's fine -- as long as it works for me.
Programming is full of little nooks and crannies where people get really tightly focused on one thing or another. web design comes to mind, or picking a programming language. They create a lot of complexity where most of the times just a little is needed.
The danger here is that the rest of us take a look at that complexity and kind of throw our hands up and say "hell with it. I'm just drawing little boxes. There's no way I'm going to do "correct" UML" Part of this is because other folks make things so complex, but part of it is ourselves: many times we feel that something like a UML standard is the same as a C++ standard. They're not. If I'm coding C++, I have to conform to the standard. If I don't the code won't work. But if I'm writing a web page? I can do it just about any dang way I want to. Standards be damned. Now in practice do I use standards with things like web design? Sure -- but only as much as I personally find useful.
To make it an all-or-nothing proposition is throwing the baby out with the bath water. What's required is to dive into one of these areas for a bit, learn the ropes, then use your common sense to apply just as much of it as you need to do whatever you are working on.
Modeling is a powerful tool in problem solving in all sorts of areas. Having knowledge of how to model things visually is a nifty little tool to have in your back pocket. That's true even if you'll never code a bit of OOP in the rest of your life.
I think we have a disconnect here in most folk's opinion of UML -- what it's supposed to be used for, what kinds of people use it, how to do it "right" or not -- and actual reality.
Are you communicating effectively with others, with yourself? Do you diagram enough to help solve problems? Sounds to me like you're doing just fine. Sometimes my UML is so scratchy and rough that it looks like something from an impressionist painting. That's fine -- as long as it works for me.
Programming is full of little nooks and crannies where people get really tightly focused on one thing or another. web design comes to mind, or picking a programming language. They create a lot of complexity where most of the times just a little is needed.
The danger here is that the rest of us take a look at that complexity and kind of throw our hands up and say "hell with it. I'm just drawing little boxes. There's no way I'm going to do "correct" UML" Part of this is because other folks make things so complex, but part of it is ourselves: many times we feel that something like a UML standard is the same as a C++ standard. They're not. If I'm coding C++, I have to conform to the standard. If I don't the code won't work. But if I'm writing a web page? I can do it just about any dang way I want to. Standards be damned. Now in practice do I use standards with things like web design? Sure -- but only as much as I personally find useful.
To make it an all-or-nothing proposition is throwing the baby out with the bath water. What's required is to dive into one of these areas for a bit, learn the ropes, then use your common sense to apply just as much of it as you need to do whatever you are working on.
Modeling is a powerful tool in problem solving in all sorts of areas. Having knowledge of how to model things visually is a nifty little tool to have in your back pocket. That's true even if you'll never code a bit of OOP in the rest of your life.
I see what you mean. It makes a lot of sense. Its just that I take a dual approach in looking at it. You can look at simple diagrams as subsets of UML as I think you do? But personally, I would say that simple effective diagrams are a superset of UML. You get UML when you take simple diagrams and add bombastic restrictions.
Its a matter of perspective and I can't say which makes more sense.
Its a matter of perspective and I can't say which makes more sense.
There's a fun little book called "Back of the Napkin" (warning: affiliate link ahead) http://amzn.to/ezUvgk
In it, the author develops his own little modeling system and talks about how to use it. While he goes a bit overboard (must be something in the water) his point is an excellent one: visual modeling conveys information at a bandwidth of an order or two of magnitude above conversations or documents.
I recently had a functional project. It was a greenfield project and had a MySql backend. I wanted to use dynamic data in some parts and relational in others, fully taking advantage of both FP and the relational model.
Problem was, I wasn't sure exactly how I wanted each piece to work. So, I picked up some scrap paper, a pencil, and sketched out some ideas.
They sucked.
A couple of days later I did it again. This time I saw some things I had been missing.
It also sucked.
A day after that, I sat down and sketched out what I wanted. Finally.
It didn't suck.
This entire process took a total of maybe 2 hours spread out over three days. A pencil and scrap paper. Soemtimes I just drew boxes with labels in them, sometimes I added aggregation or composition, sometimes I put in navigation. If I had continued to be stuck, I might have used a sequence diagram, or a collaboration diagram, or any one of a hundred other tools in my UML toolbox.
The point is: I never sat down and told myself "I'm going to be doing UML right now. I didn't get out some big fancy tool, and I didn't make something that looked like the wiring diagram for a nuclear device. That's not how it works. It's a spectrum. Some of my diagrams might have looked like an ER diagram, or just a bunch of junk on paper. Doesn't matter. The process of diagramming served it's purpose of organizing and capturing my thoughts about how the system would be structured. When I needed more complexity, I reached into my UML toolkit and added new complexity. If I didn't need it, I didn't use it. UML worked for me. I didn't work for it.
So I don't see UML as even being the same kind of thing as some of the other folks. There's no dual nature at all -- it's just a technical way of drawing pictures. And who doesn't need a picture now and then?
In it, the author develops his own little modeling system and talks about how to use it. While he goes a bit overboard (must be something in the water) his point is an excellent one: visual modeling conveys information at a bandwidth of an order or two of magnitude above conversations or documents.
I recently had a functional project. It was a greenfield project and had a MySql backend. I wanted to use dynamic data in some parts and relational in others, fully taking advantage of both FP and the relational model.
Problem was, I wasn't sure exactly how I wanted each piece to work. So, I picked up some scrap paper, a pencil, and sketched out some ideas.
They sucked.
A couple of days later I did it again. This time I saw some things I had been missing.
It also sucked.
A day after that, I sat down and sketched out what I wanted. Finally.
It didn't suck.
This entire process took a total of maybe 2 hours spread out over three days. A pencil and scrap paper. Soemtimes I just drew boxes with labels in them, sometimes I added aggregation or composition, sometimes I put in navigation. If I had continued to be stuck, I might have used a sequence diagram, or a collaboration diagram, or any one of a hundred other tools in my UML toolbox.
The point is: I never sat down and told myself "I'm going to be doing UML right now. I didn't get out some big fancy tool, and I didn't make something that looked like the wiring diagram for a nuclear device. That's not how it works. It's a spectrum. Some of my diagrams might have looked like an ER diagram, or just a bunch of junk on paper. Doesn't matter. The process of diagramming served it's purpose of organizing and capturing my thoughts about how the system would be structured. When I needed more complexity, I reached into my UML toolkit and added new complexity. If I didn't need it, I didn't use it. UML worked for me. I didn't work for it.
So I don't see UML as even being the same kind of thing as some of the other folks. There's no dual nature at all -- it's just a technical way of drawing pictures. And who doesn't need a picture now and then?
UML is more naturally mapped to OOP than to flat relations, and relational programming is more natural for a functional language.
See http://web.mac.com/ben_moseley/frp/frp.html
See http://web.mac.com/ben_moseley/frp/frp.html
Sadly, programming is not a step-by-step process, and so the search for methodologies is pretty fruitless IMHO.
Instead of looking for "proven" programming methodologies, one should develop an aptitude and taste for how to write good programs. This can only come from "doing" it and seeing how other people do it.
"Bottom up programming" is not a methodology. It's not a step-by-step process, nor is it a set of strict rules and guidelines. It's just a general idea; and it's not something you implement, it's a mindset you absorb and imitate.
By the way, UML & co. are horrible abominations and you should steer away from them. If you want to sketch something out on a piece of paper, just use your common sense: draw boxes, circles, arrows, trees, or whatever you like.
Instead of looking for "proven" programming methodologies, one should develop an aptitude and taste for how to write good programs. This can only come from "doing" it and seeing how other people do it.
"Bottom up programming" is not a methodology. It's not a step-by-step process, nor is it a set of strict rules and guidelines. It's just a general idea; and it's not something you implement, it's a mindset you absorb and imitate.
By the way, UML & co. are horrible abominations and you should steer away from them. If you want to sketch something out on a piece of paper, just use your common sense: draw boxes, circles, arrows, trees, or whatever you like.
While I love doing bottom-up programming for my own projects or with a small team, I think there is something to be said for many elements of the top-down approach. It is often wasteful and overbearing in its process, but when engineering large systems, having a general consensus on modelling (UML being the most common form) and code structure does have its payoffs. Not all team members are equals in their passion for writing code, their interest in the current project, or their general competency. If you have to write code for the lowest common denominator - and I don't mean this in any deprecatory sense - clarity and consistency are tangible benefits.
A clear methodology also provides both a shield to the developers and a benefit for those involved in the project but not directly working on the code.
A clear methodology also provides both a shield to the developers and a benefit for those involved in the project but not directly working on the code.
A methodology for software engineering or a methodology for programming? Not necessarily the same, as I see it.
You may have one methodology you use for your overall engineering. A framework for supporting your problem analysis, architecting your system, and doing system design. You may have another methodology for detailed design and implementation of the parts of the system that goes into programs. It may of course be advantageous if the latter support the paradigm of your programming language. The former less so.
A software based system has three main dimensions, state, event streams, and functionality. Your methodology should support all three dimensions evenly. If it does not, you should be aware of that and do your own compensation.
You may have one methodology you use for your overall engineering. A framework for supporting your problem analysis, architecting your system, and doing system design. You may have another methodology for detailed design and implementation of the parts of the system that goes into programs. It may of course be advantageous if the latter support the paradigm of your programming language. The former less so.
A software based system has three main dimensions, state, event streams, and functionality. Your methodology should support all three dimensions evenly. If it does not, you should be aware of that and do your own compensation.
This is one of my least favorite types of questions. It's like asking, "is there a painting methodology for latex-based paints?" Well, no. You use latex-based paints just like you use oil-based paints. Apply it to the paper until your sense of taste tells you you have art.
Programming is the same way. Write a small amount of code until you have something beautiful. This methodology will get you far with any language.
The hard part is refining the sense of taste. Just like there are a lot of poor starving artists who never get it, there are going to be a lot of poor starving programmers who never get it. That's sad, but that's the reality of creativity. Not everyone has it.
But you can try, and the way you try is by reading other programs and writing your programs in a similar style. There are low-level things and high-level things to look out for, and you're not going to get them by reading Stack Overflow. You have to play with a language and its libraries on a daily basis for years. Then you will begin to find the "methodology" you are looking for.
There are no easy answers.
Programming is the same way. Write a small amount of code until you have something beautiful. This methodology will get you far with any language.
The hard part is refining the sense of taste. Just like there are a lot of poor starving artists who never get it, there are going to be a lot of poor starving programmers who never get it. That's sad, but that's the reality of creativity. Not everyone has it.
But you can try, and the way you try is by reading other programs and writing your programs in a similar style. There are low-level things and high-level things to look out for, and you're not going to get them by reading Stack Overflow. You have to play with a language and its libraries on a daily basis for years. Then you will begin to find the "methodology" you are looking for.
There are no easy answers.
Well, no. You use latex-based paints just like you use oil-based paints. Apply it to the paper until your sense of taste tells you you have art.
Except you don't use oil-based paints the same way you use latex. Most of the time you wouldn't use either on paper.
Painting is a bad example to argue your point. Painting requires adherence to a certain technical methodology or else you don't get anywhere near your envisioned artistic result.
Except you don't use oil-based paints the same way you use latex. Most of the time you wouldn't use either on paper.
Painting is a bad example to argue your point. Painting requires adherence to a certain technical methodology or else you don't get anywhere near your envisioned artistic result.
UML and its manipulations are only a relatively small thing -- the Unified Process covers much more than that.
So, as a question on process, OO or functional does not matter much: the software is treated really as modules or parts. Whether those are composed of lists or objects, methods or functions, is a minor concern. Process deliberately works at a higher level. There is some specialisation to OO, but translating it to FP would seem essentially straightforward.
But as a question on that particular sub-area, it is valid and interesting, and there does not seem to be such a functional specialisation.
Perhaps there is a way to answer by looking back to the history of the Unified Process. A substantial basis of it was developed by Jacobson while working at Ericsson on phone switches etc. And that was the same place and same kind of work where Erlang was later created. So maybe investigating how to use Erlang might lead to some approaches.
So, as a question on process, OO or functional does not matter much: the software is treated really as modules or parts. Whether those are composed of lists or objects, methods or functions, is a minor concern. Process deliberately works at a higher level. There is some specialisation to OO, but translating it to FP would seem essentially straightforward.
But as a question on that particular sub-area, it is valid and interesting, and there does not seem to be such a functional specialisation.
Perhaps there is a way to answer by looking back to the history of the Unified Process. A substantial basis of it was developed by Jacobson while working at Ericsson on phone switches etc. And that was the same place and same kind of work where Erlang was later created. So maybe investigating how to use Erlang might lead to some approaches.
Lots of talk of UML. Surely if there is a software engineering concept that suits functional programming, it's formal methods.
http://en.wikipedia.org/wiki/Formal_methods
http://en.wikipedia.org/wiki/Formal_methods
http://clojure.blip.tv/file/4457042/ This video with Rich Hickey(Creator of clojure) is the closest to a functional programming methodology you will find.
Software engineering and programming are simply problem solving with an abstract tool set.
Software engineering and programming are simply problem solving with an abstract tool set.
UML is a modeling language for talking about abstract constructs. That means it's as good for working with FP as OOP. What you model in UML and how much you do and how you do it? That's an entirely different thing.
You have to have data. In FP, more than anywhere else, data is king. So I use UML to initially model data structures, just like I would no matter what paradigm I was working with. Let Codd be your copilot.
It's never right. There are those who model code ad infinitum and try to code the world completely top-down. This never works. There are those who don't model anything and try to code completely bottom-up. This works, but can become unmaintainable if not implemented correctly. The trick is to do a little top-down, a little bottom-up, a little top-down, etc. Usually within a couple cycles of this your basic program/data structure is nailed and you can move to being bottom-up and refactoring. I've probably described it more complicated than it actually is. But it works the same in OOP as FP (although I've noticed a quicker tendency to move towards bottom-up in FP and a tendency to stay top-down in OOP)
If the questioner is talking about software development methodologies, like Agile or RUP or CMM or waterfall, then it's also the same. After years of beating ourselves up in various ways, we've pretty much all agreed that short timeboxes, with concrete deliverables, public demonstrations, and constant adaptation to changing conditions beats everything else. However else you want to do it is up to you, but it should conform to those basics. FP or OOP really has nothing to do with it.
I think maybe the questioner is making the assumption that all this _stuff_, UML sequence diagrams and use cases and tiered architectures and whatnot -- that it's really how you program in OOP. But it's not. It never was. All that crap is just various tools that are useful in various situations to communicate about complex problems. They're not actually how you _do_ anything.