Clojure.spec – Rationale and Overview(clojure.org)
clojure.org
Clojure.spec – Rationale and Overview
http://clojure.org/about/spec
130 comments
Agreed that this is a big step, but what is being included in core is very distinct from Prismatic Schema. For example the way keysets vs values are handled is intentionally different from schema.
The rationale is a good read.
The rationale is a good read.
Yes, I've read and see the difference from schema.
The huge advantage with this, is that even though the approach is different, it supersedes schema's abilities and it will be a require away for every library and application author.
The code has just landed [1]. It's not a .cljc file yet, but fingers crossed this becomes a thing at some point.
[1]: https://github.com/clojure/clojure/commit/3394bbe616c6202618...
[1]: https://github.com/clojure/clojure/commit/3394bbe616c6202618...
It will not be a cljc file, but it will be added to ClojureScript too.
Luckily Common Lisp already has ways to do that:
Functions have defined keyword argument lists and can have type declarations which for example a compiler like SBCL can partially check at compile-time.
Common Lisp uses for anything more complex than lists of keyword value combinations CLOS classes, which form a multiple inheritance hierarchy and types for slots.
That way I can look into a debugger at runtime and see what functions are actually supposed to receive and what these collects of keyword/value maps actually mean (since they are CLOS classes).
Having simple lists as maps was popular with assoc lists in the 70s - basically the same as hash tables, but slower. Then lots of dynamic object systems were developed which were in essence dynamic key/value sets. Later various object systems appeard, with CLOS as a local maximum.
Where Rich Hickey points to RDF, there is a lot prior art in Lisp as well -> Frame systems / description logics / ....
http://wilbur-rdf.sourceforge.net/docs/ivanhoe.html
http://franz.com/agraph/racer/krss-spec.pdf and zillion others
CLIM, the Common Lisp interface manager, uses 'presentation types' to specify user level data structures...
Functions have defined keyword argument lists and can have type declarations which for example a compiler like SBCL can partially check at compile-time.
Common Lisp uses for anything more complex than lists of keyword value combinations CLOS classes, which form a multiple inheritance hierarchy and types for slots.
That way I can look into a debugger at runtime and see what functions are actually supposed to receive and what these collects of keyword/value maps actually mean (since they are CLOS classes).
Having simple lists as maps was popular with assoc lists in the 70s - basically the same as hash tables, but slower. Then lots of dynamic object systems were developed which were in essence dynamic key/value sets. Later various object systems appeard, with CLOS as a local maximum.
Where Rich Hickey points to RDF, there is a lot prior art in Lisp as well -> Frame systems / description logics / ....
http://wilbur-rdf.sourceforge.net/docs/ivanhoe.html
http://franz.com/agraph/racer/krss-spec.pdf and zillion others
CLIM, the Common Lisp interface manager, uses 'presentation types' to specify user level data structures...
Clojure, with its dabbling in schemas and rule systems / logic programming (as a substitute for conditional statements) and thanks to its great tools (like figwheel or devcards), could really be establishing a cheaper, "as-good" alternative to static typing and full formal verification.
Instead of encoding constraints as type signatures, the Clojure folks (true to character) encode them in data. In my eyes a very interesting, pragmatic trade-off between expressiveness and automatic verifiability.
Instead of encoding constraints as type signatures, the Clojure folks (true to character) encode them in data. In my eyes a very interesting, pragmatic trade-off between expressiveness and automatic verifiability.
> dabbling in schemas and rule systems / logic programming (as a substitute for conditional statements
That sounds interesting. Could you elaborate or post a link?
That sounds interesting. Could you elaborate or post a link?
As a side note in his talk "Simple Made Easy" (https://www.infoq.com/presentations/Simple-Made-Easy, around minute 42)
Rich Hickey mentions, that conditional statements are complex, because they spread (business-)logic throughout the program.
As a simpler (in the Hickey-sense) alternative, he lists rule systems and logic programming. For example, keeping parts of the business logic ("What do we consider an 'active' user?", "When do we notify a user?", etc...) as datalog expressions, maybe even storing them in a database, specifies them all in a single place. This helps to ensure consistency throughout the program. One could even give access to these specifications to a client, who can then customise the application directly in logic, instead of chasing throughout the whole code base.
Basically everyone involved agrees on a common language of predicates explicitly, instead of informally in database queries, UI, application code, etc...
But Hickey also notes that this thinking is pretty "cutting-edge" and probably not yet terribly practical.
As a simpler (in the Hickey-sense) alternative, he lists rule systems and logic programming. For example, keeping parts of the business logic ("What do we consider an 'active' user?", "When do we notify a user?", etc...) as datalog expressions, maybe even storing them in a database, specifies them all in a single place. This helps to ensure consistency throughout the program. One could even give access to these specifications to a client, who can then customise the application directly in logic, instead of chasing throughout the whole code base.
Basically everyone involved agrees on a common language of predicates explicitly, instead of informally in database queries, UI, application code, etc...
But Hickey also notes that this thinking is pretty "cutting-edge" and probably not yet terribly practical.
It can work. My current company uses a rule system to represent most of our business logic since it is so dynamic. The downside is that we have to rebuild the entire graph into memory (times the number of threads, times the number of app servers) every time anything changes (which is constant).
Facebook wrote about rebuilding a similar system in Haskell that only changes memory incrementally, so it's definitely possible to do better.
Facebook wrote about rebuilding a similar system in Haskell that only changes memory incrementally, so it's definitely possible to do better.
Interesting note, thank you.
Are you referring to "Sigma" https://code.facebook.com/posts/745068642270222/fighting-spa... ?
That's the one.
Quick refs to logic programming and pattern matching in Clojure.
core.logic https://github.com/clojure/core.logic core.match https://github.com/clojure/core.match
core.logic https://github.com/clojure/core.logic core.match https://github.com/clojure/core.match
I assume @ComNik is referring to core.schema and core.logic[1] here, respectively? (if not, sorry @ComNik)
1: https://clojure.github.io/core.logic/
1: https://clojure.github.io/core.logic/
Important to note, that as data, they can be consumed by other systems for other purposes. For example, it's feasible to consume a spec within Alloy and use a SAT solver (which is made even easier given the split on keyset). This pushes spec from design and requirements validation, into runtime constraints, and on through to tests and verification.
Cheaper? How so?
Full, formal verification of complex systems (especially in the distributed case) requires a lot of modelling effort, beyond what is needed for "mere" implementation. A lot of important constraints cannot even be expressed in most type systems. And if one uses a specialised modelling language for the proof, the actual implementation might introduce bugs.
Also relevant to real, shipping software: specifying a program in such that detail as required for automatic verification makes it even harder to change (which, given infinite time and money, is indeed a very very good property!).
This just goes to say, that a type system can be very helpful, but is ultimately just a part of regular testing.
So for most companies and most developers, anything that aides in keeping documentation up-to-date, writing or generating tests and helping developers understand what they are reading is probably a better ROI.
This just goes to say, that a type system can be very helpful, but is ultimately just a part of regular testing.
So for most companies and most developers, anything that aides in keeping documentation up-to-date, writing or generating tests and helping developers understand what they are reading is probably a better ROI.
If static type systems cannot do everything, it does not make sense to me to conclude that they shouldn't then be used for anything. Type systems aren't part of "regular testing", they're part of compile-time error checking.
That was not the intension behind my comment at all. I rely on type systems a lot, myself.
I was trying to note that Clojure, as a dynamic language, is making a (to my eyes) very interesting choice, of doubling down on these dynamic methods of verification. For some uses, I can see this as the better choice, for others it isn't and won't be.
I was trying to note that Clojure, as a dynamic language, is making a (to my eyes) very interesting choice, of doubling down on these dynamic methods of verification. For some uses, I can see this as the better choice, for others it isn't and won't be.
Sure, and I can't fault anyone for wanting less dangerous dynamic languages. But to me it's kind of like wanting less dangerous cigarettes.
As I commented elsewhere, Clojure very deliberately adopts the stance that type systems like Haskell's are valuable but heavyweight, and chose development speed and concision over types.
In my own work, I chose Clojure because it mirrored my experiences: that type systems weren't worth the trade-offs for me and my field.
Would I prefer stronger typing in different domains (longer project times available, greater consequences of failure like medical devices, etc.)? Sure, and I have in the past. But it's not the case that dynamic languages are always the incorrect choice, like smoking cigarettes.
In my own work, I chose Clojure because it mirrored my experiences: that type systems weren't worth the trade-offs for me and my field.
Would I prefer stronger typing in different domains (longer project times available, greater consequences of failure like medical devices, etc.)? Sure, and I have in the past. But it's not the case that dynamic languages are always the incorrect choice, like smoking cigarettes.
Or like leaving the house without a suit of plate mail.
When you prove that a language like Clojure (untyped, but with dispatch that is no more dynamic than that of many modern typed languages) combined with a construct like specs is significantly less safe than any typed language, then you can make that statement. It's not like untyped and typed are two binary categories; otherwise, I could say that wanting less dangerous typed languages as opposed to formal tools are just like wanting less dangerous cigarettes. These ideas exist on a spectrum.
> Instead of encoding constraints as type signatures, the Clojure folks (true to character) encode them in data.
I've never bought this line of thinking when the "Clojure folks" also say "code is data", because then a syntactic type signature you write down is also just data. If you want to consume the type signature (or spec signature or whatever) as syntax (as a sibling comment suggests), use a macro or any other program which consumes programs.
The thing I do see as being important is not how you write these things down, but whether they have an accessible representation at run/read/compile/whenever time.
Lastly, I strongly disagree these approaches are meant as a replacement for static typing or formal verification because (as far as I can tell) a value can be checked against a spec on demand, but makes zero guarantees as to what that value does in other parts of a program. This is also in contrast to Racket contracts, which will give you the correct blame information between parties using and not using contracts.
I've never bought this line of thinking when the "Clojure folks" also say "code is data", because then a syntactic type signature you write down is also just data. If you want to consume the type signature (or spec signature or whatever) as syntax (as a sibling comment suggests), use a macro or any other program which consumes programs.
The thing I do see as being important is not how you write these things down, but whether they have an accessible representation at run/read/compile/whenever time.
Lastly, I strongly disagree these approaches are meant as a replacement for static typing or formal verification because (as far as I can tell) a value can be checked against a spec on demand, but makes zero guarantees as to what that value does in other parts of a program. This is also in contrast to Racket contracts, which will give you the correct blame information between parties using and not using contracts.
You make a very good point, the comparison was flawed there.
> The thing I do see as being important is not how you write these things down, but whether they have an accessible representation at run/read/compile/whenever time.
This is a better way to put it. The second important factor for me is expressiveness. With spec or any other contracts-like system one gets the full power of the language to express constraints. Of course type systems are not artificially restricted in this regard, they simply make a different trade-off.
I hope my comment did not come off as a riff on static vs dynamic typing, and I don't think any contract system is meant to replace type systems. Until expressing all important program specifications formally becomes viable for everyone (maybe through this work https://www.math.ias.edu/vladimir/current_work?), a less-formal, dynamic approach seems very attractive.
> The thing I do see as being important is not how you write these things down, but whether they have an accessible representation at run/read/compile/whenever time.
This is a better way to put it. The second important factor for me is expressiveness. With spec or any other contracts-like system one gets the full power of the language to express constraints. Of course type systems are not artificially restricted in this regard, they simply make a different trade-off.
I hope my comment did not come off as a riff on static vs dynamic typing, and I don't think any contract system is meant to replace type systems. Until expressing all important program specifications formally becomes viable for everyone (maybe through this work https://www.math.ias.edu/vladimir/current_work?), a less-formal, dynamic approach seems very attractive.
I hear you on expressiveness of using the language. Of course (and as you mention) the strength of restricting the expressiveness is that it becomes decidable in some amount of acceptable time for most programs.
I agree with your last sentiment but I don't think it has to be any less "formal" than something like Coq. I just think we shouldn't be so concerned with proving all properties of a program before running. And then we can turn the knob to adjust how much to prove during vs. before running.
I agree with your last sentiment but I don't think it has to be any less "formal" than something like Coq. I just think we shouldn't be so concerned with proving all properties of a program before running. And then we can turn the knob to adjust how much to prove during vs. before running.
> a value can be checked against a spec on demand, but makes zero guarantees as to what that value does in other parts of a program
The spec is just metadata. How it's used is part of the tooling. I see nothing precluding using specs in tools that employ and analyze that information in different ways.
The spec is just metadata. How it's used is part of the tooling. I see nothing precluding using specs in tools that employ and analyze that information in different ways.
Yep you are totally correct. I only point this out because the tool presented here does not do the same thing as other contract systems, but could (as you say) be used as a building block to get there.
> spec has been designed from the ground up to directly support generative testing via test.check. When you use spec you get generative tests for free.
Great, I've been holding out on digging into the test.check generator syntax... looks like my laziness paid off!
Great, I've been holding out on digging into the test.check generator syntax... looks like my laziness paid off!
It's still worth digging in! There are times when you will want to write your own generators.
It's unfortunate that Rich Hickey (assume he's the author) can't write this without trolling about type systems. It's pretty clear to anyone who's read the literature, or has thought for a few moments, that contract systems provide different guarantees to type systems, so no need to poke at type systems throughout the Guidelines.
I think you're "seeing what you want to" in his comments. One of the reasons I appreciate clojure and the community so much is that it's one of the few that doesn't actually troll things.
If anything, the community tends to give really well thought out reasoning around why they make the decisions they are making.
When making an argument, sometimes it's important to explain why something is different, or why you've made the choice you made. Just because you explain why your idea is different doesn't make it trolling.
If anything, the community tends to give really well thought out reasoning around why they make the decisions they are making.
When making an argument, sometimes it's important to explain why something is different, or why you've made the choice you made. Just because you explain why your idea is different doesn't make it trolling.
Yes, especially since there's no reason you can't have contracts in a language with a type system.
> or has thought for a few moments
As is obvious even to a child...
As is obvious even to a child...
I don't think he's trolling, he's proposing spec due to issues he personally sees in type systems whilst acknowledging that clojure without types or spec or whatever is definitely missing "something" ... it's just that something doesn't have to be types.
Given his comments here he seems well aware that not everyone shares his views on type systems and that is fine:
https://groups.google.com/forum/#!msg/clojure/DUKeo7sT4qA/TU...
For one it goes with the territory. When designing a language you need to emphasize its strong points and Clojure's strong point is its (sane) dynamic nature and they are playing to its strengths for example in solutions such as their Transducers.
I don't view it as trolling. The mentality and style in dynamic languages are simply different. Once you add static typing, you break that mentality, replacing it with something different. Not inferior, just different. The patterns, the solutions end up looking different. This is why static typing added to a LISP like Clojure can only be optional and probably less powerful than what you can get with something like Haskell.
Or in other words, type systems are not practical for Clojure. There are attempts, like core.typed. Such solutions don't work very well because as said, adding types changes what you're allowed to do, even if those types are optional. So he's speaking primarily in the context of Clojure.
Also, poking fun at your competition is good marketing. People love it :-P
I don't view it as trolling. The mentality and style in dynamic languages are simply different. Once you add static typing, you break that mentality, replacing it with something different. Not inferior, just different. The patterns, the solutions end up looking different. This is why static typing added to a LISP like Clojure can only be optional and probably less powerful than what you can get with something like Haskell.
Or in other words, type systems are not practical for Clojure. There are attempts, like core.typed. Such solutions don't work very well because as said, adding types changes what you're allowed to do, even if those types are optional. So he's speaking primarily in the context of Clojure.
Also, poking fun at your competition is good marketing. People love it :-P
It's quite annoying, but he definitely has a tendency to do this, though I wouldn't go so far as to call it 'trolling' per se. I think he's more sort of speaking to the 'in-group' and that can often be quite off-putting for people who are not members of that group because all the "but..." caveats are missing.
Btw, remember 'transducers'? He did the exact same thing there and claimed (in so many words) that it couldn't really be done in a typed language until people showed that, yes, it really could be done in a typed language such as Haskell. I suspect it's because he just doesn't know enough Haskell and couldn't get the type signatures to work. However, it's a big leap from "I cannot figure out how to do it" to "It cannot be done" and it requires a wee bit of (probably 'well-earned') arrogance to make that leap. :)
Btw, remember 'transducers'? He did the exact same thing there and claimed (in so many words) that it couldn't really be done in a typed language until people showed that, yes, it really could be done in a typed language such as Haskell. I suspect it's because he just doesn't know enough Haskell and couldn't get the type signatures to work. However, it's a big leap from "I cannot figure out how to do it" to "It cannot be done" and it requires a wee bit of (probably 'well-earned') arrogance to make that leap. :)
Rich provided the Haskell translation at https://www.reddit.com/r/haskell/comments/2cv6l4/clojures_tr.... His point was that some aspects of transducers were not possible to capture well in a type signature.
Some aspects, as in: not possible to describe state machines by types.
This is a well known problem, an active subject of research actually. Add asynchrony in the equation and you're out of luck. And transducers can also have asynchronous behavior.
This is a well known problem, an active subject of research actually. Add asynchrony in the equation and you're out of luck. And transducers can also have asynchronous behavior.
> Some aspects, as in: not possible to describe state machines by types.
I really don't understand this, I mean:
Add a function
EDIT: Oh, I think I see what you're saying. This doesn't provide that many guarantees at the type level about state transitions, but I'd counter that: a) You have other options than "Int", so you could manage valid states more strictly, and b) There's such a thing as Dependently-Typed languages, say Idris, so you can go even further. Then, perhaps Dependently-Types languages was what you meant by "active subject of research". I'd probably agree on that :)
> Add asynchrony in the equation and you're out of luck.
Not sure how that's to be interpreted? Are you talking about asynchronous updates to state, or...?
> And transducers can also have asynchronous behavior.
Is there a formal description of what semantics they have in that scenario? Or are we just going by implementation details?
EDIT: I should say: I'm not being faceitous or going for any kind of 'gotcha' type thing here. I'm really curious; maybe I'm just misunderstanding your post.
I really don't understand this, I mean:
data TrafficLight = Red Int
| Green Int
| Yellow
Add a function
nextState :: Input -> State -> State
nextState = ...
EDIT: Oh, I think I see what you're saying. This doesn't provide that many guarantees at the type level about state transitions, but I'd counter that: a) You have other options than "Int", so you could manage valid states more strictly, and b) There's such a thing as Dependently-Typed languages, say Idris, so you can go even further. Then, perhaps Dependently-Types languages was what you meant by "active subject of research". I'd probably agree on that :)
> Add asynchrony in the equation and you're out of luck.
Not sure how that's to be interpreted? Are you talking about asynchronous updates to state, or...?
> And transducers can also have asynchronous behavior.
Is there a formal description of what semantics they have in that scenario? Or are we just going by implementation details?
EDIT: I should say: I'm not being faceitous or going for any kind of 'gotcha' type thing here. I'm really curious; maybe I'm just misunderstanding your post.
I'm as clueless as you are about that asynchrony business, but would like to point out that you are not fully describing your state machine within the type system unless the typechecker prohibits you from defining (nextState Yellow == Green).
That would be an invalid definition since you must provide an (integer) parameter to Green. Effectively you must "magic" a default value for what happens when Yellow goes to Green.
(Obviously in my sketched scenario the value would really come from Input.)
(Obviously in my sketched scenario the value would really come from Input.)
That's true. My point was just that your type definition does not tell the compiler/typechecker that the light, if it is currently yellow, must become red before it can become green.
That's a rule of the state machine and, if the compiler doesn't stop you from writing a function that breaks that rule then your state machine has not been modeled fully within the type system.
I'm pretty sure this is what Hickey et al mean when they say that Haskell's type system is not powerful enough to describe a state machine.
That's a rule of the state machine and, if the compiler doesn't stop you from writing a function that breaks that rule then your state machine has not been modeled fully within the type system.
I'm pretty sure this is what Hickey et al mean when they say that Haskell's type system is not powerful enough to describe a state machine.
> That's true. My point was just that your type definition does not tell the compiler/typechecker that the light, if it is currently yellow, must become red before it can become green.
Ah, yes, I understand what you mean now, but you can actually encode that if you really want to. (If we assume Haskell, there's very limited DT programming which will probably let you state which transitions are legal. I must admit I'm not very up-to-speed on it, but see the next paragraph.)
The thing is that people usually don't want to go that far. Whether you're working in a typed scenario or not. In the unityped scenario, you can't go that far without reinventing a (run-time?) type-checker.
> That's a rule of the state machine and, if the compiler doesn't stop you from writing a function that breaks that rule then your state machine has not been modeled fully within the type system.
> I'm pretty sure this is what Hickey et al mean when they say that Haskell's type system is not powerful enough to describe a state machine.
Certainly this is true, but if you're in an entirely unityped setting... do you have any guarantees?
(To which the answer is, of course, "no".)
Foregoing all guarantees just because you can't guarantee everything seems... rash. (I believe this is the position that Hickey takes, for better or worse.)
Obviously, that's why I, personally, prefer languages with multiple types.
Note: I do think there are geniune scenarios where unityped may win out, e.g. "generic data transformation without knowing anything about the structure", but there's even "typed" answers for that, e.g. lenses. For my money -- and I'm obviously semi-joking -- ekmett >> hickey :)
Ah, yes, I understand what you mean now, but you can actually encode that if you really want to. (If we assume Haskell, there's very limited DT programming which will probably let you state which transitions are legal. I must admit I'm not very up-to-speed on it, but see the next paragraph.)
The thing is that people usually don't want to go that far. Whether you're working in a typed scenario or not. In the unityped scenario, you can't go that far without reinventing a (run-time?) type-checker.
> That's a rule of the state machine and, if the compiler doesn't stop you from writing a function that breaks that rule then your state machine has not been modeled fully within the type system.
> I'm pretty sure this is what Hickey et al mean when they say that Haskell's type system is not powerful enough to describe a state machine.
Certainly this is true, but if you're in an entirely unityped setting... do you have any guarantees?
(To which the answer is, of course, "no".)
Foregoing all guarantees just because you can't guarantee everything seems... rash. (I believe this is the position that Hickey takes, for better or worse.)
Obviously, that's why I, personally, prefer languages with multiple types.
Note: I do think there are geniune scenarios where unityped may win out, e.g. "generic data transformation without knowing anything about the structure", but there's even "typed" answers for that, e.g. lenses. For my money -- and I'm obviously semi-joking -- ekmett >> hickey :)
> Certainly this is true, but if you're in an entirely unityped setting... do you have any guarantees?
This is what clojure.spec is all about. The answer is not, of course, "no". They are "reinventing a run-time type checker" and prefer it that way because it allows increased expressive power at the cost of delayed verification.
> Ah, yes, I understand what you mean now, but you can actually encode that if you really want to. (If we assume Haskell, there's very limited DT programming which will probably let you state which transitions are legal. I must admit I'm not very up-to-speed on it, but see the next paragraph.)
So you don't actually know how to do it, but assume that you can. This a little strange. Are you sure that "people usually don't want to go that far"? Because the real answer is probably "it's too hard to do".
By the way, you probably want linear types in order to encode state machines in the type system in a natural way, which Haskell does not have.
This is what clojure.spec is all about. The answer is not, of course, "no". They are "reinventing a run-time type checker" and prefer it that way because it allows increased expressive power at the cost of delayed verification.
> Ah, yes, I understand what you mean now, but you can actually encode that if you really want to. (If we assume Haskell, there's very limited DT programming which will probably let you state which transitions are legal. I must admit I'm not very up-to-speed on it, but see the next paragraph.)
So you don't actually know how to do it, but assume that you can. This a little strange. Are you sure that "people usually don't want to go that far"? Because the real answer is probably "it's too hard to do".
By the way, you probably want linear types in order to encode state machines in the type system in a natural way, which Haskell does not have.
> reinventing a run-time type checker
It's not a runtime type checker, but a spec (not a type[1]!) that tools can use to check in different ways.
------------
[1]: A spec is a proposition; a type is a judgment (that corresponds to a proposition). E.g., you can have a type: `data A = 1 | 2 | 3`, and `data B = 3 | 4 | 5`, but then testing `a < b` (assuming `a:A b:B`) may not be straightforward (depending on the language), while a spec of `A = {1, 2, 3}, B = {1, 2, 3}, a ∈ A, b ∈ B` does let you test `a < b`.
It's not a runtime type checker, but a spec (not a type[1]!) that tools can use to check in different ways.
------------
[1]: A spec is a proposition; a type is a judgment (that corresponds to a proposition). E.g., you can have a type: `data A = 1 | 2 | 3`, and `data B = 3 | 4 | 5`, but then testing `a < b` (assuming `a:A b:B`) may not be straightforward (depending on the language), while a spec of `A = {1, 2, 3}, B = {1, 2, 3}, a ∈ A, b ∈ B` does let you test `a < b`.
> So you don't actually know how to do it, but assume that you can. This a little strange.
Also a little strange to follow this ticking off with a false claim "you probably want linear types in order to encode state machines". State machines certainly do not require linear types.
Also a little strange to follow this ticking off with a false claim "you probably want linear types in order to encode state machines". State machines certainly do not require linear types.
My claim is not false: state machines do not "require" linear types, but that is not what I said. I said linear types are what you want if you want to encode these things in a natural way.
Substructural (linear) types are ALL ABOUT encoding stateful propositions in the type system. Even if you had an expressive dependent type system, the best way to type check a stateful protocol would be to add substructural types to it!
See http://users.eecs.northwestern.edu/~jesse/pubs/dissertation/... for a good discussion of encoding stateful propositions with substructural (linear) type systems.
Substructural (linear) types are ALL ABOUT encoding stateful propositions in the type system. Even if you had an expressive dependent type system, the best way to type check a stateful protocol would be to add substructural types to it!
See http://users.eecs.northwestern.edu/~jesse/pubs/dissertation/... for a good discussion of encoding stateful propositions with substructural (linear) type systems.
> increased expressive power at the cost of delayed verification.
I don't see where the increased expressive power comes from. Dynamic checks are equally doable in a typed and an untyped setting.
But let's say it gives you more expressive power. It does so at the cost of no verification. Validation is a very poor substitute.
> Because the real answer is probably "it's too hard to do".
It's common usage of GADTs and DataKinds nowadays. Granted, strictly speaking not Haskell, but GHC Haskell.
> By the way, you probably want linear types in order to encode state machines in the type system in a natural way
You only need linear types if you need to enforce that (some) values will have a single logical future.
I don't see where the increased expressive power comes from. Dynamic checks are equally doable in a typed and an untyped setting.
But let's say it gives you more expressive power. It does so at the cost of no verification. Validation is a very poor substitute.
> Because the real answer is probably "it's too hard to do".
It's common usage of GADTs and DataKinds nowadays. Granted, strictly speaking not Haskell, but GHC Haskell.
> By the way, you probably want linear types in order to encode state machines in the type system in a natural way
You only need linear types if you need to enforce that (some) values will have a single logical future.
> Dynamic checks are equally doable in a typed and an untyped setting.
Yes, but specs are not "dynamic checks" but rather "static metadata". How they are checked depends on the tooling. It's true that a typed language could support specs, but those would either be an additional orthogonal mechanism (like Java and JML/C# and Spec#, or somewhat differently, Isabelle/HOL) or encoded in dependent types, which require the tooling to be built into the compiler, which would support either manual/interactive proofs (as in Agda/Coq respectively) or automatic inference (as in Liquid Haskell).
> It's common usage of GADTs and DataKinds nowadays. Granted, strictly speaking not Haskell, but GHC Haskell.
That's not the same level of expressivity, and certainly not the same level of effort.
Yes, but specs are not "dynamic checks" but rather "static metadata". How they are checked depends on the tooling. It's true that a typed language could support specs, but those would either be an additional orthogonal mechanism (like Java and JML/C# and Spec#, or somewhat differently, Isabelle/HOL) or encoded in dependent types, which require the tooling to be built into the compiler, which would support either manual/interactive proofs (as in Agda/Coq respectively) or automatic inference (as in Liquid Haskell).
> It's common usage of GADTs and DataKinds nowadays. Granted, strictly speaking not Haskell, but GHC Haskell.
That's not the same level of expressivity, and certainly not the same level of effort.
> They are "reinventing a run-time type checker" and prefer it that way because it allows increased expressive power at the cost of delayed verification.
Well, to me "delayed verification" is worth nothing. There is an almost unbounded infinity of program states. I want proof before I run the program.
(That's, like, the point, man! I'm not going to send up a space shuttle and just count on "oh, well, our error recovery is great and we saw no errors during testing anyway". To quote a great comedian: "Are you having a laugh?")
> So you don't actually know how to do it, but assume that you can.
Oh, come on, be a little more charitable. I don't know, but people in the DT community know and will tell you... at length how to do it.
> By the way, you probably want linear types in order to encode state machines in the type system in a natural way, which Haskell does not have.
Indeed it does not... but I must say I don't miss them that much for all practical purposes. If what you want to do is encoding state machines and prove things about said machines, then you probably want an even more specialized language. I'm OK with that.
I'm not OK with saying: "Can't be done in language X, therefore all effort at static verification is futile."
So there.
Well, to me "delayed verification" is worth nothing. There is an almost unbounded infinity of program states. I want proof before I run the program.
(That's, like, the point, man! I'm not going to send up a space shuttle and just count on "oh, well, our error recovery is great and we saw no errors during testing anyway". To quote a great comedian: "Are you having a laugh?")
> So you don't actually know how to do it, but assume that you can.
Oh, come on, be a little more charitable. I don't know, but people in the DT community know and will tell you... at length how to do it.
> By the way, you probably want linear types in order to encode state machines in the type system in a natural way, which Haskell does not have.
Indeed it does not... but I must say I don't miss them that much for all practical purposes. If what you want to do is encoding state machines and prove things about said machines, then you probably want an even more specialized language. I'm OK with that.
I'm not OK with saying: "Can't be done in language X, therefore all effort at static verification is futile."
So there.
> I want proof before I run the program.
Except that for most interesting propositions you can't have proof. You need to show that the crude propositions that your type system can prove are what makes all the difference. I'm not so sure that's the case.
Except that for most interesting propositions you can't have proof. You need to show that the crude propositions that your type system can prove are what makes all the difference. I'm not so sure that's the case.
> So you don't actually know how to do it, but assume that you can. This a little strange.
Not all that strange. I trust the members of the Algorithms Research community when they tell me it's possible to test a number for primality in polynomial time. I have no idea how to do it.
Not all that strange. I trust the members of the Algorithms Research community when they tell me it's possible to test a number for primality in polynomial time. I have no idea how to do it.
AKS is the name of that algorithm, by the way, after the initials of the authors. The paper is called "PRIMES in P".
https://en.wikipedia.org/wiki/AKS_primality_test
https://en.wikipedia.org/wiki/AKS_primality_test
See e.g. https://hackage.haskell.org/package/foldl-transduce
(That's just a random package from a 5 second Google search -- I think there are several at this point. Some of them even generalize transducers, I believe.)
EDIT: Thanks for the link, btw.
(That's just a random package from a 5 second Google search -- I think there are several at this point. Some of them even generalize transducers, I believe.)
EDIT: Thanks for the link, btw.
To me, his comments on transducers & types sounded more like: I don't care to write a type signature for this, and good luck to you if you do.
In one of his posts he actually provided Haskell-like type signatures (and stated why they weren't sufficient), so that's my basis for concluding that he thought about it in terms of types.
(Perphaps my memory is fuzzy, but I'm 99% sure on this point.)
(Perphaps my memory is fuzzy, but I'm 99% sure on this point.)
He did provide them but didn't do a very good job, tbh. Types capture the behavior you want to express in your program and Rich ultimately concluded that types could no achieve this for transducers. On the other hand, many libraries since then have generally shown that types can and do... It's just that Rich's types didn't.
> He did provide them but didn't do a very good job, tbh.
Well, yeah, that was what I pointed out in my original post :).
I think we agree.
Well, yeah, that was what I pointed out in my original post :).
I think we agree.
I highly doubt he's trolling. In his own words from a post where he asked people to refrain from language wars: "Haskell is a tremendously ambitious and inspiring language, for which I have the highest respect. There is zero point in bashing it (or its users) in this forum."
Clojure very deliberately adopts the stance that type systems like Haskell's are valuable but heavyweight, and chose development speed and concision over types.
Clojure very deliberately adopts the stance that type systems like Haskell's are valuable but heavyweight, and chose development speed and concision over types.
Could someone explain what the paragraphs under "Map specs should be of keysets only" are getting at?
I'm trying to map it onto my experience with Clojure and plumatic/schema, but nothing is clicking.
I'm trying to map it onto my experience with Clojure and plumatic/schema, but nothing is clicking.
If you have a function that accepts a map of a certain shape and want to write a specification for that map, you should only be specifying the required/optional keys in that spec. You should not be forced to also specify the values for those keys. When you are (as in Schema) then you wind up with redundancies when you have a slightly different map structure that also includes one or more (but not all) of the same keys with the same semantic meaning.
Generally in most of the existing libs like Schema you specify that a "person entity" is a map that has particular attributes that have particular properties and a "company entity" is a map that has other attributes (some of which might overlap).
In spec, you specify that a first-name attribute is a string and an email attribute matches some regex. Those are specifications about the attributes and can be used across many entity types. A person entity spec is then defined as a map that contains a set of required or optional attributes. A company entity spec is a different map with different attributes, some of which might be shared with a person (like email address).
This is in many ways a shift in perspective, allowing you to start defining and reusing attribute specs to a much greater degree.
The use of structural systems with mathematical properties (regular expressions for sequence structure and sets for map keys) turns out to have many useful effects in computing error sets, etc.
In spec, you specify that a first-name attribute is a string and an email attribute matches some regex. Those are specifications about the attributes and can be used across many entity types. A person entity spec is then defined as a map that contains a set of required or optional attributes. A company entity spec is a different map with different attributes, some of which might be shared with a person (like email address).
This is in many ways a shift in perspective, allowing you to start defining and reusing attribute specs to a much greater degree.
The use of structural systems with mathematical properties (regular expressions for sequence structure and sets for map keys) turns out to have many useful effects in computing error sets, etc.
The division -- between predicates for attribute values vs. specs for maps/seqs -- is interesting because AFAICT Datomic schemas ~= the former, while leaving the latter to your app? (Although I guess :db.cardinality/many straddles that line.)
thanks for this excellent explanation; this provides just about all the justification I need. looking forward to toying around with this.
Here's my understanding...
In Schema, you would represent a map's spec like this:
In Schema, you would represent a map's spec like this:
(def my-map {:a s/Str :b s/Int})
(s/def ::a string?)
(s/def ::b integer?)
(s/def ::my-map (s/keys :req [::a ::b]))
I think that (spec/keys ...) specifies the possible keys of a map, i.e., required and optional keys. The specification of what values these keys take are external to the specification of which keys go into a map. So somewhere else you will have to specify the valid value(s) for some (namespaced) key.
And that's the problem. With Schema you can compound key specs and values. This mashes the code self documenting. With Specs you have to point to another spec. Ask the way down the complex tree.
Those regular expressions for sequences seem related to "Efficient dynamic type checking of heterogeneous sequences", or maybe some other prior work I don't know about.
See https://github.com/jimka2001/regular-type-expression and http://www.lrde.epita.fr/dload/papers/newton.16.rte.report.p...
See https://github.com/jimka2001/regular-type-expression and http://www.lrde.epita.fr/dload/papers/newton.16.rte.report.p...
Incidentally, the source is here: https://github.com/clojure/clojure/blob/master/src/clj/cloju...
I enjoy using racket and contracts make it quite enjoyable to use - error messages in particular are much improved by their presence. Looking forward to this!
If you are new to using a snapshot of clojure (like me), you can find some helpful information here: http://dev.clojure.org/display/community/Maven+Settings+and+...
tl;dr
tl;dr
In a Leiningen project.clj file:
:repositories {"sonatype-oss-public" "https://oss.sonatype.org/content/groups/public/"}
Also, in that lein project file, reference clojure as
[org.clojure/clojure "1.9.0-master-SNAPSHOT"] (1.9.0 being the dev head right now)This won't work with Cider btw. I had to change it to 1.9.0-SNAPSHOT in pom.xml before a mvn install.
error in process filter: version-to-list: Invalid version syntax: '1.9.0-master-SNAPSHOT'
error in process filter: Invalid version syntax: '1.9.0-master-SNAPSHOT'Interesting. It works for me, but I'm using a dev snapshot of cider too.
Having -master- in there is pretty non standard, but I was poking in oss.sonatype.org and that's what the jar looked like in there. Maybe I goofed. There is a little bit of magic that happens with maven and snapshots (magic to me anyway).
Having -master- in there is pretty non standard, but I was poking in oss.sonatype.org and that's what the jar looked like in there. Maybe I goofed. There is a little bit of magic that happens with maven and snapshots (magic to me anyway).
Excellent. This definitely looks to be an improvement over schema, which I use but don't particularly like. Though, I'm fairly new to Clojure (about a month or so), so it may just be my inexperience :)
> Defining specifications of every subset/union/intersection, and then redundantly stating the semantic of each key is both an antipattern and unworkable in the most dynamic cases.
This is the reason I haven't jumped on the Schema bandwagon. Surprised to see I'm not the only one who had reservations about it.
> Finally, in all languages, dynamic or not, tests are essential to quality. Too many critical properties are not captured by common type systems.
Glad to see some acknowledgement of the need for at least some tests from Rich, who previously critized TDD, saying that you don't keep a car on the road by banging into the guardrails, and advocated H(ammock)DD.
> But manual testing has a very low effectiveness/effort ratio. Property-based, generative testing, as implemented for Clojure in test.check, has proved to be far more powerful than manually written tests.
(Assuming by manual testing he means manually writing automated tests,) I can't help but wonder if he came to that conclusion by writing the wrong tests and, naturally, finding them unhelpful.
> Enable and start a dialog about semantic change and compatibility
Okay, slightly rant-y tangent, but out of all the fluff phrases and buzzwords I've ever heard, "start a dialog" is probably the worst offender, and I wish it was wiped from the English language.
> I hope you find spec useful and powerful.
Looks useful so far. But where's the Github repo? Or is it coming in the next version of Clojure? Or, like, how/when can we use it?
This is the reason I haven't jumped on the Schema bandwagon. Surprised to see I'm not the only one who had reservations about it.
> Finally, in all languages, dynamic or not, tests are essential to quality. Too many critical properties are not captured by common type systems.
Glad to see some acknowledgement of the need for at least some tests from Rich, who previously critized TDD, saying that you don't keep a car on the road by banging into the guardrails, and advocated H(ammock)DD.
> But manual testing has a very low effectiveness/effort ratio. Property-based, generative testing, as implemented for Clojure in test.check, has proved to be far more powerful than manually written tests.
(Assuming by manual testing he means manually writing automated tests,) I can't help but wonder if he came to that conclusion by writing the wrong tests and, naturally, finding them unhelpful.
> Enable and start a dialog about semantic change and compatibility
Okay, slightly rant-y tangent, but out of all the fluff phrases and buzzwords I've ever heard, "start a dialog" is probably the worst offender, and I wish it was wiped from the English language.
> I hope you find spec useful and powerful.
Looks useful so far. But where's the Github repo? Or is it coming in the next version of Clojure? Or, like, how/when can we use it?
Bits will be coming into the Clojure repo today and we'll have an alpha release in the next couple days along with a lot more info.
>of all the fluff phrases
You're forgetting "it is worth noting", "it is important to note," "it should be noted that," "I should note," and their many derivatives which serve no purpose and are nothing but filler. [0]
[0] https://en.wikipedia.org/wiki/Wikipedia:It_should_be_noted
You're forgetting "it is worth noting", "it is important to note," "it should be noted that," "I should note," and their many derivatives which serve no purpose and are nothing but filler. [0]
[0] https://en.wikipedia.org/wiki/Wikipedia:It_should_be_noted
Rich was never against testing, he was against TEST DRIVEN development, and thats what TDD stands for.
>The basic idea is that specs are nothing more than a logical composition of predicates.
This reminds me of this technique:
https://github.com/astoeckley/Eat-Static#going-further
This reminds me of this technique:
https://github.com/astoeckley/Eat-Static#going-further
I'm confused what a function spec would look like using fdef (etc). Can someone write an example?
If in Haskell we have a type signature
f :: Type1 -> Type2
...what will a clojure function spec roughly look like?
If in Haskell we have a type signature
f :: Type1 -> Type2
...what will a clojure function spec roughly look like?
Here's another example showing how you can use the :fn spec to relate the :args and :ret values in arbitrary ways.
(defn title-case
"Capitalizes the first letter of each word"
[s]
(str/replace s #"(\w+)" (comp str/capitalize second)))
(title-case "moving pictures") ;; => "Moving Pictures"
(spec/fdef title-case
:args (spec/cat :s string?)
:ret string?
:fn #(= (-> % :args :s str/capitalize) (-> % :ret str/capitalize))) (spec/fdef clojure.core/range
:args (spec/alt
:infinite (spec/cat)
:range-0 (spec/cat :end number?)
:range (spec/cat :start number? :end number?)
:step (spec/cat :start number? :end number? :step number?))
:ret #(instance? clojure.lang.Seqable %))```Interesting. I don't know what to think. It's a lot to read. But in some ways adds more information than types (you can make reference to values). Maybe you could make it more concise with more definitions? For example:
(def UnboundedInterval (spec/cat))
(def PositiveUnboundedInterval (spec/cat :end number?))
(def ClosedInterval (spec/cat :start number? :end number?))
(def StepClosedInterval (spec/cat :start number? :end number? :step number?))
(def SeqableInstance #(instance? clojure.lang.Seqable %))
(spec/fdef clojure.core/range
:args (spec/alt
:infinite UnboundedInterval
:range-0 PositiveUnboundedInterval
:range ClosedInterval
:step StepClosedInterval
:ret SeqableInstance)
> (you can make reference to values)
Dependent types let you do just that as well. While theoretical computer scientists for the most part focus on dependently typed systems capable of expressing and proving any mathematical proposition (which obviously results in increased complexity), there exist more pragmatic variants of dependent types aimed at solving the everyday needs of programmers, with a minimum of annotation burden, see: http://hackage.haskell.org/package/liquidhaskell
Dependent types let you do just that as well. While theoretical computer scientists for the most part focus on dependently typed systems capable of expressing and proving any mathematical proposition (which obviously results in increased complexity), there exist more pragmatic variants of dependent types aimed at solving the everyday needs of programmers, with a minimum of annotation burden, see: http://hackage.haskell.org/package/liquidhaskell
Looks sane, specially because you can (s/and ...) the specs so you should have reusable specs in some common namespace.
clojure.spec is available in 1.9.1/alpha1 via [org.clojure/clojure "1.9.0-alpha1"] in your project.clj.
There is an excellent guide available here: http://clojure.org/guides/spec
There is an excellent guide available here: http://clojure.org/guides/spec
Will there be performance hits to using this?
This site could really benefit from a custom stylesheet for printing. Having to delete lots of nodes with devtools just to get a printable doc.
Noted. Won't get to it anytime soon though.
https://github.com/clojure/clojure-site/issues/99
[deleted]
I'm waiting for the day somebody with write a Clojure transpiler for a non-Lispy syntax! Although I love Lisp (I used to use it 20+ years ago daily), but the year is 2016!
For those who find all of those parenthesis off-putting: http://clochure.org/
I suspect that if you think parentheses are the problem, they are not the problem.
Yeah, I actually love the parentheses!
How could you implement macros in any sane way without s-exps? You need an AST and the hoops involved if you didn't have something like lisp would be terrible. Imagine running a macro over python or C code.
Elixir has an interesting way of approaching the problem. http://elixir-lang.org/getting-started/meta/macros.html
But this introduces additional ast format. Something in between simplicity of the syntax and complexity of the real ast. In lisp like languages you are working with syntactic forms from the language. Seems like extra overhead without any gain.
I would personally rather optimize for the convenience of reading the language than writing the macro, but I understand the tradeoff.
(And though I came to S-exprs early in my learning, algol-style still is easier for me to parse and read.)
(And though I came to S-exprs early in my learning, algol-style still is easier for me to parse and read.)
And this is precisely the point of divergence, as "readable" understandably means different things to different people.
Personally I find Clojure very readable, but of course I don't mean that in some kind of objective sense of the word.
Personally I find Clojure very readable, but of course I don't mean that in some kind of objective sense of the word.
Coming from a Clojure background, "extra overhead without any gain" is more or less the definition of Elixir.
Isn't the Erlang VM and ecosystem sort of the selling point of Elixir? Not that I'm particularly well versed in the specific strengths and weaknesses of that platform.
Yes, but you can have that with Erlang as well and I think this is exactly what grandparent was alluding to.
Runtime overhead possibly. Developer productivity is likely increased. Hard to measure that though.
What syntax do you feel would be more expressive and as versatile, while not losing macros?
Well, Rust has macros [0], too!
[0]: https://doc.rust-lang.org/book/macros.html
[0]: https://doc.rust-lang.org/book/macros.html
Rust has tons of problems with macros and they are still not fully in the language (last I checked). Rust macros actually do go back to Lisp. Specifically the Dylan language, a Lisp once envisioned as the equivalent of modern Objective C by Apple. Their are a number of papers from that period that you can read.
Macro systems are not all the same, they don't all have the same power. Clojure uses a very powerful form that is not as safe. Their is a lot of modern research in Racket.
A extremely powerful macro in a language with syntax was specified but never implemented by David Moon, you can find it here: http://users.rcn.com/david-moon/PLOT/
Macro systems are not all the same, they don't all have the same power. Clojure uses a very powerful form that is not as safe. Their is a lot of modern research in Racket.
A extremely powerful macro in a language with syntax was specified but never implemented by David Moon, you can find it here: http://users.rcn.com/david-moon/PLOT/
"Macros" can be an overloaded term in Rust. Specifically, "macros" mean "macro_rules!", which is very much in the language. Compiler extensions, which are sometimes called "procedural macros", are still unstable.
Not sure what the year has to do with anything, but most things in computer science were invented in the sixties and the seventies or even earlier and since then we've only been refining them. In terms of programming languages, the theoretical breakthroughs have been in terms of static typing, even though note that even the Hindley-Milner type system has been first described in 1969.
In other words, strange thing to complain about a syntax that has stood the test of time. After all, most mainstream languages (C, C++, Simula, Pascal, Java, etc) are Algol-based, a language that first appeared in 1958. And no, you can't have LISP without its syntax, because that's what's giving it power. The "code is data" mantra only works because Lisp is homoiconic, because it basically has no meaningful syntax other than syntax for describing its primitive data-structures, which makes its macros first class and its "eval" natural.
But in the LISP family Clojure is actually quite modern. It runs on both the JVM and Javascript engines. It does have extra syntax for describing sets and maps. Its data-structures are immutable and it encourages immutability to a far larger degree than other LISPs. But it also has very good interoperability with the underlying platform.
In other words, strange thing to complain about a syntax that has stood the test of time. After all, most mainstream languages (C, C++, Simula, Pascal, Java, etc) are Algol-based, a language that first appeared in 1958. And no, you can't have LISP without its syntax, because that's what's giving it power. The "code is data" mantra only works because Lisp is homoiconic, because it basically has no meaningful syntax other than syntax for describing its primitive data-structures, which makes its macros first class and its "eval" natural.
But in the LISP family Clojure is actually quite modern. It runs on both the JVM and Javascript engines. It does have extra syntax for describing sets and maps. Its data-structures are immutable and it encourages immutability to a far larger degree than other LISPs. But it also has very good interoperability with the underlying platform.
> And no, you can't have LISP without its syntax, because that's what's giving it power. The "code is data" mantra only works because Lisp is homoiconic
No. Homoiconicity is nice for "code is data" and macros, bot not necessary. Being able to reify and generate ASTs is sufficient. See for example Dylan, Terra/Lua, Scala LMS, MetaML, or Julia's or Rust's macros.
No. Homoiconicity is nice for "code is data" and macros, bot not necessary. Being able to reify and generate ASTs is sufficient. See for example Dylan, Terra/Lua, Scala LMS, MetaML, or Julia's or Rust's macros.
Note sure what you're arguing against. I stand by my claim.
I'm actively working with Scala's macros and played with Scala LMS. In fact working with those convinced me of how much all LISP alternatives suck for expressing macros.
Do you know why? Because in these languages macros are not first class, hence (1) they tend to be an afterthought, (2) hard to use and (3) filled with bugs because, wouldn't you know, they are targeting "library authors" and not users.
Scala's macros for example are exposing the compiler's internals. And boy, I can tell you stories. Like how it wants an "untypecheck" call on the AST if you modify it, because those ASTs happen to be mutable (the horror), but then "untypecheck" has a bug in it, crashing the compiler if you have an implicit "unapply" in a pattern match, so you have to rewrite your AST first to get rid of implicit "unapply" calls to work around it. Behold Sincron, an otherwise simplistic library, for which it took me a whole month to achieve inlining Function0 literal arguments, with help from others and copy/pasted code, between cries that I crashed the compiler, again and again: https://sincron.org - and btw, this rewrite for getting rid of unapply, this was the still-dangerous shortcut I took, because the general consensus at this point is that if you want to workaround the bugs of "untypecheck", you need to fix that AST manually.
You can't blame Scala too much really. This is definitely preferable to hacking your own compiler plugin or to forking the compiler. And eventually beautiful solutions can emerge from that.
The alternative to something like this is to expose a limited form of macros. For example F# quotations, or Rust's macros, since you mentioned those. The problem with these macro systems is that they are very limited, only applicable to a narrow set of use-cases. Sorry, but I personally think Rust's macros are next to useless as they needlessly complicate the language without that big of a gain. And the irony is that macros are basically used for error-handling in Rust. You know, if they added higher-kinded types, there wouldn't have been such strong demand for those macros in the first place.
LISP is the only language (family) exposing macros to users.
I'm actively working with Scala's macros and played with Scala LMS. In fact working with those convinced me of how much all LISP alternatives suck for expressing macros.
Do you know why? Because in these languages macros are not first class, hence (1) they tend to be an afterthought, (2) hard to use and (3) filled with bugs because, wouldn't you know, they are targeting "library authors" and not users.
Scala's macros for example are exposing the compiler's internals. And boy, I can tell you stories. Like how it wants an "untypecheck" call on the AST if you modify it, because those ASTs happen to be mutable (the horror), but then "untypecheck" has a bug in it, crashing the compiler if you have an implicit "unapply" in a pattern match, so you have to rewrite your AST first to get rid of implicit "unapply" calls to work around it. Behold Sincron, an otherwise simplistic library, for which it took me a whole month to achieve inlining Function0 literal arguments, with help from others and copy/pasted code, between cries that I crashed the compiler, again and again: https://sincron.org - and btw, this rewrite for getting rid of unapply, this was the still-dangerous shortcut I took, because the general consensus at this point is that if you want to workaround the bugs of "untypecheck", you need to fix that AST manually.
You can't blame Scala too much really. This is definitely preferable to hacking your own compiler plugin or to forking the compiler. And eventually beautiful solutions can emerge from that.
The alternative to something like this is to expose a limited form of macros. For example F# quotations, or Rust's macros, since you mentioned those. The problem with these macro systems is that they are very limited, only applicable to a narrow set of use-cases. Sorry, but I personally think Rust's macros are next to useless as they needlessly complicate the language without that big of a gain. And the irony is that macros are basically used for error-handling in Rust. You know, if they added higher-kinded types, there wouldn't have been such strong demand for those macros in the first place.
LISP is the only language (family) exposing macros to users.
The standard library contains many macros that are not about error handling: http://doc.rust-lang.org/stable/std/#macros Only try! is, specifically. vec!, write!, panic!, and println! are also very popular.
It's just a research language (as far as I know, at least), but Honu[1] has support for syntax-case style macros.
[1] https://www.cs.utah.edu/~rafkind/papers/honu-2012.pdf
[1] https://www.cs.utah.edu/~rafkind/papers/honu-2012.pdf
Don't know about the others, but Julia is homoiconic.
I wish Julia the best after 7 years of having to use Matlab, but despite its metaprogramming facilities, it's not actually homoiconic. It appears enough people pointed this out that they finally stopped using the word on their website.
"Like Lisp, Julia represents its own code as a data structure of the language itself. Since code is represented by objects that can be created and manipulated from within the language, it is possible for a program to transform and generate its own code. This allows sophisticated code generation without extra build steps, and also allows true Lisp-style macros operating at the level of abstract syntax trees." (http://docs.julialang.org/en/stable/manual/metaprogramming/#...)
Doesn't this qualify? I may certainly be missing something in my understanding of what "homoiconic" means.
Doesn't this qualify? I may certainly be missing something in my understanding of what "homoiconic" means.
Homoiconic means that the code structure is similar to the actual AST. In Lisps, this is easy to see, because a nested list of symbols is pretty much the definition of an AST. In truth, real homoiconicity is hard to achieve in any language that doesn't represent code as trees/S-expressions.
In Julia, they built a data structure that can represent code, and can be accessed via reflection, but it's no more homoiconic than most languages (note that "homoiconic" is nowhere on the page you linked to). In Lisps, the code is just nested lists, like other list data, and can be manipulated using the all of the standard list-processing functions. Whereas, if you look at the examples in Julia, you'll see that using macros vs functions involves special syntax all over the place.
A good page for this is http://c2.com/cgi/wiki?HomoiconicLanguages
In Julia, they built a data structure that can represent code, and can be accessed via reflection, but it's no more homoiconic than most languages (note that "homoiconic" is nowhere on the page you linked to). In Lisps, the code is just nested lists, like other list data, and can be manipulated using the all of the standard list-processing functions. Whereas, if you look at the examples in Julia, you'll see that using macros vs functions involves special syntax all over the place.
A good page for this is http://c2.com/cgi/wiki?HomoiconicLanguages
Usually I avoid splitting hairs, but since the terminology bikeshed is all about that, let me make a little correction.
In julia, there is a data structure called `Expr`, which is what's used internally by the compiler to represent code and is exposed to julia. It's not a list, it's a tree, but the power of being able to access it like any other tree still remains. Of course julia's surface syntax is significantly more complicated than lisp, so macros are more complicated.
I see - you're exactly right. Thanks for the clear explanation.
By the way, it's now "Lisp", not "LISP".
I love the original Lisp, because it was simple Syntax was so suitable for AI and treating code as data, but Clojure is not as simple syntax as Lisp! So, it's both complicated and unusual to the nowaday developer!
I love the original Lisp, because it was simple Syntax was so suitable for AI and treating code as data, but Clojure is not as simple syntax as Lisp! So, it's both complicated and unusual to the nowaday developer!
[deleted]
Usually people do the opposite — write transpiler from lispy syntax to some language's syntax. For example: lfe, hy, to some extent clojurescript.
The clojure community as a whole embraced prismatic's schema as a way to provide occurence typing for data. Since library authors often tend to reduce their dependency surface, most libraries do not ship with it though.
With the inclusion of this in clojure core, it will now be possible to provide occurence typing at the edge of functions instead of tracking malformed input deep inside apps.
The racket-type contract notation is also a very welcome change from other similar approaches in my opinion.
[1]: https://github.com/plumatic/schema