The Prolog Story (2010)(kylecordes.com)
kylecordes.com
The Prolog Story (2010)
http://kylecordes.com/2010/the-prolog-story
11 comments
Have you been feeling dire expressing your code while having only (Subject, Predicate, Object) first order logic? What kind of requests your app could manage and how large was DB (how many triplets)?
Summary: They needed to do some complex queries on data in a SQL database, queries that were clearly better suited for Prolog. So they 1) queried for the relevant data from SQL, 2) formatted the data into the form of Prolog terms and dumped it into a file, 3) fired up the Prolog interpreter and loaded the data along with a small amount of Prolog query code, 4) ran the Prolog queries, 5) formatted the results into CSV and dumped it into a file, and finally 6) imported the results back into SQL. While it has some hacky elements, this approach took just one day to implement, as opposed to the projected months it would have taken to do it without Prolog.
If I understand it right, Prolog is being used kind of like a more powerful version of Awk / Perl. The Power of Prolog [1] describes a somewhat similar use of Prolog: storing server logs as Prolog terms and then running complex queries directly on the log files.
[1] https://www.metalevel.at/prolog/business
If I understand it right, Prolog is being used kind of like a more powerful version of Awk / Perl. The Power of Prolog [1] describes a somewhat similar use of Prolog: storing server logs as Prolog terms and then running complex queries directly on the log files.
[1] https://www.metalevel.at/prolog/business
> Prolog is being used kind of like a more powerful version of Awk / Perl
I have limited experience with Prolog, but it is not just a quick and dirty scripting language (how I view Perl). They were able to build the system so fast, because they only needed to translate the customer's requirements into Prolog declarations, and the Prolog engine is powerful enough to solve the constraints. In my experience, imperative code rarely has a 1-to-1 relationship with the requirements like this.
I have limited experience with Prolog, but it is not just a quick and dirty scripting language (how I view Perl). They were able to build the system so fast, because they only needed to translate the customer's requirements into Prolog declarations, and the Prolog engine is powerful enough to solve the constraints. In my experience, imperative code rarely has a 1-to-1 relationship with the requirements like this.
Maybe it would be clearer if I said Prolog was being used like a "much" more powerful version of Awk / Perl. Consider the server log case I mentioned. A question you might ask of a server log is: Are there any client requests that were not served? That query is an easy one-liner. You can go a little further and say: Give me the IPs of all the client requests that were not served. That query is also easy, maybe just another line or two. It would be a lot harder to get that information with Awk etc, but the spirit is the same: you have a bunch of data, and you want to extract some particular information, and you want to do so without a lot of pomp.
I recently used prolog for a complicated binary reverse-engineering task: bit packing. Likewise prolog is perfect for compilation.
https://savannah.gnu.org/forum/forum.php?forum_id=9203
In fact it's a better prolog, picat, which also allows pretty straightforward statements, like loops, and has solver support.
http://picat-lang.org/
https://savannah.gnu.org/forum/forum.php?forum_id=9203
In fact it's a better prolog, picat, which also allows pretty straightforward statements, like loops, and has solver support.
http://picat-lang.org/
A big point of using Prolog for the commercial projects fitting its niche (1) is that it's based on an ISO standard, though, and implementations are (theoretically) interchangeable. To make Prolog programs actually portable across major Prolog implementations, there's been the Prolog Commons initiative for an extended standard predicate lib.
1) personally I think Prolog is much more broadly applicable in all kinds of apps rather than just type inference, policy languages, games; but with the recent rise of Datalog as actual query and inference language (and not just in academic database literature), it seems Prolog's time has finally come
[2]: http://www.prolog-commons.org/
1) personally I think Prolog is much more broadly applicable in all kinds of apps rather than just type inference, policy languages, games; but with the recent rise of Datalog as actual query and inference language (and not just in academic database literature), it seems Prolog's time has finally come
[2]: http://www.prolog-commons.org/
Sources: https://github.com/LibreDWG/libredwg/blob/master/examples/AC... for some generated sample data
and https://github.com/LibreDWG/libredwg/blob/master/examples/un... for the logic.
What types of problems does one use constraint programming for? Large optimization problems require something like CPLEX usually.
Constraint logic programming is generally proposed as a more natural alternative for arithmetic, in Prolog, than Prolog's native arithmetic functions.
It's a biig discussion and I'm not the one to make the argument for CLP, but, for example, in straight-up Prolog arithmetic is implemented using the is/2 predicate, which looks a bit like this:
You can find more information in the CLP(FD) library documentation, here:
http://www.swi-prolog.org/man/clpfd.html
Or, if Markus Triska (the author of the library) is around, he can probably elucidate its use a bit better than I can :)
It's a biig discussion and I'm not the one to make the argument for CLP, but, for example, in straight-up Prolog arithmetic is implemented using the is/2 predicate, which looks a bit like this:
A is 1 + 2.
X #= 1+2.
?- 3 #= Y+2.
Y = 1.
?- X #= A + B, A in 1..3, B in 4..6.
X in 5..9,
A+B#=X,
A in 1..3,
B in 4..6.
You can find more information in the CLP(FD) library documentation, here:
http://www.swi-prolog.org/man/clpfd.html
Or, if Markus Triska (the author of the library) is around, he can probably elucidate its use a bit better than I can :)
[deleted]
Wow, thanks for sharing!
I agree with the other commenter’s Wow! Picat also supports satisfiability apps like MiniZinc does. Thanks for posting that link!
Initially I choose MiniZinc, but then was turned off by its baroque syntax. Straight prolog syntax and picat syntax was much easier in the end.
writing prolog is mostly about writing in the most natural and readable syntax possible. using haskell-like types put me off.
writing prolog is mostly about writing in the most natural and readable syntax possible. using haskell-like types put me off.
I would like to know what Prolog implementation the author used. In particular, the article claims:
1. > An initial analysis found that we would need to implement a complex depth/breadth search algorithm either in the client application or in SQL.
2. The Prolog runtime would efficiently solve this problem given rules that naively described it.
I am skeptical, as this is emphatically not my experience with Prolog. In my experience, for any problem requiring a complex search strategy, Prolog is not your friend. Most Prologs do something approximating a naive depth-first search with unification, and on complex search problems this approach rapidly blows up. This can _sometimes_ be fixed by:
1. Making careful use of cut (`!`) or other non-declarative operators to change the search order.
2. Using tabling, essentially an advanced form of memoization. But only some Prologs support tabling, and it's useful only for some sorts of problem.
However, the author mentions none of these. Are they glossing over something, or is my (probably more limited) Prolog experience uncharacteristic? Are there "smart" Prologs out there I'm unaware of?
1. > An initial analysis found that we would need to implement a complex depth/breadth search algorithm either in the client application or in SQL.
2. The Prolog runtime would efficiently solve this problem given rules that naively described it.
I am skeptical, as this is emphatically not my experience with Prolog. In my experience, for any problem requiring a complex search strategy, Prolog is not your friend. Most Prologs do something approximating a naive depth-first search with unification, and on complex search problems this approach rapidly blows up. This can _sometimes_ be fixed by:
1. Making careful use of cut (`!`) or other non-declarative operators to change the search order.
2. Using tabling, essentially an advanced form of memoization. But only some Prologs support tabling, and it's useful only for some sorts of problem.
However, the author mentions none of these. Are they glossing over something, or is my (probably more limited) Prolog experience uncharacteristic? Are there "smart" Prologs out there I'm unaware of?
On rereading, it seems like what may be happening is simply that the problem isn't very large (10-100,000 rows) and doesn't need to be solved very fast (a few minutes), and so the standard Prolog search strategy may have worked just fine. A good example of avoiding unnecessary over-engineering.
Without knowing the details of the problem it's hard to be sure, of course. Combinatorial explosion can make even searches involving a rather small number of choices take impractically long to finish. But that depends on the structure of the search problem.
Edit: The author says in the comments on the original post that he used SWI Prolog, which I believe doesn't do anything special in its search strategy, although it does have support for CLP.
Without knowing the details of the problem it's hard to be sure, of course. Combinatorial explosion can make even searches involving a rather small number of choices take impractically long to finish. But that depends on the structure of the search problem.
Edit: The author says in the comments on the original post that he used SWI Prolog, which I believe doesn't do anything special in its search strategy, although it does have support for CLP.
Part of searching is knowing how to reduce your search space. This is not just a prolog problem but a CS problem. Knowing your data, using cut, swiprolog might not have tabling support but it's easy to memorize with assertz.
I've used prolog on production systems, there's odbc drivers, he could have connected directly to the DB and ran his rules. Checkout the swiprolog libs, they are quite extensive
I've used prolog on production systems, there's odbc drivers, he could have connected directly to the DB and ran his rules. Checkout the swiprolog libs, they are quite extensive
>> Most Prologs do something approximating a naive depth-first search with unification, and on complex search problems this approach rapidly blows up.
Most modern Prologs use indexing on predicates' symbols and arguments to speed up queries and the days when Prolog was a "slow" language are long gone by.
As to the search strategy, it's a bog-standard, depth-first search (no approximation). Unification is part of the theorem-proving algorith, SLD-resolution. Depth-first search is used to find "resolvents", i.e. candidates for resolution (ish).
The parent post describes a backward-chaining problem, where rules must be selected depending on their arguments, most likelly to perform some complex reasoning procedure. In that context, search is required to select relevant rules at each step of the reasoning process- not, say, to search a large database for all records of persons with a name starting from "m".
For this kind of use-case, Prolog's search is not only perfectly adequate, but also nearly perfectly optimised, due to long years of development as a language.
That said, "searching a database for all records starting from m" is very much like searching for resolvents and thanks to modern practices, like indexing, Prolog can do that kind of search just fine also.
Most modern Prologs use indexing on predicates' symbols and arguments to speed up queries and the days when Prolog was a "slow" language are long gone by.
As to the search strategy, it's a bog-standard, depth-first search (no approximation). Unification is part of the theorem-proving algorith, SLD-resolution. Depth-first search is used to find "resolvents", i.e. candidates for resolution (ish).
The parent post describes a backward-chaining problem, where rules must be selected depending on their arguments, most likelly to perform some complex reasoning procedure. In that context, search is required to select relevant rules at each step of the reasoning process- not, say, to search a large database for all records of persons with a name starting from "m".
For this kind of use-case, Prolog's search is not only perfectly adequate, but also nearly perfectly optimised, due to long years of development as a language.
That said, "searching a database for all records starting from m" is very much like searching for resolvents and thanks to modern practices, like indexing, Prolog can do that kind of search just fine also.
(I am the author.)
The implementation I used was SWI-Prolog. I had tinkered with a couple of others prior to that in school (sorry, I don't remember more details), but this was free/open, and up to the job at hand without requiring an acquisition process from the customer of the system.
About the search strategy; yes I had to use a few cuts to get acceptable efficiency. Still, this was much easier than re-creating the same thing in SQL or imperative application code. The particular details, I don't remember well; sadly I've had little occasion since 2010 to use Prolog.
The implementation I used was SWI-Prolog. I had tinkered with a couple of others prior to that in school (sorry, I don't remember more details), but this was free/open, and up to the job at hand without requiring an acquisition process from the customer of the system.
About the search strategy; yes I had to use a few cuts to get acceptable efficiency. Still, this was much easier than re-creating the same thing in SQL or imperative application code. The particular details, I don't remember well; sadly I've had little occasion since 2010 to use Prolog.
Hi Kyle, Peter Sherman here. (I posted this HN article). You and I had an email dialogue around 2012, and of course I remember you from your Delphi days (BDE Alternatives Guide, etc.) and the old Joel On Software forum. Anyway, the other day I was reading about CoQ and it's use for formal verification, which turns out to solve problems in various different diverse domains, and I said to myself, "Hey, this sounds a whole heck of a lot like Prolog, plus a type system", and so I was attempting to compare and contrast the two systems when I thought back to your great video detailing the customer problem you had and how you used Prolog to solve it. So, that's why the link appears on HN now, that's why the lag time -- to answer your other question. Hey, if you've got a spare minute or two, you should shoot me an email, peter.d.sherman AT gmail DOT com. I might have an interesting business proposition for you...
Prolog looks like the exact right language for at least some part of Netflix's OPA[0] -- I wonder why they didn't use it or why it wasn't a good fit (if someone considered it).
I often want to reach for prolog when I face a problem like this, but I just don't know enough about how it degrades/breaks and of course don't want to use it to do any of the rest of the program stuff (web server, DB access), etc.
[0]: http://www.openpolicyagent.org/
I often want to reach for prolog when I face a problem like this, but I just don't know enough about how it degrades/breaks and of course don't want to use it to do any of the rest of the program stuff (web server, DB access), etc.
[0]: http://www.openpolicyagent.org/
(OPA co-founder here.)
The semantics of OPA's policy language are based on Datalog, a non-Turing complete subset of Prolog. This means that all policy queries in OPA are guaranteed to terminate (which makes it a good fit for problems like authorization.)
Beyond regular Datalog, OPA adds first-class support for querying complex/nested data structures like JSON.
As a side note, OPA was not developed at Netflix, but they were one of the early adopters and continue to use it today.
The semantics of OPA's policy language are based on Datalog, a non-Turing complete subset of Prolog. This means that all policy queries in OPA are guaranteed to terminate (which makes it a good fit for problems like authorization.)
Beyond regular Datalog, OPA adds first-class support for querying complex/nested data structures like JSON.
As a side note, OPA was not developed at Netflix, but they were one of the early adopters and continue to use it today.
Thanks so much for the answer -- I thoroughly enjoyed the OPA talks I've seen[0][1]. I apologize for mistaking OPA as a netflix product, I think one of the first times I saw it was as it was being used by Netflix so I assumed it was one of their F/OSS projects or built by someone there.
Did you guys build your own engine? I took a quick look at the repo but don't see anything that looks like a datalog library in your glide package list.
Last but not least, thanks for making and open sourcing such an awesome tool! Will definitely be passing the word on about Styra[2], I had no idea there was a whole company/more efforts behind OPA. I plan on using OPA in a bunch of upcoming projects -- it looks like a fantastic, stable addition to the toolbox of people looking to build robust programs/services.
[0]: https://www.youtube.com/watch?v=XEHeexPpgrA
[1]: https://www.youtube.com/watch?v=4mBJSIhs2xQ
[2]: https://www.styra.com/
Did you guys build your own engine? I took a quick look at the repo but don't see anything that looks like a datalog library in your glide package list.
Last but not least, thanks for making and open sourcing such an awesome tool! Will definitely be passing the word on about Styra[2], I had no idea there was a whole company/more efforts behind OPA. I plan on using OPA in a bunch of upcoming projects -- it looks like a fantastic, stable addition to the toolbox of people looking to build robust programs/services.
[0]: https://www.youtube.com/watch?v=XEHeexPpgrA
[1]: https://www.youtube.com/watch?v=4mBJSIhs2xQ
[2]: https://www.styra.com/
> Did you guys build your own engine? I took a quick look at the repo but don't see anything that looks like a datalog library in your glide package list.
Yes, the language implementation (parser/compiler/evaluator) is implemented from scratch.
> Last but not least, thanks for making and open sourcing such an awesome tool!
Thanks for the kind words! If you have questions or need help, feel free to file issues on Github or ask questions on Slack.
Yes, the language implementation (parser/compiler/evaluator) is implemented from scratch.
> Last but not least, thanks for making and open sourcing such an awesome tool!
Thanks for the kind words! If you have questions or need help, feel free to file issues on Github or ask questions on Slack.
Previous discussion from 2010: https://news.ycombinator.com/item?id=1363680 (60 comments)
There is a modern book about Prolog - Power of Prolog [1].
[1] https://www.metalevel.at/prolog
[1] https://www.metalevel.at/prolog
Love seeing a good Prolog story surface. I was blown away by how productive it was to use back in university along with XPCE for UI. Became the first language I expertised in and it has paid off in terms of opening up a lot of other avenues like being able to easily learn Erlang and Elixir.
There isn't always a good reason to use it for many situations nowadays but it the way you deal with graphs and trees and such is fantastic. Though fun fact Allegrograph has a Prolog rules interface
There isn't always a good reason to use it for many situations nowadays but it the way you deal with graphs and trees and such is fantastic. Though fun fact Allegrograph has a Prolog rules interface
Hey, this is me!
This is some serious latency between recording and writing a thing, and is appearing on hacker news. I will go through here and try to answer various questions etc.
This is some serious latency between recording and writing a thing, and is appearing on hacker news. I will go through here and try to answer various questions etc.
I recently went through what I'm calling a "conversion experience" with Prolog. I was writing a compiler and a link to Warren's 1980 paper "Logic Programming and Compiler Writing" went by here on HN ( https://news.ycombinator.com/item?id=17674859 .) After a brief learning curve I now have a much more powerful compiler in about 1/5th of the code.
There are a few problems that don't fit well with Prolog, but not many. For everything else, if you're not using Prolog you're probably working too hard.
Consider that achieving feature parity with 1/5th the code means 1/5th the bugs, right off the top.
But often Prolog code is more useful than some equivalent imperative code, for example, a Sudoku relation defined in Prolog serves to solve puzzles, but it can also generate new puzzles and verify partial puzzles (count the solutions and assert that there's only one.) https://swish.swi-prolog.org/p/Boring%20Sudoku.swinb
Prolog is also old.
I keep thinking, "What about FOO?", only to find that FOO has been explored years ago by a group of researchers, and often there is working code off-the-shelf and ready to go to solve whatever problem.
Anyhow, TL;DR: For goodness' sake please check out Prolog. It's like time-traveling into the future.
There are a few problems that don't fit well with Prolog, but not many. For everything else, if you're not using Prolog you're probably working too hard.
Consider that achieving feature parity with 1/5th the code means 1/5th the bugs, right off the top.
But often Prolog code is more useful than some equivalent imperative code, for example, a Sudoku relation defined in Prolog serves to solve puzzles, but it can also generate new puzzles and verify partial puzzles (count the solutions and assert that there's only one.) https://swish.swi-prolog.org/p/Boring%20Sudoku.swinb
Prolog is also old.
I keep thinking, "What about FOO?", only to find that FOO has been explored years ago by a group of researchers, and often there is working code off-the-shelf and ready to go to solve whatever problem.
Anyhow, TL;DR: For goodness' sake please check out Prolog. It's like time-traveling into the future.
> I used Prolog in a production system a few years ago,
Nice pun!
Nice pun!
I have also used SwiProlog and occasionally Amzi Prolog also since then. SwiProlog’s semantic Web library was the basis for all my early work and experience with semantic Web and linked data. I definitely suggest trying Prolog on a few small projects and add it to your tool set.