Stripe is building a Ruby typechecker(medium.com)

351 pointsby thibaut_barrere8 anni fa183 comments

https://medium.com/byteconf/stripe-is-building-a-ruby-typechecker-d6cd7cee6abf

186 comments

Kalium8 anni fa

"Why don't they just use (insert static-typed system here) instead?" someone will inevitably ask.

"Because converting a large, complex set of living codebases from one language to another is a non-trivial task that requires boiling any number of oceans. This approach doesn't have that problem." comes the wiser, if less interesting, response.

sanderjd8 anni fa

This seems like boiling oceans as well. It's hard to guess which oceans are bigger or harder to boil. Clearly Stripe thinks this is the easier way to go. That wouldn't have been my instinct. Maybe they made a really good estimate for how it will be though. I'd be very curious to see their full analysis!

emilsedgh8 anni fa

This might be as hard, but it's not as risky as rewriting the whole thing.

This approach, at worst, will just fail.

Rewriting the whole thing can have real big consequences if not done carefully and takes a real long time to bear fruit.

Kalium8 anni fa

You're absolutely right! Building probably was a lot like boiling an ocean. Grafting static typing onto a very dynamic language is a massive and complex undertaking.

That said, it's one item that can be built greenfield. In some senses, that makes it easier to apply to other codebases once the tool is in a working state. It also means one thing to work on, rather than N, which is much easier to plan for.

You're completely right that this is an ocean to boil, and it's rarely easy to guess the size of any given one. Yet it's possible that the naive intuition - when boiling oceans, prefer fewer - might be reliable in this case.

vidarh8 anni fa

Grafting a complete and safe static typing system onto a very dynamic language is a massive and complex one.

But writing an advisory type checker (that just warns) with a specific code base as a starting point is a very different thing. Biggest question to me is if they support duck-typing. Secondly, if I was to do type checking for Ruby I'd start with parsing yard-style annotations, as a lot of libraries etc. come with that as part of generating their documentation (and if not, writing them means you're now writing comments a lot of tools can generate documentation from.

Kalium8 anni fa

You're right. An advisory system isn't incredible difficult or intimidatingly large and complex, and doing it for a small set of codebases is much easier than solving it as a general problem.

But uh, I didn't want to tell the commenter that they weren't thinking it through and were being utterly betrayed by their instincts. It's hard to get people to listen to that kind of thing. So I opted for telling them that they're right, implying that they have the opportunity to become more right, and reminding them of how right they are.

rurban8 anni fa

It's only massive if you are doing it wrong. All examples I found so far were doing it wrong. For cperl (perl5 with classes and types) I did it right instead. It took about half a year, with most parts improving the rest of the system and signatures. Ruby already has signatures, so thats out if the way. Python eg did it completely wrong, PHP is doing it right.

The goal is catch type violations with optionally declared types and constant literals, and help the compiler and optimizer to specialize ops on types, and do array and loop bounds checks, essentially moving runtime checks to compile-time checks. A good type checker should make overall code faster and safer, what you see out there are rudimentary and not scalable efforts to make it safer, but not faster.

It certainly looks like ocean boiling if you look at all the horribly bad efforts out there, comparable to pypy, unladen swallow, perl6, ... But if you look at proper type checkers and optimizers it's trivial, really.

pmontra8 anni fa

I'm one of those people. I'm not sure I'd keep loving Ruby as much as I do now if the majority of Ruby developers end up using the type checker and I'll also have to. I like Ruby in part because it's strongly typed but I don't have to lose time writing types. That was very welcome coming from C and Java in 2006. I prefer to lose time when I pass a type that I shouldn't have passed. It doesn't happen often (not every year) vs at every single method definition.

To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?

klmr8 anni fa

> I like Ruby in part because it's strongly typed but I don't have to lose time writing types.

That’s why type inference exists. Static typing doesn’t imply annotating everything manually with types. Granted, most static type systems require (or at least encourage) this but it’s not necessary everywhere (or indeed for most usage). Annotating public interfaces with types is, I’d argue strongly, already best practice everywhere (in languages that don’t support it in the language, it’s done in documentation).

> To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?

Or rather: How popular are TypeScript or Flow? Very.

nailer8 anni fa

I'm not a typing person, similar to pmontra. Any programming language where print("Hello world") is longer than print("Hello world") bothers me.

And most of my JavaScript is untyped, because (and apparently this is common, someone did some stats on it a while ago) typing errors aren't a great source of errors.

However I love optional typing. Being able to build a thing quickly, and then add those guarantees if/when you feel they're needed is great. My current codebase is about 80 modules. Nearly all are untyped JS. The database code is TypeScript because that's where I feel type handling would add the greatest benefit - not that I have had typing errors, just that I like the additional peace of mind.

klmr8 anni fa

> typing errors aren't a great source of errors.

That’s not true. On the contrary, typing accounts for a substantial fraction of all errors (and refactoring could transform even more logic errors into catchable type errors). The best research (best methodology, sample size, by far) on the subject [1] puts the lower bound for the fraction of type errors in JavaScript at 15%.

[1] https://paperpile.com/app/p/3070170e-933e-0956-82ec-879fb924...

nailer8 anni fa

Not doing a URL hunt as I'm at work, but as mentioned in the comment you're replying to (literally after you chopped it off), there was a large study showing the opposite on HN a while back.

klmr8 anni fa

> there was a large study showing the opposite on HN a while back.

Yes, sorry: I didn’t mean to ignore that part of your answer but the research I’ve linked to essentially supersedes everything else in that area. Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors. I’m assuming you’re referring to [1] and while that was an impressive study in its own right, their classification of bug sources is indirect and, ultimately, simply didn’t allow any assertions regarding the prevalence of type errors.

[1] https://paperpile.com/app/p/54ee3fdd-a6ef-0767-add5-55b60419...

didibus8 anni fa

You'll have to justify this: "Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors."

Your recent study is pay walled, so I can only go by the abstract, but it doesn't sound any more rigorous to me.

I also find its comparing very different things. It says bugs in release, but it sounds like they consider commits to be releases, which would be very unfortunate.

Its also specifically targeting JavaScript, which prior study already showed to be one of the poorest performers in terms of defects (i.e., it tends to have more). For example, it would be interesting to know how many bugs were due to weak typing.

I'm also curious, adding the typed annotations should be a blind activity to be rigorous. Was it? Or did they knew there was a bug and what the bug was then they added the type info?

I also feel they're doing a one way comparison, having to add types can introduce other forms of bugs, but since this study is targeted, I find it to suffer a little from confirmation bias. Like type checker detects type errors, is a bit of a no brainer conclusion. I understand they mean this in the sense that 15% of bugs were type errors. But that doesn't mean TypeScript code would have 15% less bugs, because they did it one way only. It could introduce other bugs in the process.

Would have been interesting too to analyse how long it took for the bug fix commit to be made over the introduction of the bug. So you would know the real cost of them. Which is important, since annotating adds dev time, but so does big fixing. Would be nice to have time data.

Finally, the prior study demonstrate the same outcome. That static tends to have lower defect rate then dynamic and functional less then imperative. The difference is in the outliers. Clojure and Ruby were both outliers that outdid almost all static languages.

So actually, it would be great to reapply your study to them. Clojure has an optional type checker already so it could be a good one. It was also the strongest outlier.

klmr8 anni fa

> You'll have to justify this: "Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors."

That topic has filled whole blog posts. But in a nutshell, no previous study was able to compare directly the effect of typing disciplines due to confounders that they couldn’t control for in their experimental setup. For instance, virtually all previous studies compared different programming languages, which obviously differ by more than just their typing discipline. Many studies had very small sample sizes due to using human test subjects to write sample programs. These test subjects were almost exclusively students without real-world experience — or, in some case, any prior experience in the respective programming languages. Thus, many studies — from the outset — tested beginner-friendliness of languages rather than everyday use. Fair enough, but not the same as type checking benefits. Furthermore, most of these studies, for the same reason, were restricted to testing on artificial, academic, small toy programs rather than real-world applications.

The few studies that looked at big real-world data sets (the biggest and most rigorous being linked above) didn’t even look at typing discipline as an individual factor — again because they couldn’t regress it out as a single factor.

> Your recent study is pay walled

The link I posted has the full text PDF (that’s the whole reason for me to share Paperpile links rather than the original or DOI). See also an academic discussion [1] of the manuscript.

> Its also specifically targeting JavaScript

Which is explained in the paper: no other constellation allows to compare the effect of added static type checks as directly as JavaScript/TypeScript/Flow. That’s because (a) these languages only differ in their type checker, and are gradually typed, i.e. types can be added to just a subset of the program, which enabled the study methodology. And (b) there’s an extensive database of real-world code to analyse.

> For example, it would be interesting to know how many bugs were due to weak typing.

Fair enough. I’d expect the effect to be (much) less pronounced in a language with stronger type guarantees. Then again, the 15% number is absolutely an underestimate to begin with (see the paper and the blog post [1]).

> adding the typed annotations should be a blind activity to be rigorous

That’s not at all obvious, and from experience I disagree. Thinking about types is automatic neither in static nor in dynamic languages. Once you’ve figured out the correct type, yes, it’s a blind activity … but that’s a pretty empty statement.

> having to add types can introduce other forms of bugs

Again, this claim is far from obvious, beyond the trivial “if I make this type an `int` even though it should be a `string` then that’s a bug”. Fair enough, but adding the type annotation and performing a type check merely reveals this bug. The bug itself was present in the programmer’s flawed assumptions about the invariants in the code. Your assertion is equivalent to saying “compiling or interpreting the code [rather than writing it on a piece of paper and never touching it] can introduce bugs”. — The bugs are already in the code, we just didn’t detect them.

> Would have been interesting too to analyse how long it took for the bug fix commit

They only considered bugs they could fix within a highly constrained time frame, and only looking at local code around the bug. Hence, again, why the 15% is an extreme underestimate. Furthermore,

> So you would know the real cost of them.

The study calculated the token cost of adding types (and provide a justification for this metric). In a nutshell, the cost is negligible, especially in languages with strong type inference capabilities (Flow outperforms TypeScript here).

> Clojure and Ruby were both outliers that outdid almost all static languages.

Right, and I’d expect the same to still hold. But it shows one of the crucial shortcomings of previous studies: they primarily did not examine the effect of static vs dynamic typing. Rather, they examined the effect of different programming languages, to which typing is just one contributor amongst many. It should come as no surprise to experienced programmers that some languages (regardless of static vs dynamic discipline) vastly outperform others. For me, this is all the more reason to be excited about the headline topic: If done well, statically type checked Ruby could be an amazing language, with the added benefit of static typing at (as shown) virtually no cost.

[1] https://blog.acolyer.org/2017/09/19/to-type-or-not-to-type-q...

didibus8 anni fa

> If done well, statically type checked Ruby could be an amazing language, with the added benefit of static typing at (as shown) virtually no cost.

I think this is where I feel the jump in causation to be too high. No one knows why Ruby and Clojure match up to Scala and Haskell. The prior study couldn't isolate the impact of a particular feature, but it did a pretty good job, at least similar to your study in finding overall defect rates.

Ruby with types, if we take your conclusion, should end up even outperforming Haskell by almost 10% in overall defect rates. That just doesn't sound believable to me.

I fear types are not independent variables. They affect other variables. And that's where I argue that they might in fact come at some cost which we still do not understand.

I still strongly feel they should have added types without knowing the bug. There's many ways to type data, those choices do matter. There's code that can't be typed precisely, or requires considerable efforts to type, there's even code that can not be typed at all, and must be rewritten differently to be typed.

Now I'm not going to argue JavaScript doesn't benefit immensely from static type checking. What I argue is that doesn't mean every language will see benefits from static types.

That said, I'm a big fan of gradient typing, especially when it can add runtime contracts over boundaries between typed and untyped. But, I'm not sure how successful they would be in Ruby. They've failed in Clojure, while no real analyses was performed, the community rolled back most adoption of it due to their impression of it not providing any value while adding extra effort. Obviously, the Clojure gradient typing system isn't as well maintained as Flow and TypeScript, so maybe that played a role.

nailer8 anni fa

No problem! Thanks for clarifying.

fouc8 anni fa

Ruby has the advantage of 'duck typing', due to its strong OOP foundations. Not quite sure what Python has that reduces the demand for static type declaration.

I think part of the popularity of TypeScript could be down to a large proportion of javascript coders that have a background/preference for static typing. It could also be that certain classes of bugs are more prone to happen in JavaScript or it's harder to debug when bugs do happen?

oelmekki8 anni fa

Ruby has been my first professional language and I've sticked with it for a decade, but I never really understood duck typing before switching to Go, with its interfaces (a type implements an interface if it implements its methods, so we can use interface as function parameter type).

I don't get why we talk of duck typing about ruby : if I pass as parameter an object that does not quack like a duck, nothing will prevent me to do it and it will generate an exception when quacking. Did I miss something obvious?

vidarh8 anni fa

The obvious part is that Ruby does not have a distinction between "compile time" and "runtime". As such, if you want to catch those cases, you need tests.

Duck-typing can be boiled down to: The type of a parameter to a function is defined by the methods that will be called on it, not by its class.

E.g. if you have:

    def foo source
      dosomething(source.shift)
    end

Then absent additional restrictions inferred from requirements of dosomething(), the class of "source" is irrelevant. The relevant type information is whether or not "source" responds to "shift".

A proper Ruby-ish type checker will need to be able to handle that, or it'll push people to write non-idiomatic Ruby.

ninkendo8 anni fa

I wonder if this would be a good opportunity to use a typing system that's more like Go's, where you don't need to explicitly declare that a class implements an interface, but instead any class that has the right methods will implement the right interfaces automatically.

(In your example the method could be:

    def foo(source: Shiftable<?>)
      dosomething(source.shift)
    end

where `Shiftable` is an interface that's something like

    iface Shiftable<T>
      .shift(): T
    end
)

oelmekki8 anni fa

I've digged a bit on the subject (better late than never), it seems that duck typing is seen in ruby as a guideline, rather than a language feature. Basically, it's about recommending to use `#respond_to?` rather than `#is_a?`, `#kind_of?` or `Class#===` in end user code.

vidarh8 anni fa

It's a language feature in as much as the absence of static typing and the ability to check for methods is what makes it possible.

But yes, it it a guideline, and Ruby code that violates it will generally be seen as not being idiomatic Ruby.

oelmekki8 anni fa

Oh, I see, given the interpreter is written in strongly typed language, allowing to go without static typing is a feature by itself, and allowing to test for methods presence allows to do duck typing, thus it allows to say that ruby implements duck typing.

It makes more sense than talking of duck typing explicit usage by end users, given it's quite rare we assert proper received parameters extensively.

EDIT : which makes me realize we have a machine language without typing (binary), on top of which we use a language implementing strong typing (C), used to build a language without strong typing (ruby), on top of which Stripe is building strong typing. Making a definitive decision is hard :)

mdpye8 anni fa

Nitpick: C is weakly typed, Ruby is strong.

C is statically typed, Ruby is dynamic.

Strong typing is about objects having a definite type. In Ruby, you cannot cast a String to a Fixnum, an exception will be raised. But in C you may cast anything to anything really, and your program will believe you, whether it could possibly make sense or not...

oelmekki8 anni fa

I thought "strong typing" and "static typing" were interchangeable. I stand corrected, thanks.

emodendroket8 anni fa

> A proper Ruby-ish type checker will need to be able to handle that, or it'll push people to write non-idiomatic Ruby.

It probably won't be idiomatic (maybe just add interfaces?) but I think duck-typing is overrated anyway. Just because an object responds to a method doesn't mean it's going to do anything like what you expect. To take a contrived example, Array.Shift and Keyboard.Shift are probably unrelated methods.

vidarh8 anni fa

You don't have that guarantee with interfaces either - you're trusting that the developer isn't lying to you. My experience is that I've more often run into developers that have artificially barred me from doing what I wanted by checking for a given class, than that I've accidentally passed in something that satisfies a given type but does something different.

klmr8 anni fa

> You don't have that guarantee with interfaces either

True, but you can give the interfaces/contracts proper names (or namespaces). To use the example in the parent comment, `Array` and `Keyboard` wouldn’t implement the same contract even if both have a `Shift` method.

emodendroket8 anni fa

I guess now that I think about it Scala does have a strongly-typed duck-typing thing, but it's kind of silly and rarely used.

emodendroket8 anni fa

JavaScript and Python also have "duck typing."

aczerepinski8 anni fa

In my experience I lose less time specifying that an argument is a string in Go than I do adding a test to handle the possibility that nil gets passed in Ruby.

arthur_pryor8 anni fa

amen to that.

i appreciate type checking so much more now than i used to, and i find that i disliked it most when i needed it most (i.e., i blanched at it more when i was a worse programmer, even more likely to make type errors than i am now).

there are categories of checks in application code (as well as whole categories of simple unit tests) that i just wouldn't have to write with a static type checker. additionally, as other comments have pointed out, it'd make it easier to automatically refactor things.

i started coming around on it with groovy a number of years ago (after sorta hating the type checker in java/c/c++ for years before that). i'd find myself writing a bunch of untyped code, but then i'd also find myself going back and specifying types as i wrote tests, because it was often easier to just specify the type than to write a test for the possibility of the wrong type coming in.

as someone who mostly writes ruby code these days, i'd love to see an optional type checker get added.

vorg8 anni fa

You switched from Java to Apache Groovy to lose the static type checking, then later stuck types in your Groovy code to make it act like Java. Sounds like Groovy's value-add to the ecosystem was providing the syntax for doing that without writing `Object` everywhere.

golergka8 anni fa

> I don't have to lose time writing types

In C#, I write type names when I declare a type, when I create instance of a type, when I declare method argument or return type (which you would probably do even in a dynamic language as a comment - you don't want people to have to go through your code to figure it out themselves, right?) and when I declare a collection of that type. Out of all of these, only the last one seems as "wasting time", because if I declare a collection and instantly use it on the next line, the reader can probably infer what type I meant to use - but in all other cases, not writing types would make the reader's job harder, not easier.

Don't the same principles apply when you're working in a dynamic language?

gazarsgo8 anni fa

am i mistaken, or do generics specifically eliminate that last case...?

golergka8 anni fa

Generics allow such a case - without generics I wouldn't declare what type collections are because they would be just collections of objects.

emodendroket8 anni fa

It's not just about preventing errors; it's also about making maintenance work much easier.

edwinyzh8 anni fa

So it might (or not) be the best option for them at this point, but on the other hand, is it another case that shows a static-typed language is a better option for developing complex systems? IMHO, it is.

Kalium8 anni fa

I think this shows that static typing becomes more desirable and a better investment as size and complexity of a codebase - and company - grow. This might not be the same as evidence that starting with static typing is preferable.

gmaster14408 anni fa

To your earlier point, larger systems are harder to migrate. This suggests that starting off with an investment in static type systems is more tenable as a long-term strategy.

Kalium8 anni fa

All other things being equal, that's probably true!

toasterlovin8 anni fa

If only all other things were equal...

Kalium8 anni fa

That's the catch, yes. All other things are very rarely equal. There are almost always other factors worth paying attention to when forming a long-term strategy. A handful that spring to mind are availability of talent, software runtime performance, schedule impact of static vs dynamic typing, and expected runway of the business.

Producing a statically typed system that runs slowly and correctly after the business has folded is not always more valuable than producing a buggy one in time to start producing revenue.

blain_the_train8 anni fa

It doesn't come even close to that. There are works that attempt to tackle this issue, if your interested I would look at those

random38 anni fa

I think making Ruby typed is a cool endeavor. It's just that it's a bit hard to justify from a business perspective..

It's not like changing to typed Ruby wouldn't affect the code bases. They'll end up having something that will resemble Ruby, but likely won't be quite Ruby. It's unlikely they could automate it. They'll have to rewrite a lot of code and typically, when you do that you also want to refactor obvious problems, so you end up rewriting big parts. And that's in addition to the cost of trying to make Ruby typed.

Or you could leave the code base as it is and extract functionality out and rewrite it in something more manageable.

Kalium8 anni fa

I think it becomes a reasonable business pitch if you look at it as a tool that significantly reduces the defect rate and improves on-time delivery.

zem8 anni fa

i work on pytype [https://github.com/google/pytype], a similar project for python, and it has definitely made business sense. it lets developers code in the language they find most productive (largely due to its dynamic nature), and adds back some of the benefits of static type checking. in particular, it has caught a lot of bugs that would otherwise only have shown up in production, and perhaps not until some rare code path was hit.

_pmf_8 anni fa

> "Because converting a large, complex set of living codebases from one language to another is a non-trivial task that requires boiling any number of oceans. This approach doesn't have that problem."

Sometimes, if a baby is really, really ugly, it's OK to throw it out with the bathwater.

Kalium8 anni fa

You're absolutely right! But just because it's OK doesn't make it pragmatic engineering or good project planning.

amelius8 anni fa

> boiling any number of oceans

How many lines of code are we talking about here?

Kalium8 anni fa

How many services do you think a big company like Stripe has? Let's be exceptionally conservative and assume fifty, with a minimum of a million LoC between them.

All of them need to keep working for the business to keep working.

amelius8 anni fa

Ok, but they also have a lot of developers.

Kalium8 anni fa

They do! And most of them are considered valuable employees because they can produce features the business wants in acceptable timeframes.

It's perhaps possible that the business and management side of the house at Stripe might not want to give that up for a while so all of those developers can re-implement every service. Given that historically something like half of all rewrite projects fail, there's some non-trivial level of risk there.

Though I understand if some think Stripe can afford it.

thibaut_barrere8 anni fa

Not yet open-sourced, but this is definitely planned.

You can try it out there though: https://sorbet.run

By the way, a number of RubyKaigi 2018 talks were dealing with type checking (and the other were about improving performance & memory usage).

bogdanu8 anni fa

I had the "pleasure" of creating and integrating a parser for a custom DSL into ACE. I've created the parser in ~3 days; it took another 3 to make it work with ACE.

If it was up to me, I'd chose monaco editor (as the guys from mozilla did it with their webassembly editor)

ptarjan8 anni fa

(author of the demo site here)

I just was looking for a quick editor for the demo, and ace seemed to work well for this. Thanks for the pointer to manaco, I'll look at that for next time.

Using an editor in the browser won't be the final product. We're planning on integrating into your editor of choice instead.

ptarjan8 anni fa

Thank you so much for all the interest! We're flattered and excited to see all the discussion about the project.

Try it out: https://sorbet.run

If you would like to get in touch with us about anything, please email us at [email protected].

In the presentation at RubyKaigi (which will be available online soon) we explicitly mentioned we'd like to chat with folks trying to scale Ruby into the millions of lines of code, or folks also working on similar typechecking projects. Of course feel free to email even if you aren't in those groups, but I wanted to get your attention if you were.

vidarh8 anni fa

One thing that springs to mind: Does it handle duck-typing? In other words, can I specify "any object that responds to methods x and y" as the expected type of a variable?

Because if treats classes as equivalent to types, then to me it encourages non-idiomatic Ruby.

inopinatus8 anni fa

We only have the published examples to go on, but unfortunately they all seem to assume that type means class.

If that’s borne out I could never use this library. It’d be the antithesis of duck typing and object messaging.

mbesto8 anni fa

So you know when people ask "why does X startup have X,000 developers? what do they do all day for a single app[0]. Here is your answer:

"Technical details: - 9 month of work by 3 people; - real thing, runs over all code of #Stripe"

https://twitter.com/darkdimius/status/1002103748875902978

Now let's look at this from a financial perspective:

So let's say average $175k salary per employee + benefits and you're looking at easily north of $600k to do this.

I'm not saying it's "right" or "wrong", but thats just what companies at this scale do because they create development cultures and subsequent financial controls that allow for indirect production software development to happen.

[0]for the record, Stripe is much more complicated than just a "single app"

draw_down8 anni fa

sankha938 anni fa

Any reason why existing typecheckers for Ruby didn't make the cut? There are projects like RDL [1] which provide similar functionality. I know for certain that people from Stripe were taking a look at it last year from the issues they had raised.

Disclosure: I am a grad student, recently started working on RDL.

[1]: https://github.com/plum-umd/rdl

ptarjan8 anni fa

Hi and thank you for working on RDL! We do love the project and spoke with Jeff Foster a few times last year.

We did use your standard library annotations for Sorbet and have many fixes to them sitting in your pull request queue: https://github.com/plum-umd/rdl/pull/68 https://github.com/plum-umd/rdl/pull/72 https://github.com/plum-umd/rdl/pull/57 . We stopped submitting more since these weren't being upstreamed.

Before we started our project, I evaluated rolling out RDL instead of building our own typechecker. Trust me, I would MUCH rather use an existing project than have to build our own, but sadly it just didn't scale to our millions of lines of code.

I'm more than happy to chat about the details of why we didn't use RDL if you'd like to email me at [email protected]. Thanks again for your great project, we're standing on the shoulders of giants.

nextos8 anni fa

You are embarking onto a very interesting, but tough project due to the extremely dynamic nature of Ruby!

Have you considered the property-based QuickCheck approach? Instead of building a type checker, you just annotate your code with properties (preconditions, postconditions and class invariants). You can generate tests to verify your code and also inject runtime checks to fail early in case of violations.

In my experience, this tends to work much better on dynamic languages and it also scales to larger codebases with heavy usage of dynamic idioms.

sankha938 anni fa

Oh, I am not sure how these went unmerged! We should do something about the PRs.

Will shoot you an email to discuss more.

hendzen8 anni fa

I'm sure they will come up with some plausible reason like the project having incompatible goals, or embracing a different design architecture than the one they wanted.

Truthfully, these 'prestige' projects are done for reasons other than actually improving the development standards. They probably didn't use RDL because it was more fun to write their own then figure out someone else's thing.

For all we know they may not have even had a real need for a new typechecker. Stripe could have a valued engineer who seemed bored so engineering management gave him or her an intellectualy interesting project to do - even if it wasn't that important to the business.

mattbessey8 anni fa

It would seem your cynicism was unfounded https://news.ycombinator.com/item?id=17226332

exolymph8 anni fa

Sure, that's possible, but we don't really have enough information to say either way.

ksec8 anni fa

I think they mentioned they will present more information in an up coming conference. And it makes this TypeChecker possibly the most battle tested solution out of all the competing ones.

klmr8 anni fa

Sorbet appears to be a language extension rather than a library, similar to TypeScript or Flow in the JavaScript ecosystem. I’m guessing that one of their goals was ease of use for type annotations. I’ve had a quick look at RDL and while I quite like its syntax it is somewhat more cumbersome than having type annotations “inline”, as it were. Furthermore, the annotations in RDL are themselves stringly typed, which comes with its own issues.

tonyhb8 anni fa

Quick plug for RDL. It's pretty great. There are a couple issues with Rails, but for standard Ruby code and modules it's a much better process to read, write, and maintain code with RDL than without

chrisseaton8 anni fa

What's the next step for RDL?

It seems like Hummingbird is much more powerful than Sorbet, but it's hard to tell without them publishing a paper yet.

sankha938 anni fa

There are a bunch of interesting stuff coming along. I don't know if you have seen the Refinement types in Ruby paper based on RDL recently published in VMCAI [1], this opens the window to verification of Ruby code.

[1]: https://arxiv.org/abs/1711.09281

chrisseaton8 anni fa

Thanks I added that to rubybib.org - please encourage your colleagues to create PRs when they publish papers so the whole community can find Ruby research more easily.

taf28 anni fa

This looks really nice... i wish it was just part of the language to be able to write ruby with type information e.g. instead of having a separate more verbose sig method that's called before each method signature...

``` class Foo extend T::Helpers

  sig(
      # Both positional and named parameters are referred to by name.
      # You must declare all parameters (and the return value below).
      foobar: Integer,
      widget: String
  )
  .returns(Symbol)
  def foo(foobar, widget: nil)
    (foobar.to_s + widget).to_sym
  end

end ```

I'd love it if we could write this as :

``` class Foo

  extend T::Helpers

  def foo(foobar:Integer, widget:String:nil):Symbol

    (foobar.to_s + widget).to_sym

  end

end

```

Not sure the best way to handle default values... e.g. widget:String:nil is kind of awkward... but still i'd be better this way then the additional method call 'sig'

ptarjan8 anni fa

(one of the authors here)

We'd love that too! We're chatting with the Ruby folks to see if anything like that is feasible. We experimented with lots of other syntaxes (comments, yarddoc, .rbi files, monkey-patching stdlib classes to be callable) and the `sig` syntax seems to be the easiest for folks to use inside Stripe.

After writing sigs for the past few months, I don't dislike it as much as I thought I would. You get the syntax highlighting and autocomplete of your editor, while `sig` and `T` are short enough to not feel like too much boilerplate.

We're very open to other suggestions if you have any for other syntax suggestions. Just email us at [email protected].

transfire8 anni fa

Tomdoc?

unfunco8 anni fa

The second version would be difficult, maybe impossible currently due to keyword arguments, since:

    def foo(foobar:Integer)
    end

is already valid, and I think the lexer might struggle with the ambiguity; but I'm not 100% certain.

Edit: I seemingly missed the last line of your comment where you state something similar.

tsycho8 anni fa

If foo:SomeType is problematic, let’s just use foo::SomeType instead. I agree with the parent that inline types are much nicer and more intuitive than he sig based approach.

faitswulff8 anni fa

I don't know, as a long-time Rubyist, I actually prefer Sorbet's fluent interface with `sig(...).return(...)` to these (albeit more typical) type annotations. Reminds me of RSpec.

spitfire8 anni fa

For some reason that reminds me of Pascals interface/implementation split...

For those not in the know Pascal units had to declare the function (procedure) prototypes before implementing them.

bogdanu8 anni fa

I know the editor is not the news, and it's used just for demo, but more info about it: currently it's an ACE editor strapped to a wasm that performs the checking. The integration looks more like a quick poc since it's not using an webworker for performing the checks nor does it use the editor's markers for errors.

exabrial8 anni fa

Is Crystal still a thing? I thought it sought to keep Ruby ease but statically for speed and to reduce bugs?

vinceguidry8 anni fa

Crystal is a completely different language with a different architecture and object model that only superficially looks like Ruby. Porting code from Ruby to Crystal is somewhat easier than, say, porting the same code to Python, but is still an act of porting. You can't use any kind of automated drop-in process to rewrite Ruby code to Crystal. They're just too different.

A good type system for Ruby is very much needed. Matz himself has stated that a type system is very much on the horizon for Ruby. Can't wait.

qop8 anni fa

With Ruby getting an AST, and potentially a type system in the future, I think that ruby 3 could lend itself to bring transpiled to crystal relatively easily, depending on exactly how much type information there is, how difficult / possible it is to correlate that with the AST.

Scala 3 is doing the typed AST thing, so hell, why not?

vidarh8 anni fa

The type system is the least of your problems. Ruby's type system is fairly simple and straight-forward.

The problem is that the ruby execution model is fundamentally incompatible with ahead-of-time compilation without making a lot of decisions about what occurs before compilation and what is deferred until later. E.g. a lot of Ruby programs execute code to determine which files to "require" (e.g. iterating over all files in a directory is a common pattern). In some cases that is a convenience for developers. In others it's meant to e.g. provide "plugin" abilities at runtime. In both cases the included code can totally change the behaviour of large parts of your code base.

Those decisions are possible to make, but you must make them and you probably can't make them satisfactorily without providing a mechanism for the developers to specify intended behaviour. If your decision is to defer everything until after compilation, you'll basically end up JIT-compiling most of the code at runtime, and all type information from before that point is suspect.

chrisseaton8 anni fa

> I think that ruby 3 could lend itself to bring transpiled to crystal relatively easily

I think for this to be tractable you'd basically have to implement a complete Ruby interpreter in Crystal and then transpire the Ruby program to bytecode to be run by the Crystal interpreter. So not really a meaningful or useful transpilation any more, and certainly not fast.

Even basic things like method dispatch do not have the same semantics in Crystal as in Ruby, so almost nothing could be tarnspiled 1-to-1.

qop8 anni fa

We don't know very much about what ruby 3 types will look like or what other information we'll have.

Even a small subset of ruby being transpiled would be a useful thing for some developers.

You're much more aware of some of the constraints here than I am, as Oracle doesn't pay me to hack ruby for a living.

With a typed AST, there'd be enough information there to build a foundation for a rb2cr tool. I didn't say the two languages are best friends and it'll be a breeze.

Most of the ruby code I want to rescue isn't littered with string-based define_method nonsense. Most of ruby I think worth saving at all is outside of rails and exists in various tools or maybe some metasploit modules or stuff like that. The pieces of code that utilize the grossest dynamic elements of ruby, like define method or objectspace, that stuff doesn't need to survive.

Even getting 50% of a codebase in ruby compiled to reasonable crystal is a much better foundation than previously purported successors (elixir, scala, swift) can do.

If ruby gets a typed ast, I'll try and write a simple transform tool for the simplest ruby.

chrisseaton8 anni fa

> Most of the ruby code I want to rescue isn't littered with string-based define_method nonsense.

I see what you are saying - but this is where the issue is I think. You may not write code that does sophisticated metaprogramming, but the gems you use are probably fundamentally based on it. Even just requiring some of the standard library uses a surprising amount of of metaprogramming. For example you can't load something as basic as 'fileutils' without doing a lot of metaprogramming.

The entire Ruby ecosystem is built on metaprogramming.

qop8 anni fa

That's where the boundary for transpilation will lie then, but I don't consider that to be a good enough reason to justify not trying to write a tool like that.

I really appreciate your thought on this! I look up to your work a lot. Sometimes I wish I had spent my career with compiler a instead of disassemblers...

Yes, there will be a lot of ruby that can never by crystallized. Yes that kinda sucks. So yes it's very good to explore optional or gradual for incremental refactors and modernization. But if even 1% of ruby code CAN be crystallized, then it's worth it to do it. Maybe not for you, but my time isn't nearly as expensive as yours must be, so I understand your reasoning here I think.

I don't mean to be adversarial here.

chrisseaton8 anni fa

Part of the reason it might be coming across as a slightly negative reaction is that I'm passionate about implementing Ruby exactly as it is. If people want to write in Crystal that's great it can be fast. But if people want to write in Ruby, doing all sorts of metaprogramming, or that's the code they actually have today and need to run in order to keep their business going, then I want to make that just as fast for them automatically. Without telling them to use a subset, or not to use awkward features. I want people to bring me their insane code and I'll find a way to make it fast for them. That's the challenge I'm enjoying at the moment.

If people are happy to use a subset of Ruby then yes it could be transpiled. But a simple subset of Ruby should work great in the new JIT compilers we're getting anyway, without using Crystal.

qop8 anni fa

I used to be happy with Ruby, but learning Scala and then Crystal really changed my mind about where the line ought to be drawn wrt to expressiveness and safety.

I don't and have never used the vast majority of Ruby's metaprogramming features, but I'm the odd duckling in an equation where probably 95% or more Ruby programmers are Rails programmers, and I am not. I mean, I can, but I don't. That factor makes less sympathetic to the majority of Ruby users who won't have any stake in a project like what I'm thinking about. And that's ok. And either way, if they want to make use of all those features then they will or maybe already have realized that Ruby already does what they want and Ruby3's care for backwards compat surely will cater to that majority in order to keep them on board. Metaprogramming I think is the primary style of Ruby, and that's OK.

The Crystal team has essentially reprogrammed the way I think about how it should feel to write good OO code. Scala did the same thing for a time, but just the sheer pain of SBT drove me away. With Crystal, Having parametric modules is an incredible advantage. I can write mixins essentially like traits that specialize on some new type in my program.

I have to consider how my methods end up typing out, and that changes the way I think about how I'm going to get from here to there. With Ruby, I really like that left-to-right idea of chaining until I arrive at the structure I want, and the more Crystal I write the more I realized how irritating it was to hunt down Ruby bugs by:

whatever.tap{ |o| puts "#{o} is a: #{o.class} here" }

or even deeper with responds_to? or whatever I'm trying to figure out. With crystal, all I have to do is look at the stacktrace, it is getting a nil where it shouldn't be at SomeClass#some_method on line 230842349, and I immediately can work on the bug at the source of it, instead of the tedious extra step of finding it.

Being able to make Ruby do magic was never part of the equation that held me a romantic captive to Ruby, it was always the succinctness. Elixir came close, but Crystal gets me all the way there.

If nothing else, maybe I will learn something new trying to find a subset of ruby3 that I can transpile.

I haven't read much about the new JIT, but if you're happy about it, maybe I should stop procrastinating and find out about it.

Kalium8 anni fa

You're absolutely right! Any Turing-complete language can be transpiled to any other.

That said, you can't always expect sensible or human-brain-friendly results if you're moving between two languages that work very different.

workinthehead8 anni fa

Why must you keep telling people "You're absolutely right!" It seems insincere and condescending at best. If you really think you know better than the person you're responding to, use evidence and logic, not cheap conversational ploys.

Kalium8 anni fa

I do need to mix up the verbiage more.

In practical terms, people respond much better to having their egos stroked than they do to being told they're wrong. Evidence and logic work much better when someone doesn't feel like their ego is at stake. Telling someone they're right before pointing out that they could be more right is one way to do this. Telling people that they're wrong is more likely to provoke a defensive reaction and a closed mind than it is genuine thoughtfulness and consideration. We've all seen people refuse logic, reason, and evidence because they feel personally attacked, I suspect.

This is a lesson I gleaned from the (in)famous "How To Make Friends And Influence People". It's not wrong, it's just cynical.

So of course it's insincere. Human interaction is greased with little insincerities. It's how we deal with egos.

vinceguidry8 anni fa

You're not supposed to acknowledge insincerity. You're supposed to make up another reason that merges the sincere and the insincere viewpoint together.

Kalium8 anni fa

Yup. But the Dale Carnegie stuff is exhausting.

I'm sincere in that I want people to listen to my points without getting their egos in the way.

workinthehead8 anni fa

The only place this works the way you think it does is in your mind. Any normal person is going to see that and register it as goofy and ingratiating. Like Spock or some alien that thinks it's cracked the human code when really it's nowhere even near the uncanny valley.

Kalium8 anni fa

I understand why you think that.

This may surprise you, but I really wish my experience agreed with you. I really wish most people saw immediately through insincerity and ego-stroking and empty complements. Indeed, some do.

Most of the time, it's disturbingly effective. My data to date suggests that most people won't look too closely when you feed them kind words to hide the others.

qop8 anni fa

Yes definitely, however -> gets better response and cooperation than -> Well, actually

Just my two cents, not parent btw

vinceguidry8 anni fa

Syntax is only the first obstacle. You also have the object model and the standard library to deal with as well.

qop8 anni fa

Crystal is progressing well, but they don't have the resources of a lab full of incredible engineers thrown at it.

It's definitely a threat though. The Amber Framework is very familiar to people who know Phoenix or Rails.

Personally, coming from Ruby, Crystal has a slightly steeper learning curve. However, I feel that magic spark deep down now that I've spent more time with it and I'm more aware of what inference can do and how to use it for my benefit.

Little safety things that seem irritating at first, but then you put the pieces together and understand them a bit more.

T | Nil is a culprit here. It seems obnoxious, but then you realize, I can just chain methods in a case statement, I don't have to check for nil over and over, just in the topmost part of whatever code it is. It's not how I normally think, and my instinct was wrong based on my initial understanding, but now I'm much more comfortable with it.

As for tooling, Crystal is rather weak. This too will come with time.

Willamin8 anni fa

If larger companies (eg. Stripe) would put more effort into improving Crystal instead of improving Ruby, Crystal would absolutely blow Ruby out of the water. Instead we get improvements to Ruby because larger companies have larger codebases and don't want to refactor everything all at once. They'd rather slowly add type annotations to their Ruby codebase until eventually it's all done. I get it, it's better from a business perspective short term. In the long term, Crystal would be able to run at least hundreds of times faster than Ruby.

pejrich8 anni fa

There's absolutely no chance that a company like Stripe would spend resources improving a language like Crystal, or even think about porting their current code base to it. If porting was what they wanted to do, Crystal would only be slightly higher than Ancient Greek on their list of potential languages. Not saying I don't like Crystal, but it makes ZERO business sense for Stripe, and they are after all a business, not a programming language improvement charity.

anthonybullard8 anni fa

Definitely still a thing, but development looks like it's slowed a bit. Maybe the Manas team is overwhelmed? 550 open issues and 130 open PRs. Nice language though, hope things improve!

everdev8 anni fa

Yes, I've enjoyed using it and there are several well written frameworks like Kemal, Amber and Lucky for apps.

And it's really useful for CLIs and APIs.

leshow8 anni fa

There's so much unanswered here, does it handle parametric polymorphism? Does it allow sum/product types? Is there inference? Can you bound generic types?

IMO it would be a complete waste of time to add a type system that only prevented the most trivial of type errors: mismatching Integer and String.

paulddraper8 anni fa

Interesting choice, considering that Stripe is a massive Scala user.

Is it really cheaper to write type checking for one of the most dynamic languages ever, than port to their other language? Seems to be strong evidence against the "stack doesn't really matter" viewpoint.

shaneos8 anni fa

Yes it is. Stripe can do this with three (badass) engineers while the hundreds of other engineers continue to build their applications in Ruby. Stripe's Ruby code base is far larger than their Scala codebase, with each being used for the types of uses that play to their strengths. Rewriting millions of lines of Ruby into another language would more or less involve pausing all application development for one to two years, which simply is not feasible.

This way, a small team of skilled engineers make this typer work, and guide its general adoption, getting the codebase well typed over the course of years, all the while Stripe continues to grow its product suite and further conquer the world of e-commerce.

Twitter did a huge Scala rewrite. There are many lessons to be taken from their lack of product velocity while doing so that the Stripe team took onboard when making this decision.

zarvox8 anni fa

It's substantially less risky, less invasive, and easier to do in parallel with other ongoing feature development than a port to another language would be. It's much easier to get incremental value for the investment if you're keeping the same language and codebase than if you're trying to replace everything all at once.

masukomi8 anni fa

ok this is going to sound snarky but it's an honest question: Why not just use Crystal[1]?

It's basically just compiled ruby with static types, and it can infer types correctly in most cases without you explicitly noting which they are.

The best argument for this (instead of Crystal) is keeping access to ruby gems.

[1]: https://crystal-lang.org/

CGamesPlay8 anni fa

My 100k LOC code base is already written in ruby though. I have to rewrite everything?

masukomi8 anni fa

yeah. that's a pretty compelling reason, but on the other hand, there's a lot of value to be had from modularizing your codebase and microservices so... you could do all future dev in a new language, and if you wanted, gradually convert your existing modules. Sadly, few places think far enough ahead to see the values of modularity (even ignoring language switching) and end up with giant monolithic codebases so language switching just isn't an option without a complete rewrite.

mmartinson8 anni fa

While you have some good points, I'm not sure monolith and modularity are mutually exclusive. Some places I'm sure think some places think far ahead, and determine that the cost of microservices isn't worth the later benefit while they're trying to bootstrap a project.

naiyt8 anni fa

For me it would have the same things that make Typescript appealing:

- Full access to a large and mature ecosystem (one that Crystal doesn't quite have)

- The ability to gradually add typing to an existing Ruby codebase. No need to rewrite an entire application. Just gradually add types to new code and during refactors.

If I was starting a new project I would definitely be considering Crystal, but I still feel like something like this would be great. Typescript has been a godsend for my frontend development life, and I would love to have something like this for my backend Ruby development.

creyes8 anni fa

I've been working with Lucky and working with a bit of Crystal - it's still not quite there yet. When we were working on a recent release of Lucky we found a kinda breaking in the compiler that the team fixed pretty quickly but it still shows just how new it is. I'm optimistic for 1.0 though

That said, Flow style gradual/optional typing in Ruby would be awesome

parvenu748 anni fa

Is Crystal a superset of Ruby where any valid Ruby project is a valid Crystal project -- just lacking type definition? If Crystal can do that then I think adoption of Crystal would benefit greatly.

chrisseaton8 anni fa

> Is Crystal a superset of Ruby where any valid Ruby project is a valid Crystal project

No. So the idea pretty much stops there.

masukomi8 anni fa

As @chrisseaton said, no, BUT some ruby is valid Crystal. I think it would be more accurate to say that a subset of Crystal is valid Ruby, and a subset of Ruby is valid Crystal.

freedomben8 anni fa

That's correct. You can run your Crystal programs with the Ruby interpreter, but you can't compile your Ruby to Crystal (simple scripts usually work, but bigger things that use use gems probably won't).

djur8 anni fa

You can't run a lot of Crystal code in Ruby, either. They implement keyword arguments completely differently (Crystal does it more like Python, where all arguments can be specified by name or position) and of course Ruby doesn't support Crystal syntax relating to types. Crystal also has a tuple type with a literal syntax ({1, 2, 3}) that isn't valid Ruby. But otherwise it's fairly close.

freedomben8 anni fa

Thank you for the correction, I was not aware of this.

jashmatthews8 anni fa

It’s not just compiled Ruby, but a completely different language with a superficially similar syntax. You can’t really port anything.

cpeterso8 anni fa

This Ruby typechecker is a similar approach to Facebook's "Hack" language, incrementally upgrading their legacy (PHP) codebase to a new statically-typed language instead of rewriting everything in a new language.

https://code.facebook.com/posts/264544830379293/hack-a-new-p...

samratjp8 anni fa

Putting this out here in case if anyone finds it useful - I like a lot of good ideas from Contracts.rb such as the type signature looking more expressive (personal taste - looks Haskell/Elm ish) and support of Maybe's. Granted it's not static but def some good ideas to steal from.

https://egonschiele.github.io/contracts.ruby/

vemv8 anni fa

Typical Rails codebases already enjoy a reasonable layer of 'typing' via ActiveRecord coercions, validations, and possibly some system that declares/documents the types/structure of your REST API.

Also, I advocate keyword arguments (especially 'required' ones), which tend to kill 80% of the use case of typing for dynlangs ("what's the type of argument x?")

A meaningful test suite would hammer the last nail in the coffin.

ninjakeyboard8 anni fa

Is the idea that you would take your existing code and add typesafety to it? A statically typed language is faster because it does type checking at compile time and does not need to do "if string, ok else throw exception", so adding a bunch of type checking to a dynamic language adds the development overhead without the benefit. I suppose there is a false economy in that statement as defects caught early save time vs late defect discovery, which in my experience, is very common due to type issues in dynamically typed languages.

I personally prefer working with statically typed languages but I've been working with elixir for the last 8 months or so and do not feel very compelled by the use of dialyze (type checker) in the dynamically typed language - you get used to the idioms and try to keep the code type-y but it seems unidiomatic to take type safety too far in a dynamically typed language when you don't get the performance benefits. You cover the things you're likely to get wrong but embrace it for the sake of development speed otherwise. For a new project the right question to ask is "why would we use this instead of a statically typed language." I suppose for an existing project the question this is trying to solve is "how can we reduce bugs now that we've made it to market."

olingern8 anni fa

I was at RubyKaigi. This presentation was one of the best delivered, so kudos to the Stripe team for all the work put into it.

A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced?

I've been a user of both Typescript and Flow (both great tools), but saw Typescript's popularity soar because of community written types.

darkdimius8 anni fa

> A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced?

Yes. This was one of goals of our talks. We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims.

The details of how this would work out might be different. Typescript and Flow change syntax and thus they cannot be included inline in arbitrary JavaScript libraries. We're intentionally compatible with Ruby syntax. Thus we would like this "repository of types for libraries" to only contain types intermittently, until types have been accepted to upstream of respective library.

olingern8 anni fa

> We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims.

I see. Do you see this being a mono-repo of types that's under a Sorbet org or something Sorbet looks for in a gem if provided? Opening issues for a project's types with Typescript has been somewhat painful since many live in one repo [1].

1 - https://github.com/DefinitelyTyped/DefinitelyTyped

seanwilson8 anni fa

Is there any discussion on why Stripe chose to write so much code in Ruby? I would have thought for financial software where subtle bugs can have a huge impact you would want to opt for a language with static checks. Or maybe only a small part of the codebase requires this?

lettergram8 anni fa

I have to say, after using Stripe for a few years - I'm amazed at how well it works. In fact, I can even create a "refer a friend" system on a rails app in some 15 lines of code[1].

I'm excited if they can bring the same simplicity to their additions to languages and frameworks as well.

[1] https://blog.projectpiglet.com/2018/03/refer-friend-using-st...

sandstrom8 anni fa

Interesting! Anyone with insight into the ruby core team know if Ruby may add typing (optional or mandatory) in the future?

lewisinc8 anni fa

Matz has talked about it as an opt-in. I think his opinion is to not force people to adopt it if they won't enjoy it or their productivity will be hampered by it.

griffinheart8 anni fa

Was at the Kaigi and on the opening keynote Matz talked a bit about it. He doesn't really support adding it since he feels like in 20/30+ years, advances in technology will make writing types "obsolete".

quelltext8 anni fa

It's a weird argument. If it's not necessary in 30 years (I doubt it because some of the type inference issues are simply undecidable, and that in much simpler languages already), then the language can just opt to ignore type annotations or merely check their consistency. Having them is great either way for the benefit of documentation.

My take is he doesn't really like (or "get") types and feels protective of his language, which is fair. I think Python made an interesting choice here, i.e. provide syntax for types without semantics. The drawback there is potential fragmentation on type system semantics.

lobo_tuerto8 anni fa

So, Sorbet is to Ruby what Dialyzer/Dialyxir is for Erlang/Elixir right?

Are they using basically the same approach?

emodendroket8 anni fa

Having done some legacy Ruby work I'll say this is sorely needed.

mslate8 anni fa

Is this analogous to mypy?

ptarjan8 anni fa

It is a similar idea, yes.

qop8 anni fa

I think the most fascinating thing that Ruby 3 could approach is an approach similar to Dart 1's optional types. That way a fast unchecked mode could be available for developing, and REPL usage, and then a separate switch could be flicked for building a release.

Seeing the Ruby devs take compatibility so seriously is such a confidence builder for me, either way. Ruby is such a joy to program in, and for certain projects, Crystal fits the "square enough to go in a square-ish hole" role, but I really miss some of the ease-of-use that Ruby has to offer.

require 'Something'

Something.new.methods.sort

I use that every single day. Crystal can't do that. Yet.

Little things like that make Ruby such a human-friendly language.

Now that we've learned our collective lesson that naive dynamic types are too slow for big projects, I am glad to see Ruby-thinkers begin to explore the type system space and search for something that will benefit all Ruby hackers.

I suspect that if we arrive at the point where Ruby has a gradual or optional type system available, that refactoring some of the old Ruby tools will be very pleasant. If or when that day comes, I know I'll devote a chunk of my own time to try and catch the tools back up, and the rest will follow if only there are hackers who want to hack badly enough.

Willamin8 anni fa

A lot of people that love Ruby and dislike Crystal seem to dislike Crystal for reasons like what you mentioned. I'm pretty sure that you can get a sorted list of methods in Crystal (not with the same syntax unfortunately) at compile time in macros. I'm not sure why you would ever want to dynamically get a sorted list of methods at runtime... unless you're adding methods at runtime.

qop8 anni fa

I use it in the repl for classes I'm not familiar with. I use Ruby for duct taping a lot.

iamleppert8 anni fa

People created dynamic languages for the exact reason of not having to define types.

Now the trend is to bolt on type checking to dynamic languages because people don’t want dynamic behavior. Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter.

If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness.

If you have a huge codebase in Ruby that is full of bugs and is unpredictable, don’t blame the language or lack of types.

There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist?

Tehnix8 anni fa

> Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter.

I don't see how seeking to add types precludes also understanding "closures, concurrency passing or functional programming", especially since these are in no way foreign concepts in statically typed languages.

> If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness.

I definitely see what you're trying to say, but as I understand it, these projects to "add types" often come on later, when the project is so big, that it is simply less time consuming to just develop a type checker, than it is to rewrite your entire codebase in a new language.

iamleppert8 anni fa

My argument is that you can always implement your own breed of type checking either via convention (with appropriate enforcement via static analysis or linting tools if so desired without having to run the code), or you can do type checking at run time -- most dynamic languages have a facility to interrogate a variable to determine its contents and you can implement any number of tests on it as you see fit to validate it.

hderms8 anni fa

In practice the difference in guarantees between a static type checking system and a runtime one are so vast that I think they're mostly incomparable. Runtime checking is like doing most of the work of modelling a program with static types but achieving a fraction of the benefit.

Kiro8 anni fa

But let's say you have a function that takes an integer and goes completely bonkers when you pass in a string. Why would you not want to prevent that?

zxcvhjkl8 anni fa

> People created dynamic languages for the exact reason of not having to define types.

Not having to define types is not the biggest merit of dynamic languages.

The biggest benefit is to be able to do things that you couldn’t do if you had a type system. You can do some crazy stuff with Ruby’s meta programming.

hderms8 anni fa

How many problems out there are really apropriately solved by runtime metaprogramming?

freedomben8 anni fa

I'm not sure how well you understand functional programming either if you think it needs dynamic typing. The king of functional programming languages (Haskell) is statically typed.

iamleppert8 anni fa

The king of functional languages is not Haskell but Lisp in my opinion, and Lisp is dynamically typed. With Common Lisp, there is no difference between runtime and compile time: it's one in the same.

adamnemecek8 anni fa

> There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist?

This is a bad question. Refactoring Rails is AFAIK a nightmare. Given enough devs it can probably be done, however it might be painful.

f00_8 anni fa

pg_bot8 anni fa

I don't understand the appeal of static typing. It always feels like I'm adding overhead without receiving any benefits. I've worked in some fairly large dynamically typed codebases and have never run into issues with type errors. Static typing does not alleviate the need to test your code, which is a more effective way of reducing bugs in your system than annotating your methods/functions.

matchbok8 anni fa

Not sure I've ever heard anyone say static typing removes the need for testing.

Types are self-documenting. Onboarding new devs in a large static codebase will be 10x easier than a dynamic one.

Coding is actually faster in statically typed projects - autocomplete works every single time and you never have to dig into another source file to see what objects/methods you have.

And, most modern type systems (Typescript, Swift, etc) don't have much overhead at all with their impressive type inference.

mping8 anni fa

It's a double edged sword - onboard some devs on "advanced" scala code and let me know how the 10x is working for ya. I also feel that sometimes types cause additional congitive load - you have to mentally model the types and what they do. OTOH I'm pretty sure most people can onboard a js project that's well written.

I think exploratory, debug-based onboarding can be easier with poorly written static lang codebases that dynamic, but then again depends on what convetions are used. I don't care much about autocomplete unless I have to type ReallyLongClassNamesThatAreSupposedToBeMeaningful; besides coding is mostly about avoiding errors, not typing faster.

In the end, whatever floats your boat but dyn langs surely have their place.

matchbok8 anni fa

I've heard the "mentally model" argument a few times, and tbh I don't quite understand it. You have to model the object regardless, either with types or tons of extra checks to make sure that object has what you need. I've seen so much javascript code do exactly that because the methods had no idea what they were going to be given. The contract of static typing eases that load, I think.

mmartinson8 anni fa

Ya for sure. Without getting a debugger into the application code, it sometimes seems completely impossible to look at a piece of dynamic code you're not already familiar with and decide if it's going to work. I've found this especially tough during code reviews for new stuff that make heavy use of abstractions, even when the abstractions are good.

mping8 anni fa

Personally I feel that my mind is registering what each type does, kinda like building a mental dictionary. Dynlangs also have types but in terms of reading code it feels you can just skim through and skip the details. I guess there is a difference in the cognitive load of reading code and writing code.

pmontra8 anni fa

Actually the only language I used autocomplete was Java, mainly because it's almost impossible to work in Java without a IDE. However sometimes I turned autocomplete off because those long menus are a pain to see. I don't use a real autocomplete in Emacs with Ruby, Python, Elixir, JavaScript. I use pabbrev which remembers the words I type. It's usually enough not to make typos and it helps to remember the methods/functions.

jrs958 anni fa

At this point I don’t think anyone sane is seriously advocating for the adoption of Scala.

hderms8 anni fa

What alternatives are there to what Scala provides? What newer language would a 'sane' person advocate?

jrs958 anni fa

Unless you really need it for a platform like Spark where there aren't many other options, pretty much any other statically typed language is likely to be more productive in the long run, once you account for the costs of Scala's complexity. Java, Kotlin, C#, whatever. And if you don't need or want the benefits of statically typed languages, then the same is also true for most major dynamic languages. It's difficult to be more specific than that because I don't think there's a very compelling case to use Scala to begin with.

markmark8 anni fa

When using a dynamic language you need to know what properties are on an object right? So you can work with them, access them, whatever. So just write it down so everyone else knows too. There, that was easy.

wtetzner8 anni fa

If that’s the case, then why not have the compiler check them as well?

markmark8 anni fa

Yes, that's exactly my point. Right them down in your program so that the compiler can check them rather than holding them in your head.

wtetzner8 anni fa

Sorry, I misunderstood your comment.

pg_bot8 anni fa

I would disagree with the statement that coding is faster in statically typed projects. The ease of onboarding new devs is solely based on the code quality of the project. Well written programs are easy to understand regardless of typing. Terribly written programs are opaque regardless of typing.

eropple8 anni fa

Those Scotsmen are pretty untrue, huh?

The claim is pretty facially silly at a glance besides, though. You literally don't know the fields and methods, to say nothing of what they actually give you, on a Ruby object without opening either the code or pulling up Pry and doing `ls`. Which I have done, often enough, to fail rather stunningly to miss doing so when in Kotlin or when using TypeScript. Checked that one? Good news--now you get to drill down into the next one, and the next one. I hope that isn't coming from another library, too, 'cause jumping from library to library in Ruby or Python is a chore. And even if not? God help you if you're using ActiveRecord, I hope you enjoy scanning your database migrations or describing tables just to learn method names.

It is a bad scene. And don't get me wrong; I like Ruby quite a lot. But I won't make excuses for it.

Personally, with the actual tooling available to modern systems, I write TypeScript much, much faster--we're talking probably twice as fast--than I do even ES6, which I otherwise quite like. And TypeScript can't even assert whether something is a whole number or not, unlike better languages. It has replaced Ruby for me on the strength of that alone.

Tehnix8 anni fa

Sure, any code base can be horrible to learn, but two equally well written code bases, types can help significantly. When your new employee can be confident that changes something in one place, won't invisibly break something in another place, they are more prone to start making changes.

Vimal Kumaar touches on how it has helped them get employees contributing to the code a lot quicker, after they moved to PureScript https://www.youtube.com/watch?v=HLEwYghBjo8 (I think around 24+ minutes in, but I recommend the whole talk).

matchbok8 anni fa

If I have to connect a couple classes in separate files with a few objects/stuff inside of them, I'd bet good money that a statically typed language will provide a faster coding experience. Especially if you have leveraging third party libraries. Working in huge libs in JS (Sequelize comes to mind) is a horrible development experience that requires you to have the documentation open.

Tehnix8 anni fa

What languages have you worked with before, with static types?

If it's something like Java, C, C++, etc, then I can see your point, but if you pick any of the languages with type inference (like Elm, Haskell, PureScript, OCaml, F#, etc), then you rarely actually have to write types.

Working with the type system is a different way of working. It's more beneficial to see the compiler as a helping friend, instead of an adversary that is just there to complain.

Some interesting reads might be:

- http://elm-lang.org/blog/compilers-as-assistants

- https://matthew.brecknell.net/post/hole-driven-haskell/

- http://bitemyapp.com/posts/2017-09-23-please-stop-using-type...

Personally, I honestly hate every time I don't have a time system to rely on. I don't make any false pretenses that I have the whole program in my head 100% of the time. I don't want to have to think about things that a computer can solve for me, hence I like my compiler to do as much work as possible.

As for unit testing, a type system is never supposed to remove the need for unit testing, and if you see anyone promote a language because of this, they are probably (hopefully) over simplifying. What it does do though, is significantly cut down the need for certain types of tests, since you can model so much more logic via types (ADTs, Union types, etc), and get it checked by the compiler.

As an aside: I simply do not believe you've never run into a type error, unless you've only written projects that 100 lines of code, and even then it's quite easy to run into type errors.

pg_bot8 anni fa

The languages that I've used with static typing are Java and Objective-C so perhaps those languages have negatively impacted my perception of type systems. I can do more research on other languages to see what they have to offer.

I used to run into type errors when I first started programming but I've learned how to eliminate them by changing my programming style. I add a lot of constraints to how I program, and follow a system of naming conventions that make it obvious what type something is.

I take a lot of pride in my work and try my best to build maintainable systems that can be extended easily or run forever without changes. I typically write around 250K-350K lines of code per year in mission critical and highly regulated industries. If you want to learn how to write systems that have zero defects research companies that are known for quality design and apply their principles to your own work. I would suggest Netflix, and Toyota as the two companies that I would emulate.

emodendroket8 anni fa

> I add a lot of constraints to how I program, and follow a system of naming conventions that make it obvious what type something is.

And you find it unbelievable that someone would prefer that the compiler simply enforce these conventions?

pg_bot8 anni fa

I don't find it unbelievable, I just personally don't get the appeal because you have to do that work when you name things anyway. e.g. I would never expect a variable named "first_name" to be anything other than a string. It's also fairly easy to handle most type conversions so I would rather be permissive in what I accept (to a reasonable point) than be rigid.

emodendroket8 anni fa

Well what are the properties of config_options or app_user or objects like these? Is it even documented, or am I forced to scan the entire method trying to figure out?

pg_bot8 anni fa

Typically I would initialize an AppUser object and print out its default representation. You can quickly find info on an object by doing something like:

    AppUser.new 
      => prints all properties of the object with nil values 

    AppUser.instance_methods - ActiveRecord::Base.instance_methods 
    => prints all unique instance methods on that class

values

config_options is extremely generic but I would expect that it is a Hash of key/value pairs. Given the fact that it has options in the name you are likely going to have to look it up regardless as you have multiple choices when using it. I'd expect some documentation if it is truly configurable, or you can transparently look at the method signature if it uses keyword args. (say for a library that handles formatting currency)

Quickly reading other people's code is a definite skill. While it can be a pain to do, it is worthwhile since the code is the only real source of truth.

emodendroket8 anni fa

Yeah, of course there are ways of doing it. But if you define types you can in a second be looking at all the possible values. The tools help you be more productive and make fewer mistakes.

veebat8 anni fa

Well, what kind of projects? And what order of magnitude big?

Whether dynamic type errors become an issue depends a lot on how often all codepaths are run.

If what you're running is a batch processing script, it's likely to crash out as quickly as a type-checking compiler, and then static types have no direct appeal.

On the other hand, if it's a long-running server with a lot of finite state machines(maybe it's running a multiplayer game or something) it's really hard to reach type errors without an extensive manual test.

This becomes a relatively bigger issue as the codebase gets bigger, again depending on the type of code. If the main way the system expands is by handling a more diverse set of data types, dynamic types allow less code to be written, but also make it harder to verify.

In practice what I tend to see happening is that a glue layer appears that needs dynamic behavior, while the codepaths underneath that benefit more from static typing.

rickyc0918 anni fa

The usefulness really depends on the team. If you're working with another engineer or two and they're all senior engineers that you've coded with for a while, then it's probably not necessary since you'll understand each other.

However, if you work with a junior team or you have to inherit a project, then I've found static typing to be extremely useful. One might argue that code review is suppose to solve for that, and while you can catch some of the problems, you won't catch all.

It's definitely helped to make refactoring faster. Specs will help to some degree, but static typing will traverse through every function to ensure the parameter you're passing is as expected.

No more is this suppose to be a nil, a blank value? You can specifically define what you expect. While some people hate optionals with Swift, I've found it to be an interesting concept.

bastawhiz8 anni fa

I started working with TypeScript over vanilla JS a year or so ago. It's completely changed the way I work. Having autocomplete, pointing out silly errors (forgetting to return a value, forgetting to set a prop on a React component, forgetting to add a member to an object) before I refresh the page, and being able to pick up old code without having to reread through it to figure out what the hell it actually spits back out is a godsend. The latter has sped up my productivity significantly; being able to get the output from a library, hover over the variable, and see the exact shape of the output saves an incredible amount of time when doing mundane day-to-day tasks.

kjeetgill8 anni fa

You've gotten a lot of responses but let me throw one more idea at you: bad or mediocre code bases. Or old codebases that have the scars of years of tweaks, fixes, and hacks. Thats where I think static types shine the most.

Comparing my Python and Java experience, jumping into the middle of a codebase and correctly untangling a few particularly problematic spots or reworking and refactoring packages without introducing errors is dramatically easier with static typing.

It's feels like the difference between untangling actual wet noodles vs solving a sliding block puzzle. It's just more mechanical.

Even BEFORE running tests there is a lot of confidence that you haven't overlooked something. Hell, you can't catch typos in python without 100% code coverage. The sanity bar is just that much higher before you even get to unit tests.

zxcvhjkl8 anni fa

For me, static typing actually makes it easier and faster to write code. It’s like someone looking continuously over my shoulder for mistakes.

I am the least efficient when I write code in JS.

On the other hand with Ruby, and meta programming, I can do stuff that would be impossible to express in the most popular type systems.