Python 3.5 type hinting in PyCharm 5(blog.jetbrains.com)
blog.jetbrains.com
Python 3.5 type hinting in PyCharm 5
http://blog.jetbrains.com/pycharm/2015/11/python-3-5-type-hinting-in-pycharm-5/
14 comments
What I don't understand is why it's not:
def greeting(name: str): str:
def greeting(name: str): str
The name: implies that the type of the "name" object is str. If you were consistent, then, def greeting(...):str would imply that the type of the greeting object is str, which is definitely not the case (it is "function"). The -> isn't defining the type, it's specifying what is returned.
But it's not the name alone, it's the name with the signature. `greeting` might be `function`, but `greeting("nikolay")` type would be `str`.
For Python 2 you could
The syntax isn't the magic sauce, the type checker is.
http://www.mypy-lang.org/
https://github.com/google/pytype
greetings.__annotations__ = {'name': str, 'return': str}
The syntax isn't the magic sauce, the type checker is.
http://www.mypy-lang.org/
https://github.com/google/pytype
I'm insanely happy that I have a new job that is not stuck on Python 2.x
This is one reason. All my personal fun-type projects are in the latest Python, and I have the leeway to update work to this.
I know there are a lot of people who don't like or want this, but coming from a C# background before Python, I really like this feature. (not that I want Python to be more like C#, I just like the safety net and guaranteed documentation.)
This is one reason. All my personal fun-type projects are in the latest Python, and I have the leeway to update work to this.
I know there are a lot of people who don't like or want this, but coming from a C# background before Python, I really like this feature. (not that I want Python to be more like C#, I just like the safety net and guaranteed documentation.)
The "typing"¹ backport package on pypi claims to be compatible back to 2.7.
EDIT: A gem i found on Reddit: Configure setuptools to install it only if required².
[1] https://pypi.python.org/pypi/typing
[2] http://stackoverflow.com/a/32643122
EDIT: A gem i found on Reddit: Configure setuptools to install it only if required².
[1] https://pypi.python.org/pypi/typing
[2] http://stackoverflow.com/a/32643122
How is it backwards compatible? Was there syntax added to the language in preparation?
Correct. Annotations were added in 3.0 with no encouragement to use them for anything in particular, so they could observe what the community would use them for. They were just put in a dictionary attached to the function, and by inspection you could access them and do whatever the heck you wanted with them.
A lot of third-party libraries cropped up that used them to type-check stuff. Type-checking was a popular enough usage for them it's official now, to help gather people under the same libraries and terminology.
You can still use the annotations for other things, of course – the dictionary is still attached to the function and can be inspected as before – but if nothing else is specified, we can now assume they are type declarations.
A lot of third-party libraries cropped up that used them to type-check stuff. Type-checking was a popular enough usage for them it's official now, to help gather people under the same libraries and terminology.
You can still use the annotations for other things, of course – the dictionary is still attached to the function and can be inspected as before – but if nothing else is specified, we can now assume they are type declarations.
https://www.python.org/dev/peps/pep-3107/
They are just arbitrary expressions (though in Python, type names like 'int' or 'str' are themselves objects). They don't do anything by default but are accessible as fields of the function object.
They are just arbitrary expressions (though in Python, type names like 'int' or 'str' are themselves objects). They don't do anything by default but are accessible as fields of the function object.
It was in there for a long time, just nobody used it.
I have no idea how they managed to find something that didn't throw a syntax error, yet looked reasonable. Perhaps someone who followed the process will chime in.
[deleted]
That's the syntax? Looks to me like another reason to stick with Python2.
It's optional typing: you don't have to use it with Python 3.
That sounds more like a C++ than Python design sensibility.
To me it sounds like addressing a pain point for larger projects, moving the language forward, and adding something many in the community has long wanted (as evident by blog posts and articles, earlier type checking tool attempts, the acceptance of the relevant PEP, etc).
Yes and that sounds unfamiliar from Python cultural POV. Python used to be the language that didn't try to please everyone and kept a coherent design philosophy that resulted in a clean and simple language.
>Python used to be the language that didn't try to please everyone
On the contrary. I use Python professionally since 1997, and it has had all kinds of features added on for different needs, from decorators and generators to async and virtual environments, context managers, list comprehensions, dict comprehensions and what have you.
Of the languages I know of, only C# has had a similar pace of adding new stuff in to the core language (not just libraries).
When did Python shy away from adding new features?
It seems that Python didn't try to please only 2 categories of people: those who dislike significant whitespace, and those who want proper closures.
>and kept a coherent design philosophy that resulted in a clean and simple language
At this point I'd say that Python, with its full feature set, is as complex as a scripting language gets. Maybe Perl 6 trumps it, not sure -- but what you describe is closer to a language like Lua, or tcl or even Go than Python.
On the contrary. I use Python professionally since 1997, and it has had all kinds of features added on for different needs, from decorators and generators to async and virtual environments, context managers, list comprehensions, dict comprehensions and what have you.
Of the languages I know of, only C# has had a similar pace of adding new stuff in to the core language (not just libraries).
When did Python shy away from adding new features?
It seems that Python didn't try to please only 2 categories of people: those who dislike significant whitespace, and those who want proper closures.
>and kept a coherent design philosophy that resulted in a clean and simple language
At this point I'd say that Python, with its full feature set, is as complex as a scripting language gets. Maybe Perl 6 trumps it, not sure -- but what you describe is closer to a language like Lua, or tcl or even Go than Python.
It's true that the pace of cruft accumulation is worrying lately. Note that I said "used to"! Personally I think it all started to go overboard around the introduction of list comprehensions when we had perfectly good map, filter and lambda.
For those curious, list comprehensions were introduced in 2000 with Python 2.0.
> Yes and that sounds unfamiliar from Python cultural POV.
Addressing pain points is unfamiliar to the Python culture?
In practice, type intransparency is a serious problem – duck typing is only fun as long as everyone a) knows what a duck is and b) agrees on what it should do. Type hints make it a lot easier to work with opaque libraries and inside large codebases.
Addressing pain points is unfamiliar to the Python culture?
In practice, type intransparency is a serious problem – duck typing is only fun as long as everyone a) knows what a duck is and b) agrees on what it should do. Type hints make it a lot easier to work with opaque libraries and inside large codebases.
> Python used to be the language that didn't try to please everyone and kept a coherent design philosophy that resulted in a clean and simple language.
Uh, ever seen "Batteries included" referenced in their documentation?
Uh, ever seen "Batteries included" referenced in their documentation?
Yes, but it's definitely not the same thing. The vice of trying to please everyone means giving various people the features they ask for instead of weighing against the common good of the user base.
Still is clean and simple. I like the new syntax.
What more agreeable syntax could you possibly ask for?
def (str) greeting((str) name):
I didn't downvote, but I don't like the syntax for a couple of reasons:
1. Instead of being more like C-style function declarations, they look a lot more like C-style type conversions.
2. However closer it may be to C (type before identifier I guess?) it's pretty far from any other mainstream language. For instance, C++ has support for -> syntax for function types (http://stackoverflow.com/questions/22514855/arrow-operator-i...) and some other languages (I can think of Scala) use name: Type declarations.
As a personal preference, I find the python type hinting more familiar than your example, despite (myself) being most familiar with C-like syntax.
1. Instead of being more like C-style function declarations, they look a lot more like C-style type conversions.
2. However closer it may be to C (type before identifier I guess?) it's pretty far from any other mainstream language. For instance, C++ has support for -> syntax for function types (http://stackoverflow.com/questions/22514855/arrow-operator-i...) and some other languages (I can think of Scala) use name: Type declarations.
As a personal preference, I find the python type hinting more familiar than your example, despite (myself) being most familiar with C-like syntax.
You're too familiar with C-style languages. When I initially picked up Scala the : type syntax felt a bit strange, but I really came to like it and am happy when it's in other languages.
Putting the type after the name feels more natural, and the : also translated nicely into 'of type' (or if you prefer for functions 'returns type').
Putting the type after the name feels more natural, and the : also translated nicely into 'of type' (or if you prefer for functions 'returns type').
Python is not C so why should it suddenly "pretend to be" when doing type hinting? Also, I personally find that the type declaration format of C is one of my least favorite things about C.
Tons of extra needless parenthesis for what exactly? To look more like C? It's not like (foo:x) is not also a common type of type annotation. Besides they look more like casts than type declarations to a C programmer.
I didn't downvote, but it being C-style syntax is not a recommendation in this context.
For familiarity, while the C family of languages favors "type name" declarations, the Pascal and ML family of languages use "name: type" instead. So, familiarity is entirely a function of what language you used before.
Second, C-style declarations have their problems, which is why they aren't used much outside of languages that derive from C. Generally, you look up declarations by searching for the name of the variable to determine the type; this generally has less cognitive overhead if the variable is at the start of the clause, not the end, ideally separated by punctuation (note how the "name: type" syntax mirrors Python dictionary "key: value" syntax) [1].
[1] Also, in C/C++ the "type name" syntax complicates the parser, as the parser has to be able to distinguish between type names and other identifiers; while this wouldn't be an issue for Python, there is still no reason to perpetuate this syntax choice.
For familiarity, while the C family of languages favors "type name" declarations, the Pascal and ML family of languages use "name: type" instead. So, familiarity is entirely a function of what language you used before.
Second, C-style declarations have their problems, which is why they aren't used much outside of languages that derive from C. Generally, you look up declarations by searching for the name of the variable to determine the type; this generally has less cognitive overhead if the variable is at the start of the clause, not the end, ideally separated by punctuation (note how the "name: type" syntax mirrors Python dictionary "key: value" syntax) [1].
[1] Also, in C/C++ the "type name" syntax complicates the parser, as the parser has to be able to distinguish between type names and other identifiers; while this wouldn't be an issue for Python, there is still no reason to perpetuate this syntax choice.
Definitely not downvoting - but, all the extra parentheses are a kind of lispy in having to grok (and balance). Also, having the (str) at the beginning makes me think that it's an input, whereas having -> makes it clear (to me at least) that it's an output return type.
Something that doesn't require a -> and something with type prefix rather than postfix. Still sticking with 2.7, thanks.
I'll bite - what would have liked to see in terms of syntax to (optionally as others have noted) provide hints as to what type each data element should be?
It was intended to be so ;-)
This was talked about and was there for a while.
https://www.python.org/dev/peps/pep-3107/ [2006!]
These weren't specifically called "type" annotations but I think that was in the back of everyone's mind.
This was talked about and was there for a while.
https://www.python.org/dev/peps/pep-3107/ [2006!]
These weren't specifically called "type" annotations but I think that was in the back of everyone's mind.
Recently stumbled upon IntelliJ using the Go plugin package after a Rubyist friend of mine spoke very highly of RubyMine. I have to say I was incredibly impressed. I looked at it a couple years ago when I was mostly Python and it was ok, but it's really stepped up its game - both feature-wise and visually.
The IntelliJ IDE set is really far better than I understood. IMHO.
The IntelliJ IDE set is really far better than I understood. IMHO.
The things you could do in Go vs in (Python | Ruby) are actually are results of difference between static vs dynamic typing. Yeah, static typing requires more key presses and some thinking, but in the case of tools (navigation, refactoring) it makes a huge difference. Ruby support would be years ahead of what it is now if the tools could understand the types in all cases.
I can see this being very useful in teams. Hell, its even useful for a single developer as a form of future documentation. I generally write code today and think, when I read this in 6 months and have no memory, how can I help my future self? For new code bases I'll be adding this.
For dynamically typed languages, using a popular type annotation syntax (e.g. rdoc for ruby, closure compiler for javascript) and an IDE (like Intellij) can give you 80% of the benefits of type safety by doing code inspection on-the-fly and warning you when it finds incorrect types. The main downside is no compiler safety net when you try to refactor code.
I've been very happy about more interpreted languages adding optional typing for exactly this kind of reason.
I'm the author of the article. When I gave a preview of this at the PyRVA meetup, I gave this exact illustration, even using "my future self". So...jinx. :)
haha maybe we're twins!
I wonder why, given they call it 'type hints', they didn't go for Haskell-like type comments:
#greeting :: (str, int) -> str
def greeting(name, age):
I've wondered this exact same thing myself for other languages. I doubt its a lack of familiarity since I would bet most compiler/language authors are aware of most Haskell concepts/idioms. I think it probably has more to do with function definition semantics in non-functional languages but it seems that could be easily worked around...
Anyway the haskell way, especially with Typeclasses, has to be clearest way to express type constraints for functions I've encountered at least from a code readability standpoint.
Anyway the haskell way, especially with Typeclasses, has to be clearest way to express type constraints for functions I've encountered at least from a code readability standpoint.
I love Haskell, but I'm not sure this would actually be better for Python. To be Haskell-like, the comment would actually be different than what you wrote:
#greeting :: str -> int -> str
and this is specifically related to the functional nature of Haskell. `greeting "hello"` now instantly becomes a function with type :: int -> str. So the Haskell syntax is about currying, it's about how the function you wrote is really just a cascade of functions.
Doing something silly like making the "arguments" reside in an artificial tuple would definitely be bad practice in most Haskell cases, and so syntactically looks a bit weird and out of place, like it's shoehorned on just because Python function arguments are not like Haskell arguments.
Since Python is not based on this model of a function, I think it's a lot better to keep the syntax in the actual def statement. Adding the return type annotation at the end is nice.
Most probably, this stuff was added to the signature because there were already natural tools like the inspect library that would provide the immediate hooks for making use of it. But even if the implementation didn't need it to be part of the signature like that, I still think it's better for Python's paradigm.
#greeting :: str -> int -> str
and this is specifically related to the functional nature of Haskell. `greeting "hello"` now instantly becomes a function with type :: int -> str. So the Haskell syntax is about currying, it's about how the function you wrote is really just a cascade of functions.
Doing something silly like making the "arguments" reside in an artificial tuple would definitely be bad practice in most Haskell cases, and so syntactically looks a bit weird and out of place, like it's shoehorned on just because Python function arguments are not like Haskell arguments.
Since Python is not based on this model of a function, I think it's a lot better to keep the syntax in the actual def statement. Adding the return type annotation at the end is nice.
Most probably, this stuff was added to the signature because there were already natural tools like the inspect library that would provide the immediate hooks for making use of it. But even if the implementation didn't need it to be part of the signature like that, I still think it's better for Python's paradigm.
I don't think this is a case of shoehorning anything. It is true that in Haskell you use currying more often, while in Python you usually supply a tuple with all the arguments at once.
However there are also cases in Python where you get currying, like decorators. And for complicated types like:
However there are also cases in Python where you get currying, like decorators. And for complicated types like:
# greeting :: (str -> (str, str), int -> List[str]) -> Dict[str, int -> int]
But in Python, to do currying, you either use decorators, functools.partial, or forced early binding (like nested function factories), etc. Just because currying is a thing you can do in the language (in a much different way than in Haskell where it is fundamental to every function) doesn't seem like a good reason to me to adopt Haskell-like type signatures.
Just for example, in Python it's extremely common to use default arguments, and in fact a lot of stuff in the typing module exists to make it easier to do something like make an Optional type that can either be `None` or a `str` or something. So whatever type signature you're going to mimic from Haskell, you won't be able to give optional types or default arguments without making it totally un-Haskell. And of course no one is going to go off and change Python to work based on Haskell-like type classes (nor should they).
Just for example, in Python it's extremely common to use default arguments, and in fact a lot of stuff in the typing module exists to make it easier to do something like make an Optional type that can either be `None` or a `str` or something. So whatever type signature you're going to mimic from Haskell, you won't be able to give optional types or default arguments without making it totally un-Haskell. And of course no one is going to go off and change Python to work based on Haskell-like type classes (nor should they).
Mypy does support type comments in the checker.
http://mypy.readthedocs.org/en/latest/type_inference_and_ann...
http://mypy.readthedocs.org/en/latest/type_inference_and_ann...
They stuck with a syntax that was valid in older versions of Python, and used the function annotation work from Python 3.4. Thus, no "clean slate, choose the ideal syntax" discussion.
There's quite some dissonance between the artificial "generics" vs normal types. Why the concrete str type for strings, but not for lists? I realize there are implementation level arguments for it, but the resulting compromise does not seem good.
And specifying the concrete str type goes diametrically against the "don't check against a concrete type, embrace duck typing" mantra that Python programmers are taught early.
And specifying the concrete str type goes diametrically against the "don't check against a concrete type, embrace duck typing" mantra that Python programmers are taught early.
Probably because Python doesn't distinguish characters and strings. When you index into a string, you get a string back (that happens to be length one, of course), which means that a string is a list of strings. Thus, exploding string into a list type isn't really possible as the type of `str` is something like `List[str]`, which doesn't make a whole lot of sense.
> Why the concrete str type for strings, but not for lists
Because you would need to extend the base list type to support indexing just for this feature, which is a bit iffy if you ask me. I think you have a point about "don't check against concrete types" though, they should update the docs to suggest using an 'Iterable[type]' over a 'List[type]' unless you really really really want a list.
Because you would need to extend the base list type to support indexing just for this feature, which is a bit iffy if you ask me. I think you have a point about "don't check against concrete types" though, they should update the docs to suggest using an 'Iterable[type]' over a 'List[type]' unless you really really really want a list.
See Obiwan. pypi.python.org/pypi/obiwan/ Prettier?
(author)
(author)
something that's always had me a bit confused:
the type hinting in Python is just that -- hinting, right?
It's for IDEs like PyCharm to use to generate better docs and to be more intelligent about suggestions when writing code. But what it does NOT do is inform the interpreter/compiler about type information, correct?
I.e., Python3 code with perfectly implemented type hints would only make it easier on the developer, and not the interpreter/compiler, yeah?
the type hinting in Python is just that -- hinting, right?
It's for IDEs like PyCharm to use to generate better docs and to be more intelligent about suggestions when writing code. But what it does NOT do is inform the interpreter/compiler about type information, correct?
I.e., Python3 code with perfectly implemented type hints would only make it easier on the developer, and not the interpreter/compiler, yeah?
The CPython interpreter does not enforce any errors/warnings/etc. based on annotations. It does programmatically expose annotations if you want to inspect them at runtime, but the feature is mostly meant for other tools to optionally hook into.
Yes, but there's nothing to say that an interpreter couldn't use the type hinting. Imagine a pypy-hinting interpreter that analysed the code and kept the hinting, it could theoretically JIT it better.
Whether or not this would be helpful is discussed here:
http://doc.pypy.org/en/latest/faq.html#would-type-annotation...
http://doc.pypy.org/en/latest/faq.html#would-type-annotation...
tl;dr type hints are useless because they're hints.
>I.e., Python3 code with perfectly implemented type hints would only make it easier on the developer, and not the interpreter/compiler, yeah?
At the moment no. But there's nothing to say that it couldn't happen in 3.8 or 3.11 or 4.x, since the groundwork is there now.
At the moment no. But there's nothing to say that it couldn't happen in 3.8 or 3.11 or 4.x, since the groundwork is there now.
Oh, I was somehow hoping this would allow you to have multiple function signatures. i.e. two functions foo, both accepting one variable named bar, but one would implement str and one int, for example.
You can have that since 3.4, thanks to PEP 443 (https://www.python.org/dev/peps/pep-0443/). Here's a (slightly contrived) example: https://gist.github.com/zsiciarz/e15199d9be1b0e07faf6
To help with the phrasing: this is called function overloading. Type hinting is ignored by the interpreter so it doesn't add any new functionality, only documentation.
You could add a decorator or metaclass that dynamically checks the hints at runtime.
So far I've been using Obiwan [1] to check types during testing. It's definitely not something you want to do all the time, because it has a huge runtime cost.
[1] https://github.com/williame/obiwan
What I do is I have my test runner look for the PYTHON_TYPECHECK environment variable, and if it's set, it imports and enables Obiwan.
I've caught a few subtle bugs this way.
[1] https://github.com/williame/obiwan
What I do is I have my test runner look for the PYTHON_TYPECHECK environment variable, and if it's set, it imports and enables Obiwan.
I've caught a few subtle bugs this way.
This is a job for the little used -O flag.
Are there a any good libraries to turn on/off type checking using if debug as part of the decoration logic? If so then running with -O in production would solve some of the problem with this.
Are there a any good libraries to turn on/off type checking using if debug as part of the decoration logic? If so then running with -O in production would solve some of the problem with this.
Until an interpreter/compiler supports otherwise, anyway ;)
Correct.
"Type hinting" without checking is kind of lame. If you're going to have declarations such as
Somebody should stop Python's little tin god before he creates another giant step backwards like the Python 2 to 3 mess.
def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None:
Somebody should stop Python's little tin god before he creates another giant step backwards like the Python 2 to 3 mess.
I would have to say that I agree. A friend of mine put it this way, "If you expect me to put specify a type and not enforce it, you're just trolling".
It's an optional addition to a dynamic scripting language. Lightweight when you need it, more robust when you need it. Sounds like you are trolling.
It's not more robust though. The type hints do absolutely nothing.
False, they provide information to analysis tools, which can prevent errors.
If there is a tool which statically analyzes your code and reports errors due to type mismatches, does it matter to you whether that tool has the exact literal name "ahead-of-time compiler"?
What does matter to me is whether the standard build tool for the language will run that tool on a proposed library release before that library gets uploaded to PyPI (or equivalent). If there are type annotations in the standard package repository that are lies then that seriously limits the usefulness of the whole thing.
> If there are type annotations in the standard package repository
There aren't.
> whether the standard build tool for the language will run that tool on a proposed library release before that library gets uploaded to PyPI
It's optional. If the library maintainer wants to do that then he can.
There aren't.
> whether the standard build tool for the language will run that tool on a proposed library release before that library gets uploaded to PyPI
It's optional. If the library maintainer wants to do that then he can.
> It's optional. If the library maintainer wants to do that then he can.
A library with no annotations in PyPI is fine-ish. A library with correct annotations in PyPI is fine. A library with incorrect annotations in PyPI is extremely not-fine.
I'm fine-ish with annotations being optional, but if they're there then their correctness needs to be enforced, otherwise they're worse than useless.
A library with no annotations in PyPI is fine-ish. A library with correct annotations in PyPI is fine. A library with incorrect annotations in PyPI is extremely not-fine.
I'm fine-ish with annotations being optional, but if they're there then their correctness needs to be enforced, otherwise they're worse than useless.
> A library with incorrect annotations in PyPI is extremely not-fine.
As is a library with broken logic or one that throws exceptions randomly. You want Guido and the developers to implement some kind of system that automatically scans all PyPi packages for any kind of errors? I'm sure them solving the halting problem won't take that long.
Incorrect annotations are a bug. The onus is on the maintainers of packages to fix bugs in their packages, not PyPi which is a distribution channel.
As is a library with broken logic or one that throws exceptions randomly. You want Guido and the developers to implement some kind of system that automatically scans all PyPi packages for any kind of errors? I'm sure them solving the halting problem won't take that long.
Incorrect annotations are a bug. The onus is on the maintainers of packages to fix bugs in their packages, not PyPi which is a distribution channel.
> As is a library with broken logic or one that throws exceptions randomly. You want Guido and the developers to implement some kind of system that automatically scans all PyPi packages for any kind of errors?
Yes I do, but perfect is the enemy of good; it's ok to do something that makes things better without solving the whole problem. I want the build system to encourage people to do the right thing. E.g. any serious build system already ensures that you run the unit tests as part of the build or at least release process, because it would be just dumb to publish a library release on PyPI if it failed the unit tests.
> I'm sure them solving the halting problem won't take that long.
We already know how to avoid the halting problem, it's called types; see e.g. Idris.
But you don't need to solve the halting problem to check annotations. It's not even hard. Why would you want to not check them? I mean if you're not even going to check them then why bother having annotations in the first place? Comments would provide the same functionality and have less danger of being misleading, because everyone already knows comments often end up being out of date.
Yes I do, but perfect is the enemy of good; it's ok to do something that makes things better without solving the whole problem. I want the build system to encourage people to do the right thing. E.g. any serious build system already ensures that you run the unit tests as part of the build or at least release process, because it would be just dumb to publish a library release on PyPI if it failed the unit tests.
> I'm sure them solving the halting problem won't take that long.
We already know how to avoid the halting problem, it's called types; see e.g. Idris.
But you don't need to solve the halting problem to check annotations. It's not even hard. Why would you want to not check them? I mean if you're not even going to check them then why bother having annotations in the first place? Comments would provide the same functionality and have less danger of being misleading, because everyone already knows comments often end up being out of date.
I'm really confused here. Are you really advocating that PyPi performs some kind of magic on the thousands and thousands of packages that are uploaded to ensure that all their unit tests pass, including testing the type annotations? Can you please give me an example of another languages packaging site with a non-trivial amount of packages that does this, or at least explain why this isn't the developers responsibility?
All you seem to be saying is that "Code should be tested before being uploaded to PyPi", and somehow that equates to type annotations being bad? Again, it's up to the developer to ensure his tests pass (by using a ci, or running tests locally, or even making a blood sacrifice to Cthulhu), and yes this should include a linter to ensure type annotations are correct if they are using them.
All you seem to be saying is that "Code should be tested before being uploaded to PyPi", and somehow that equates to type annotations being bad? Again, it's up to the developer to ensure his tests pass (by using a ci, or running tests locally, or even making a blood sacrifice to Cthulhu), and yes this should include a linter to ensure type annotations are correct if they are using them.
> I'm really confused here. Are you really advocating that PyPi performs some kind of magic on the thousands and thousands of packages that are uploaded to ensure that all their unit tests pass, including testing the type annotations?
I'm saying that the build tool should run unit tests and check annotations before uploading a package to PyPI. It doesn't matter whether checking annotations is done by a "compiler" or some other tool, but it does matter that whatever tool that is runs as part of the language standard release process (and in practice it tends to be the case that when a compiler enforces types then this happens, and when an external tool does then it doesn't).
> it's up to the developer to ensure his tests pass (by using a ci, or running tests locally, or even making a blood sacrifice to Cthulhu)
Leaving it up to the developer is a recipe for it not happening. We're programmers, we should automate these things.
I'm saying that the build tool should run unit tests and check annotations before uploading a package to PyPI. It doesn't matter whether checking annotations is done by a "compiler" or some other tool, but it does matter that whatever tool that is runs as part of the language standard release process (and in practice it tends to be the case that when a compiler enforces types then this happens, and when an external tool does then it doesn't).
> it's up to the developer to ensure his tests pass (by using a ci, or running tests locally, or even making a blood sacrifice to Cthulhu)
Leaving it up to the developer is a recipe for it not happening. We're programmers, we should automate these things.
From the IDE point of view, there are other benefits besides correctness. Proper autocomplete for objects (huge whenever you have a chain of method calls), ctrl+click to go to a method, showing a method's documentation in a popup, find usages - all these require the IDE to know about types if they are to work reliably.
No, they aren't. In order to make use of type hints for anything, they need to be correct. If they aren't, anything else you build on top of them will be unreliable and confusing. As long as there is no way to check them, they're worthless.
If a library comes annotated with types, then an error in the typehints is just as much of a bug as an error in code or documentation.
Having most of the standard library plus large portions of the ecosystem come with these hints will be hugely beneficial for productivity.
And of course, there IS a way to check them. IDEs, linters, etc. - I'm sure if you say that your function returns an instance of User and then you try to return a boolean, they will yell at you. And I expect that the standard library and popular libraries WILL run these tools before releasing a stable version. A 90% percent solution is still a step forward compared to relying on much less reliable human brains at every step.
Having most of the standard library plus large portions of the ecosystem come with these hints will be hugely beneficial for productivity.
And of course, there IS a way to check them. IDEs, linters, etc. - I'm sure if you say that your function returns an instance of User and then you try to return a boolean, they will yell at you. And I expect that the standard library and popular libraries WILL run these tools before releasing a stable version. A 90% percent solution is still a step forward compared to relying on much less reliable human brains at every step.
I understand that errors in type hints are bugs just like errors in code are. That's the problem.
The entire point of type systems is to enforce correctness. They allow you to describe problems on the type level and ensure that the code being described matches that description, eliminating certain classes of bugs. That's why languages have them, that's the expectation.
Type hints are just an overhyped standardized comment format with special syntax support.
The entire point of type systems is to enforce correctness. They allow you to describe problems on the type level and ensure that the code being described matches that description, eliminating certain classes of bugs. That's why languages have them, that's the expectation.
Type hints are just an overhyped standardized comment format with special syntax support.
I agree that type hints are basically just "machine-readable documentation" (and human-readable as well!).
I still think that machine-readable documentation can be immensely useful when combined with a static code analysis tool. Not as useful as "real" static types would be, but useful nonetheless.
It is the exact same idea as Microsoft's SAL annotations for C/C++ - https://msdn.microsoft.com/en-us/library/ms182032.aspx, or Java's @NonNull, @ReadOnly, etc. annotations.
I still think that machine-readable documentation can be immensely useful when combined with a static code analysis tool. Not as useful as "real" static types would be, but useful nonetheless.
It is the exact same idea as Microsoft's SAL annotations for C/C++ - https://msdn.microsoft.com/en-us/library/ms182032.aspx, or Java's @NonNull, @ReadOnly, etc. annotations.
> If functions have type specifications, you should not be able to call them with the wrong type, ever.
This assumes you can represent a type specification for the function you are calling, this isn't always possible in a dynamic language with advanced metaprogramming facilities.
Think of it as modularizing the compiler and runtime so that the dials on dynamism can be tuned by the team.
PyCharm already used pretty sophisticated type inference this just helps it. But the real win is being able to run mypy or pytype over a codebase automatically.
Types, Contracts, Properties, Tests are different directions in a tensor field. With Python you can mix and match at your will. This is in _no way_ relatable to the 2-3 mess.
This assumes you can represent a type specification for the function you are calling, this isn't always possible in a dynamic language with advanced metaprogramming facilities.
Think of it as modularizing the compiler and runtime so that the dials on dynamism can be tuned by the team.
PyCharm already used pretty sophisticated type inference this just helps it. But the real win is being able to run mypy or pytype over a codebase automatically.
Types, Contracts, Properties, Tests are different directions in a tensor field. With Python you can mix and match at your will. This is in _no way_ relatable to the 2-3 mess.
So just run the compiler and runtime type checker, what's the problem?
Usually optional tooling tends to be ignored.
For example, C would have been a much safer language if lint was part of the compiler, instead of being optional.
For example, C would have been a much safer language if lint was part of the compiler, instead of being optional.
Sure, but how is that "another 2 to 3 mess"? If anything, the backwards-incompatible changes made the language more robust.
I was replying to the issue that this being two separate tools:
"This is the job of the compiler and the run time system, not the IDE."
Making this optional will mean the majority won't bother to run the checker.
How many do bother to run lint, jslint, core.typed or dialyzer?
"This is the job of the compiler and the run time system, not the IDE."
Making this optional will mean the majority won't bother to run the checker.
How many do bother to run lint, jslint, core.typed or dialyzer?
that doesn't make sense? Who would go to the effort of adding type hints and then not run the checker?
Just spend some months working in enterprise environments....
With the rise of online CI tools, I've seen it become much more common for Python-based projects to include a lint run as part of their tests and break the build on lint output.
If you are lucky to work on a shop that values CI, yeah.
But those are usually the same ones that take the effort to run the tools anyway.
But those are usually the same ones that take the effort to run the tools anyway.
There is no runtime type checker. There isn't even a static type checker.
Sure there is: http://mypy-lang.org/
[deleted]
I have fond memories of using Python 8-10 years ago, but looking at it today, I'm not sure I see compelling reasons not to just use C# instead.
I can have real static typing if I want it. I can also have dynamic typing if I want that. I can use lambda functions. I have LINQ. I can use ScriptCS or some of the new Roslyn stuff if I want to use it as a scripting language. Unix support is not as great, but they are coming along with that. There's no GIL.
Is there some killer feature for Python that I'm overlooking?
I can have real static typing if I want it. I can also have dynamic typing if I want that. I can use lambda functions. I have LINQ. I can use ScriptCS or some of the new Roslyn stuff if I want to use it as a scripting language. Unix support is not as great, but they are coming along with that. There's no GIL.
Is there some killer feature for Python that I'm overlooking?
Still less boilerplate than C#.
Still on average more concise to express the same idea.
Standard library is a nice mix of function-based vs. class-based solutions where appropriate, rather than forcing OO on you even when it's not the ideal approach.
Cross-platform C# usage is still not even a reality, let alone as good as what Python already has.
In several domains of programming, Python has best-of-breed, and sometimes just flat-out the best, available libraries.
Still on average more concise to express the same idea.
Standard library is a nice mix of function-based vs. class-based solutions where appropriate, rather than forcing OO on you even when it's not the ideal approach.
Cross-platform C# usage is still not even a reality, let alone as good as what Python already has.
In several domains of programming, Python has best-of-breed, and sometimes just flat-out the best, available libraries.
I think its ubiquity is important (installed by default on most non-Windows machines, at least in my experience) combined with the simplicity of running a python script.
Now that Microsoft is supporting non-Windows with .Net core, C# is more attractive. That said, if I was thinking compiled Python with static typing my mind would go to Go.
Lots of options!
Now that Microsoft is supporting non-Windows with .Net core, C# is more attractive. That said, if I was thinking compiled Python with static typing my mind would go to Go.
Lots of options!
The only domains python and C# really overlap is web with ASP.NET and Python web frameworks - and here portability used to be the determining factor.
Python use cases are completely different from C# - Python is all about fast iteration and prototyping, REPL, interfacing to native libraries etc.
C# is about static typing and all that it implies.
F# is slightly closer to Python with REPL and type inference but still on the C# side of spectrum.
Python use cases are completely different from C# - Python is all about fast iteration and prototyping, REPL, interfacing to native libraries etc.
C# is about static typing and all that it implies.
F# is slightly closer to Python with REPL and type inference but still on the C# side of spectrum.
Compared to C#, it doesn't require you to run Windows in production.
Nim has much easier syntax than C#, why not that instead, has better type inference than C# too.
Pythonesque in its core.
I like this. I use Ruby as a scripting language more often than Python and I would be happy to see similar additions to Ruby to improve code readability and IDE support (although RubyMine) does an awesome job of warning of possible code problems just using code inspection).
I sometimes wish that one programming language would work for all of my requirements but I keep needing: scripting languages for quickly getting stuff done; practical languages like Clojure and Java, and type strict languages like Haskell (that I would like to use more, but development takes longer).
Looking ahead a few years, the ideal for me would be a loose scripting language that had awesome static analysis tools. So, for example, I would get the safety of Haskell when hacking in Ruby.
I sometimes wish that one programming language would work for all of my requirements but I keep needing: scripting languages for quickly getting stuff done; practical languages like Clojure and Java, and type strict languages like Haskell (that I would like to use more, but development takes longer).
Looking ahead a few years, the ideal for me would be a loose scripting language that had awesome static analysis tools. So, for example, I would get the safety of Haskell when hacking in Ruby.
It's cool that PyCharm added support for this -- the point of the typing module and the type hinting PEP is for IDEs to adopt it for linting purposes.
For those of you using vim, emacs, and the standard UNIX editing toolchain, the mypy linter provides command-line support. This blog post from April 2015 describes the first version of mypy that includes support for the official typing library.
http://mypy-lang.blogspot.co.uk/2015/04/mypy-02-released.htm...
For those of you using vim, emacs, and the standard UNIX editing toolchain, the mypy linter provides command-line support. This blog post from April 2015 describes the first version of mypy that includes support for the official typing library.
http://mypy-lang.blogspot.co.uk/2015/04/mypy-02-released.htm...
So how do we get some of these benefits if we don't use PyCharm, but use Vim? Are there standalone tools, or VIM plugins?
If you can deal with a different file navigation setup (and outside a terminal obviously), VIM emulation in PyCharm is very good.
Having recently switched, PyCharm really does make larger projects a lot easier to refactor / manage.
Edit: Also, to actually answer your question, this is a relatively new feature so I don't think any of the VIM plugins support it yet. Jedi has had type hinting from doc strings for some time now. I'd imagine they'll update that to use this soon.
Having recently switched, PyCharm really does make larger projects a lot easier to refactor / manage.
Edit: Also, to actually answer your question, this is a relatively new feature so I don't think any of the VIM plugins support it yet. Jedi has had type hinting from doc strings for some time now. I'd imagine they'll update that to use this soon.
Has someone tried GNOME Builder? I found it quite good for C, but the Python mode of PyCharm is still superior regarding search and introspection, yet it is not Free Software…
Finally! without the additional type information, it's almost impossible to do any meaningful refactoring...
Turns out to be backwards compatible in Python3 before 3.5 (though not Python2, yet another reason, if you actually needed one, to start new stuff in Python3)