Ask HN: Python vs Scala for a fast web-project
14 comments
If your goal is to launch, stop fooling around with languages and rewriting your code. What exactly was slow?
1) A NoSQL database best suited our project. So, the non-nativeness was a hurdle. Neo4j bindings for Python used JPype, which was rather unmaintained. And I read at some places that it was a bad idea to mix python-java (especially using the unmaintained JPype).
2) Using Flask gave us so much flexibility as a framework. SQLAlchemy was probably the best thing that I've ever seen. But the work was made complex by the fact that Python was not able to give us the performance that we wanted. We tested the web-project running locally, and we noticed some exciting results. Even though we had done proper memory management from our side, the memory consumption was a little high than we had actually expected. I'm not an expert web-technologies, but as a programmer for last n years, I've learnt a lot about memory management. We projected that when the load would be actual, python performance would suffer.
3) We wanted a real-time performance application so that the user experience was the best. We implemented a part of the web-application and tested it. The results were unsatisfactory. Then we planned on shifting to scala.
2) Using Flask gave us so much flexibility as a framework. SQLAlchemy was probably the best thing that I've ever seen. But the work was made complex by the fact that Python was not able to give us the performance that we wanted. We tested the web-project running locally, and we noticed some exciting results. Even though we had done proper memory management from our side, the memory consumption was a little high than we had actually expected. I'm not an expert web-technologies, but as a programmer for last n years, I've learnt a lot about memory management. We projected that when the load would be actual, python performance would suffer.
3) We wanted a real-time performance application so that the user experience was the best. We implemented a part of the web-application and tested it. The results were unsatisfactory. Then we planned on shifting to scala.
First profile the Python app - is it being slow in Python or is it waiting on something else? If it is being slow in Python, can you rewrite the critical section in C? And speaking of which, is there a C API to your DB that you can use from Python? The Python mentality has always been that it is easy to interface Python and C where necessary for performance.
Did you have any experience doing web development with Scala? I am now choosing stack for my new project and considered Scala/Play as an option. There are some problems both with Play2 framework atm(no Wax deployment support, long rebuilding after small changes, work with actions is confusing) and the language of Scala (complexity, version migration, SBT etc.) The situation is promised to get better in future but most need their work done now. If you are choosing bleeding edge, new shiny thing be ready to face additional problems which can slow your development. If you prefer just to get things done you'd review your Python code, find and fix bottlenecks.
Maybe you can combine techs choosing what is appropriate for some work - say Python for web interface with critical parts in Java/Scala.
No, we didn't have any experience with Scala/Lift/Play. We were aware of the learning curve of scala. But what we thought was in the the best interest of project. I've read about the problems ingoing with Scala, but the benefits outweigh the problems most of the times. And yes, learning a new technology will definitely slow down the development (especially Scala).
Its not the language but your framework and architecture and your own proficiency in the language ( whatever that might be) that matters most for a fast web application.
Your code would spend most of its time fetching stuff from DB, external web services etc and rendering HTML. Does your framework allow you to easily incorporate database caches? Does it allow to to cache frequently rendered HTML segments ? Can you offload certain tasks ( such as sending email etc.) to a secondary process via a job queue ?
You could write a web application in C that could still suck big time unless you get the application architecture right.
Your code would spend most of its time fetching stuff from DB, external web services etc and rendering HTML. Does your framework allow you to easily incorporate database caches? Does it allow to to cache frequently rendered HTML segments ? Can you offload certain tasks ( such as sending email etc.) to a secondary process via a job queue ?
You could write a web application in C that could still suck big time unless you get the application architecture right.
We had integrated redis in our project for server side caching. And I'd like to clarify that Python was being slow for regular cases (atleast not as fast as we wanted it to be).
As we were newbies to web development (although we have a lot of experience in development), we had put in our best knowledge while designing the architecture. We kept independent things independent. That's why we chose flask, it gave us the flexibility we wanted. We were free to choose the extensions we wanted to integrate.
As we were newbies to web development (although we have a lot of experience in development), we had put in our best knowledge while designing the architecture. We kept independent things independent. That's why we chose flask, it gave us the flexibility we wanted. We were free to choose the extensions we wanted to integrate.
So, we had to make a decision to either keep going in Python or port to Scala (Scala seemed apt for many reasons).
What would you have done in this case?