We follow this rule (about two dozen services with in total ~100k loc of Python):
By default, use the version 1 release below the latest.
I.e. we currently run 3.11 and will now schedule work to upgrade to 3.12, which is expected to be more or less trivial for most services.
The rationale is that some of the (direct and transitive) dependencies will take a while to be compatible with the latest release. And waiting roughly a year is both fast enough to not get too much behind, and slow enough to expect that most dependencies have caught up with the latest release.
I'm maintaining an internal change-data-capture application that uses a python library to decode mysql binlog and store the change records as json in the data lake (like Debezium). For our most busiest databases a single Cpython process couldn't process the amount of incoming changes in real time (thousands of events per second). It's not something that can be easily parallelized, as the bulk of the work is happening in the binlog decoding library (https://github.com/julien-duponchelle/python-mysql-replicati...).
So we've made it configurable to run some instances with Pypy - which was able to work through the data in realtime, i.e. without generating a lag in the data stream. The downside of using pypy was increased memory usage (4-8x) - which isn't really a problem. An actually problem that I didn't really track down was that the test suite (running pytest) was taking 2-3 times longer with Pypy than with CPython.
A few months ago I upgraded the system to run with CPython 3.11 and the performance improvements of 10-20% that come with that version now actually allowed us to drop Pypy and only run CPython. Which is more convenient and makes the deployment and configuration less complex.
Usually there's some sort of engineering manager / lead engineer who's in charge of deciding how the product is built from a technical point of view. He's the main collaborator of the product manager/owner and they need to get along well.
Thanks for the writeup - very helpful. It's always good to get a view of how others are solving the same problems oneself has.
That said, the article does come off a bit as trying to be authoritative, but at the same time it doesn't leave enough room for possibilities where alternative approaches may have merit as well (i.e. "this is how to do it" vs. "what worked well for us, ymmv").
Newbies that read this article will think that the principles described are the canonical way and even try and apply them in scenarios where alternatives may prove superior.
We're looking for a skilled frontend developer who will help maintain and extend our angular.js application with focus on maintainability and performance. Bonus points if you're not afraid of touching backend code, too (Python for us).
Backend Developer - Help developing our backend services and daemons (mostly Django and Tornado), work with many Terabytes of data in PostgreSQL.
Python Infrastructure Developer - Your job is to continue automating our development and operations processes. You help built and maintain the tools to develop, deploy and run our applications on AWS.
Javascript/Frontend Developer - You help maintain and extend our frontend application (AngularJS, React) and are not afraid to venture into touching the backend code as well!
Python Infrastructure Developer - Your job is to continue automating our development and operations processes. You help built and maintain the tools to develop, deploy and run our applications on AWS.
Javascript/Frontend Developer - You help maintain and extend our frontend application (AngularJS) and are not afraid to venture into touching the backend code as well!
Backend Developer - Help developing our backend services and daemons (mostly Django and Tornado), work with many Terabytes of data in PostgreSQL.
Python Infrastructure Developer - Your job is to continue automating our development and operations processes. You help built and maintain the tools to develop, deploy and run our applications on AWS.
Javascript/Frontend Developer - You help maintain and extend our AngularJS application and are not afraid to venture into touching the backend code as well!
Backend Developer - Help developing our backend services and daemons (mostly Django and Tornado), work with many Terabytes of data in PostgreSQL.
Python Infrastructure Developer - Your job is to continue automating our development and operations processes. You help built and maintain the tools to develop, deploy and run our applications on AWS.
Javascript/Frontend Developer - You help maintain and extend our AngularJS application and are not afraid to venture into touching the backend code as well!
We're hiring another experienced Python developer to help take care and expand our backend systems. We're mostly working with Django, Tornado, PostgreSQL, AngularJS, AWS.
As an analytics SaaS we have a few Terabytes of data to wrestle with and challenging tasks ahead of us.
Our team is still small and we have multiple roles to fill, depending on your experience and goals: be it full-stack dev, postgres expert or infrastructure dev.
If you enjoy programming in Python, have a knack for RDBMS and have developed and run production systems before then write us a message to [email protected]!
- We are building an innovative SaaS Analytics Tool that consumes millions of data points every day
- We change the offline retail industry by giving them access to tools from the online world
- We are solving challenges on database performance, intelligent data aggregation and interpretation and building scalable APIs
- We rely heavily on Python, PostgreSQL and C as well as Django, RabbitMQ and AngularJS
- We want you as a full member of our international 42reports team and provide you with full access into all development areas: from hardware sensors to backend applications and frontend applications
About you
- You want to develop reliable, high performing backend systems
- You have solid experience developing software in Python
- You know at least one other programming language well: e.g. C, Go or Java
- You have a good understanding of relational database systems
- You have a passion for developing software which actually solves a real problem
- You are curious about new technologies, believe in agile development and have the ambition to learn beyond your current fields of expertise
- Ideally, but not necessarily, you also already have experience processing large amounts of time-series data, developing automatic deployment tools or using machine learning algorithms
Job Perks
- A highly motivated team of embedded, backend and frontend developers as well as mathematicians that you can rely on, learn from and support with your own experiences
- A great company culture with team spirit, a common vision, serious coffee, Friday beer and weekends off
- An environment geared towards learning and professional development to make the most of your potential
- Competitive compensation that matches your abilities and potential
One could argue that software architecture is the inner structure of an application, i.e. the libraries, modules, classes, functions and, most importantly, their interfaces. The GOF design patterns are purely a matter of software architecture, for example.
System architecture on the other hand describes the different applications/services that compromise a system, including the ones that you write yourself, plus the ones that you just "use", e.g. database, storage, load balancer, etc.
I would argue that the article mostly talks about system architecture with focus on storage and I guess the parent you replied to would too.
We're always looking for talented software engineers who join us to develop our backend/data heavy SaaS in the marketing/adwords area.
We're a small team (~8 devs) working mostly with Python and big Postgresql deployments, as well as quite a significant amount of javascript in the frontend.
We offer a friendly atmosphere, competitive pay, competent co-workers, no overtime, in the center of Berlin!
We'd love to hear from you!
We're always looking for talented software engineers who join us to develop our backend/data heavy SaaS in the marketing/adwords area.
We're a small team (~8 devs) working mostly with Python and big Postgresql deployments, as well as quite a significant amount of javascript in the frontend.
We offer a friendly atmosphere, competitive pay, competent co-workers, no overtime, in the center of Berlin!
We're always looking for talented software engineers who join us to develop our backend/data heavy SaaS in the marketing/adwords area.
We're a small team (~8 devs) working mostly with Python and big Postgresql deployments, as well as quite a significant amount of javascript in the frontend.
We offer a friendly atmosphere, competitive pay, competent co-workers, no overtime, in the center of Berlin!
I.e. we currently run 3.11 and will now schedule work to upgrade to 3.12, which is expected to be more or less trivial for most services.
The rationale is that some of the (direct and transitive) dependencies will take a while to be compatible with the latest release. And waiting roughly a year is both fast enough to not get too much behind, and slow enough to expect that most dependencies have caught up with the latest release.