We recently completed the migration of a self-hosted 3 TB PostgreSQL database from version 12 to 16, transitioning from Ubuntu 18 to Ubuntu 22. Concurrently, we had to upgrade various extensions, most notably Timescale, for which a compatible version did not exist across all scenarios. We performed the upgrade by updating a replica in the following sequence:
- Start: PG12, Ubuntu 18, TS2.9
- Step 1: Set up a read-only replica with PG12 on Ubuntu 22, maintaining TS2.9.
- Step 1.5: Enter maintenance mode and halt all services.
- Step 2: Detach the the read-only replica, upgrading from PG12 to PG15 on Ubuntu 22 with TS2.9.
- Step 3: Upgrade from PG15 with TS2.9 to TS2.13 on Ubuntu 22.
- Step 4: Upgrade from PG15 to PG16 on Ubuntu 22 with TS2.13.
- Step 4.5 : Reconnect services to the new database server, resume all services, and exit maintenance mode.
All the database upgrade steps were well-tested and automated using Ansible. Nonetheless, we did encounter an issue that had not arisen during testing. This extended our downtime to approximately half an hour, which, for our use case, was perfectly acceptable.
Employing logical replication could have mitigated the last-minute surprise. So we will consider this approach for our next upgrade cycle.
We are looking for a Backend developer with a 5+ years of relevant experience. Our stack is Python, Django, PostgreSQL, TimescaleDB, RabbitMQ. We use Docker, Ansible, and more.
Sentinel is the maker of the "App for boats". We help make the ownership experience of the boat easier with telematics, by keeping you connected to your boat, giving you at-a-glace overview of vital boat equipment, and helping you with boat maintenance with reminders. We also connect the boat owner with the dealer and the broader ecosystem of partners relevant for boat ownership. Our products are a keystone piece of equipment on boats from the largest boat builders worldwide. We are still a small team (20 people), and are making waves in the boating industry. Your contributions will have a huge and direct on the boating experience of current and new boats.
This is a position requiring at least partial onsite presence, so it's local to Ljubljana and nearby areas. This is to facilitate working with hardware/firmware engineers when necessary, and mentoring less experienced developers.
We are experiencing very high CPU load caused by tinc [0], which we use to ensure all communication between cloud VMs is encrypted. This is primarily affecting the highest traffic VMs, including the one hosting the master DB.
I am starting to consider alternative tools such us wireguard to reduce load, but I am concerned of adding too much complexity. Tinc's mesh network makes setup and maintenance easy. The wireguard ecosystem seems to be growing very quickly, and it's possible to find tools that aim to simplify its deployment, but it's hard to see which of these tools are here to stay, and which will be replaced in a few months.
What is the best practice, in 2021, to ensure all communication between cloud VMs (even in a private network) is encrypted?
This sounds great. Instead of trusting each company to implement their own billing system with some payment processor, just have the payment processors and banks provide "apps" that support these protocols, and users can chose which one to use to complete their payment.
One interesting follow up question in who covers the processing fee (e.g. the ~3% or so that Braintree or Paypal currently take)? Today, processing fees are mostly hidden to the user. If the user can chose their own payment processor, will they be required to cover the processing fee instead of the merchant?
I have been observing something similar recently, but related to other technologies. People claiming they are good at Excel, but can't do a vlookup or a pivot table. People using 5% of the power of an IDE. People not realizing that if they are able to use a web-based tool at work, they could (probably) also access it remotely.
And these are just the things I have noticed in the past week.
We utilize just a tiny percentage of options that are available to us, barely enough to get the basics of out tasks done. But we are completely oblivious that there are probably better/hidden/power tools available in the software we are already using that could help us achieve our tasks much more efficiently.
Interestingly, most people probably don't care about it, or are so oblivious of their lack of knowledge that they don't even search for better ways to do things.
It's hard to blame thought. Improving this state requires a mentality of continuous active learning, where you don't just wait for someone to show you how to do your task better, but to constantly expand your knowledge into areas you don't even think you need to improve. However, most of us usually have "better things to do" than reading software manuals.
2 months ago I discovered Kepler[0] a tool by Uber for visualizing geo data. It has been of tremendous help when analyzing large geospatial datasets. It allows me to quickly visualize the results of my computations, and spot anomalies and patterns that I wouldn't have noted without it.
This seems to be a similar tool, but for charting non-geographic datasets. It should also be extremely useful.
I am glad companies are making these tools available for the masses.
The value Twilio's offering is when you need coverage also outside Europe, in remote countries. Yes, you can get cheaper data plans to cover the EU, but those often come with very high data rates in other areas.
I have just completed a first project with OpenShot. It worked perfectly fine initially, but as the movie became more complex, everything took much longer. Saving took several seconds, sometimes freezing everything for several minutes. Dragging multiple clips in the timeline would freeze the editor for several seconds. It crashed multiple times. Just to be clear, this was from having about 30 original clips, but cut multiple times (each probably at least 10 times), so over 300 individual clips in the timeline. The bottleneck seems to be related to parsing of the JSON, which is the format in which the data is encoded internally. The final file was about 1.1MB of JSON. I've also encountered strange artifacts when adding a new layer at the bottom of the screen - I couldn't drag clips onto it, they were placed onto another layer instead.
This is on a laptop with an i7 and 12 GB of RAM.
Compared to my experience of using Premiere Pro several years ago on Windows, with just 4GBs of RAM, OpenShot has a long way to go even for basic editing.
But, having said that (and considering that Premier had years of paid development), it's a good tool and it's great that some developers are trying to built video editing tools for Linux. Things are slowly improving.
I agree, but for different reasons. I have seen a lot of code start with variables like `divisible_by_three`, only to have the requirements change to check division by 4, but programmers being too lazy to update the variable name as well.
I have been making https://www.klubi.si/ sporadically over the past many months (it was meant to be a weekend project, but it never ends there...). It's a map of sports clubs in my country.
The idea is to help people find klubs nearby, as well as provide a basic internet presence to those klubs whore founders don't have the time or knowledge to create and maintain a website, or even a Facebook page.
The website works by volunteer revision of data, as well as twice a year email reminders to founders to review the data. This helps ensure that the data is the most up to date as possible.
Unfortunately, not all klubs even have a public working email address, so thinking of it, I could probably do something about as well.
Think about the problems in your neighborhood and try to imagine creative solutions to them that you can implement with your skillset. It doesn't need to be a global problem to be worthwhile solving - sometimes helping our just a handful of people can feel extremely worthwhile.
This is how I got involved with a personal project. A few years ago I had trouble finding a listing of all Karate dojos in my vicinity. I compiled a list of such dojos in Excel, and later decided to put them online [0]. By putting this dataset online, and maintaining it up to date I am helping other people who are looking for a new place to train a sport.
This project not only helps the community (who can now easily find karate and other sports clubs easily through the project website) but also helped me learn new technologies and practice skill that I don't excel at, like design, writing, marketing, etc. Just yesterday I took the project to the next level and started doing same simple data analysis of the dataset [1].
While the project may not have a very large impact on the community, it does sufficiently so to feel worthwhile. Moreover, being able to play with different skills in a low stress environment makes is very enjoyable.
1. Last year I built a sports' klub listing website [1] which shows where you can train different sports on a map in Slovenia. The most challenging part is keeping the data up to date. This year, it's going to be about automating it fully, such that it can operate with almost no supervision, by means for crawling and automatic emailing klub owners to confirm the validity of the data.
2. Something based on hardware -- still thinking what to do with my Raspeberry 3.
The problem is knowing what you are getting into. With big accelerators, like YC, there are tons of resources online and you can get an idea quite easily. For other, it is much harder. Although their websites generally have some information, it's the founders stories that provide the most value. Somehow, it seems that there is more accelerators than content written by founders about their experiences.
The blog Post rings a tune with me. I've started organizing all the work related projects into Trello boards,and have iterated quite a bit on my workflow.
Next to a board for scrum, I also have a board reserved for planning work things to do in the long term future, like a backlog, but coverings many different aspects of company mostly related to software dev, but also some hardware and marketing.
Eventually I wanted to be able to measure how long would it take to complete some tasks, and prioritize them across lists.
The way I solved this way with a plugin for sublime text, the editor I use all day long [0]. The plugin reads metadata I add to each task on Trello like (backend 1d design 4h) and uses it to compute a schedule for all my tasks. It is also able to prioritize between posts, which in in my case are fucus areas.
When I need to prepare the next sprint, all I have to to is use the plugin to tell me which tasks have the highest priorities, and occasionally review the Planning board.
The plugin was initially developed for Trello, but I found it really useful also for offline task scheduling between different projects.
You can read more about how I use it on my blog [1].
Fully agree. Recently, I have discovered a new reason to use Ember. I have an Ember app I built, but only have a few hours a month to maintain it. Using the Ember stack makes it trivial for me to quickly (and almost painlessly) upgrade as needed, and focus the little time I have on building features. Reading this article made me cringe at the thought of using the proposed stack for this app, having to follow the development of each of these components separately, and fixing the glue between them when things go sour.
Ember gives me confidence, that I can keep maintaining the app with very low effort, especially now, that the Ember community is releasing a LTS version.
I use the free Heroku plan for the backend. Given I am limiting myself to just 1 country, the sleeping rules don't bother me, and the 14k row limit is good enough for now.
So the cost comes from serving the JS and CSS files from S3. And that's 0.30 per month
- Start: PG12, Ubuntu 18, TS2.9
- Step 1: Set up a read-only replica with PG12 on Ubuntu 22, maintaining TS2.9.
- Step 1.5: Enter maintenance mode and halt all services.
- Step 2: Detach the the read-only replica, upgrading from PG12 to PG15 on Ubuntu 22 with TS2.9.
- Step 3: Upgrade from PG15 with TS2.9 to TS2.13 on Ubuntu 22.
- Step 4: Upgrade from PG15 to PG16 on Ubuntu 22 with TS2.13.
- Step 4.5 : Reconnect services to the new database server, resume all services, and exit maintenance mode.
All the database upgrade steps were well-tested and automated using Ansible. Nonetheless, we did encounter an issue that had not arisen during testing. This extended our downtime to approximately half an hour, which, for our use case, was perfectly acceptable.
Employing logical replication could have mitigated the last-minute surprise. So we will consider this approach for our next upgrade cycle.