This is not a fly.io specific question but I always wonder how in these globally distributed system, how are databases handled?
I understand you can put your application server in any location but generally there is only one storage so are these application servers doing cross region database calls?
Having only worked with single cluster setup web apps, I am always curious about this part.
Is the answer always - use a managed replicated database and send read queries to one near your location and all write queries goes to the primary instance?
Thanks. My initial question arose where does products like SnowFlake/RedShift falls into this. They seem to be coined as datawarehouse but seems they do the same as OLAP work. But looking at Snowflake docs I read:
"""Currently there is no option similar to an SSAS cube in Snowflake. Once data is loaded into the databases Snowflake allows to query the data in a way similar to traditional OLTP databases.
For data aggregations the Snowflake library offers rich sets of in-built functions. Together with UDFs, SPs and Materialized Views we can build custom solutions for precomputed data aggregations.
For data analysis we still have to rely upon third party tools. Snowflake provides a variety of different connectors to access its database objects from other analytical tools. There are plans in near future to introduce an integrated tool for data aggregations, analysis and reporting."""
So they say, its not really an OLAP.
And how exactly it differs from OLTP? Is a web app OLTP?
>>> Weird. OK, I tried it with some other parameters, and it suddenly took 15 seconds at 100% CPU, with PostgreSQL doing some linear scan through data. Using EXPLAIN I found that with full production data the query planner was doing something idiotic in some cases. I learned how to impact the query planner, and then this query went back to taking only a few milliseconds for any input. With this one change to influence the query planner (to actually always use an index I had properly made), things became dramatically faster. Basically the load on the database server went from 100% to well under >>> 5%.
I am actually interested in this part. Figuring out issues with EXPLAIN is one of my favorite things.
This is so cool. I am actually planning to start working on something similar but for wet shaving market. Basically, I was trying to setup something where user can choose their after shave pairing based on scent details.
Slightly off-topic but is there any book or resource that a software engineer like me can read to understand the basics of how VC operate, what each round of funding mean, what one needs to know when one approaches a VC, difference between VC/LP?
Niara | Full Time | Immigration Sponsorhip Available | San Francisco Bay Area
Niara is an early stage venture startup hiring core technology engineers and UI/UX/Product Designers in
Sunnyvale, USA to build its next generation security intelligence platform. Check us out at: http://www.niara.com. You can apply through the website or send a mail to me at riteshn AT gmail
Full Stack Engineer (Member of Technical Staff)
This is the job for you if you like to work on every aspect of a product. As Niara's, MTS, you will be working on the frontend, middleware, and backend. You would work with frameworks like AngularJS backed by a Python backend.
We prefer simple, elegant solution to complex problems, so bonus points for having a history of getting things done.
UI/UX, Product Designer
As a UI/UX designer, your defined user experience is critical to Niara's success. You will work on concepts, wireframes, story boards to final user interface. Your solutions will help visualize highly technical problems in an easy to understand manner for the customers.
I have been looking into building simple recommendation engine (i have at max million data rows) using Python. I looked into Crab (https://github.com/muricoca/crab) and it seems to not have updated for 2 years.
Any suggestions for libraries or just use basic numpy/scipy and implement the algorithms?
I cannot seem to figure out how an Indian startup has mostly foreign artists. I am asking because couple of years back, I tried to do the same thing and licensing was the biggest issue. Have you got funding in India?
Moreover, most of the artists and songs I see up there are just at bandcamp.com and no where else. Are you just scrapping Bandcamp and reselling their music. E.g. googling for the album "Creatures" on homepage (https://www.google.com/search?q=creatures+new+campaigns&oq=c...) loads up only from Bancamp and other albums too.
There are examples for CakePHP and Flask but none for Rails. Do you know of any? I have a simple traditional Rails which I am looking into porting to use Backbone and Rails as backend.
Oh wow. I have been thinking of exactly the same idea. To that end, I just started playing around with Raspberry PI to see if it can work out. Looking online, seems they also have color E-Ink available which should make the cost even less.
I understand you can put your application server in any location but generally there is only one storage so are these application servers doing cross region database calls?
Having only worked with single cluster setup web apps, I am always curious about this part.
Is the answer always - use a managed replicated database and send read queries to one near your location and all write queries goes to the primary instance?