Kinda related - our product, rowzero.io, is a browser-based spreadsheet with a 2 billion row limit. We initially built the client as anyone would, using a div per cell. We tried to use an off-screen div to take advantage of the browser's native scrollbars but ran into document height limits. Firefox's was 6M pixels iirc. The solution was to do rendering in canvas and draw the scrollbars ourselves.
To measure performance the author looked at latency, but most S3 workloads are throughput oriented. The magic of S3 is that it's cheap because it's built on spinning HDDs, which are slow and unreliable individually, but when you have millions of them, you can mask the tail and deliver multi TBs/sec of throughput.
It's misleading to look at S3 as a CDN. It's fine for that, but it's real strength is backing the world's data lakes and cloud data warehouses. Those workloads have a lot of data that's often cold, but S3 can deliver massive throughout when you need it. R2 can't do that, and as far as I can tell, isn't trying to.
You should check out Row Zero (https://rowzero.io). We launched on HN earlier this year. Our CSV handling is the best on the market.
You can import multi GB csvs, we auto infer your format, and land your data in a full-featured spreadsheet that supports filter, sort, ctrl-F, sharing, graphs, the full Excel formula language, native Python, and export to Postgres, Snowflake, and Databricks.
I've been working on a better spreadsheet for a while now. https://rowzero.io is 1000x faster spreadsheet than Excel/Google Sheets. It looks and feels like those products but can open multi GB data sets, supports Python natively, and can also connect directly to Snowflake/Databricks/Redshift.
My apologies, that came off harsher than I intended. I've used xlwings in previous jobs to complete Excel automation tasks, so thank you for building it. xlwings is one of the projects that motivated me to start Row Zero. My main issue with it, and other Excel add-ins, is they break the promise of an .xlsx file as a self-contained virtual machine of code and data. I can no longer just send the .xlsx file - I need the recipient to install (e.g.) Python first. This makes collaboration a nightmare.
I wanted a spreadsheet interface, which my business partners need, but with a way for power users (me) to do more complicated stuff in Python instead of VBA.
To borrow your phrasing, our thesis is that it has to be Excel-compatible spreadsheet + something, not necessarily Excel + something. It's early days for us, but we've seen a couple publicly traded companies switch off Excel to Row Zero to eliminate the security risks that come with Excel's desktop model.
We built our spreadsheet (https://rowzero.io) from the ground up to integrate natively with Python. Bolting it on like Microsoft did, or as an add in like xlwings, just feels second class. To make it first class, we had to solve three hard problems:
1. Sandboxing and dependencies. Python is extremely unsafe to share, so you need to sandbox execution. There's also the environment/package management problem (does the user you're sharing your workbook with have the same version of pandas as you?). We run workbooks in the cloud to solve both of these.
2. The type system. You need a way to natively interop between Excel's type system and Python's much richer type system. The problem with Excel is there are only two types - numbers and strings. Even dates are just numbers in Excel. Python has rich types like pandas Dataframes, lists, and dictionaries, which Excel can't represent natively. We solved this in a similar way to how Typescript evolved Javascript. We support the Excel formula language and all of its types and also added support for lists, dictionaries, structs, and dataframes.
3. Performance. Our goal was to build a spreadsheet 1000x faster than Excel. Early on we used Python as our formula language but were constantly fighting the GIL and slow interpreter performance. Instead we implemented the spreadsheet engine in Rust as a columnar engine and seamlessly marshal Python types to the spreadsheet type system and back.
It's the hardest systems problem our team's ever worked on. Previously we wrote the S3 file system, so it's not like this was our first rodeo. There's just a ton of details you need to get right to make it feel seamless.
I think calling out Durability is a bit of a straw man. Most services get their durability from S3 or some other managed database service. So they're really only making the "do it on a beefy machine argument" for the stateless portion of their service.
I agree with the other points for production services with the caveat that many workloads don't need all of those. Internal workloads or batch data processing use cases often don't need 4 9's of availability and can be done more simply and cheaply on a chonky EC2 instance.
The last point is part of our thesis for https://rowzero.io. You can vertically scale data analysis workloads way further than most people expect.
It looks and feels like Google Sheets and scales to 1B+ row data sets. We natively support Python and Parquet, and connect directly to Postgres, Snowflake, Databricks, Redshift, and S3.
99.9%+ of data sets fit on an SSD, and 99%+ fit in memory. [1]
This is the thesis for https://rowzero.io. We provide a real spreadsheet interface on top of these data sets, which gives you all the richness and interactivity that affords.
In the rare cases you need more than that, you can hire engineers. The rest of the time, a spreadsheet is all you need.
Shameless plug: If you have bigger data sets, check out https://rowzero.io
We scale up to hundreds of millions of rows and have native Python support.
You can define functions in Python and call them as formulas from any spreadsheet cell. We seamlessly marshal Pandas dataframes from Python land to spreadsheet land and back. [1]
We're also hosted and support real time collaboration like Google Sheets. We reimplemented the Excel formula language. We connect directly to Postgres, S3, Snowflake, Redshift, and Databricks. And the first workbook is free.
You should try http://rowzero.io. We connect directly to DBs and data warehouses, support Python natively, and scale up to hundreds of millions of rows.
Kudos on the technical achievement. We considered the thick client approach you're doing, and one of the reasons we punted was because it was so hard.
One really nice thing about your approach is it minimizes infrastructure cost. That positions you well for embedding use cases, like New York Times visualizations, that we struggle to do economically.
Shameless plug: If you have bigger data sets, check out rowzero.io.
We implemented something like PySheets initially where the formula language was full Python. But we found the Python interpreter to be the bottleneck during (e.g.) large CSV imports, and the GIL prevented parallelizing evaluation. It was also harder for business users to adopt due to small syntactic differences between Python and the Excel formula language.
So we implemented the spreadsheet engine and formula language in Rust. We have a Python code window that allows you to write arbitrary Python functions. Those functions can be called as formulas from any spreadsheet cell. We seamlessly marshall Pandas dataframes from Python land to spreadsheet land and back. It gives you 90% of the benefits of pure Python without compromising on performance.
(Not posix compliant because it doesn't need to be.)