I'm a random dude on the Internet, but my partner completed her PhD at MIT. While there I knew and knew of a few PhD grads who worked at MIT in some non-tenure-track role (postdoc, staff researcher, etc). Typically for a couple years and then they get a better-paying or more permanent job. But several remained "affiliated" in some way. They kept their MIT website/email, some in academia continued to collaborate to some extent. Things like that. But AFAIK they weren't getting a paycheck from MIT. And it's somewhere between neat and genuinely professionally valuable to be affiliated w/ a prestigious university, so I don't blame them for claiming affiliation. My best guess is he's "affiliated" in a similar way.
> Right now, accessing my apps requires typing in the IP address of my machine (or Tailscale address) together with the app’s port number.
You might try running Nginx as an application, and configure it as a reverse proxy to the other apps. In your router config you can setup foo.home and bar.home to point to the Nginx IP address. And then the Nginx config tells it to redirect foo.home to IP:8080 and bar.home to IP:9090. That's not a thorough explanation but I'm sure you can plug this into an LLM and it'll spell it out for you.
Maybe a more accurate take: Half-assed soft deletion definitely isn't worth it.
If you're just going to throw in some deleted bool or deleted_at timestamp without thorough testing, you might as well just skip it. It's virtually certain to go wrong.
Maybe out of touch. I've used them all (Mint, ynab, pocketsmith, buxfer, some google-sheets-based app, probably tried a dozen others). All of the auto-sync features had obvious bugs. It works for a few months and then you end up with junk data. But if it works for you, carry on!
I guess it comes down to the tradeoffs you're willing to accept for privacy. I personally find it quite sketchy that Plaid takes your username and password to your literal money and then does some opaque screen-scraping just to grab your transactions. I also spent a lot of time working with their API and it did not inspire confidence.
I've been using firefly over two years now. Data entry has not been been a problem:
* Run the data-importer as a separate container. Takes maybe 20 minutes to configure correctly if you already know docker and docker-compose.
* Download the transactions as CSVs from each of my 5 or so credit card and bank accounts that get regular use.
* Upload them through the data-importer. It lets you configure and save settings for each CSV format. I took five minutes to set that up the first time I imported CSVs and just keeping using the same settings.
I upload all of my transactions once a month and it takes about an hour to download them, import them, and categorize all of them (I also have a bunch of rules to auto-categorize, but there are inevitably a dozen or so bespoke transactions).
I've found that any of the solutions built on top of syncing backends like Plaid inevitably have issues: duplicates, missing txs, debit/credit mixed up. I even built my own custom Plaid-to-Firefly syncer at one point and found the data quality was very mixed, even when all my accounts are at major US banks. The data-importer takes some more up-front work, but it's more secure, way more predictable, and generally a solved problem.
Sadly he's not some kind of local hero. The area much prefers sports to academics. I studied CS there for undergrad and knew of him but rarely heard his name mentioned or celebrated.
...What? $10 x 1000 = $10k / month. $10k x 12 = $120k. That is a new grad software engineer salary in any US city. You'd pay more than that for a single dev with the devops and security experience to keep GHE running and patched for 1000 devs.
I've personally had some painful experiences with refreshing materialized views in Postgres. In particular, highly variable performance on read replicas that were receiving a refreshed matview every few minutes. Maybe we were just doing it wrong, but I tend to avoid it if I can. Plus the eventual consistency can introduce confusion.
This would let us define a materialized view that gets automatically updated as the source tables change. When I last checked (late 2021), they were saying it might land in PG15.
Thanks for the feedback -- sorry I missed this a couple days ago! Had no clue it got posted to HN.
FWIW, I've found GIN is a bit faster if you're just looking to filter. IIRC it was maybe 10-15% faster for the particular use-case I was looking at. So, worth a try, but don't expect a 10x improvement.
Thanks for the feedback -- sorry I missed this a couple days ago! Had no clue it got posted to HN.
I've deployed a solution that uses roughly this same method with multiple tables. I experimented with a materialized view that would centralize all the text columns, but ultimately found that it was much simpler and fast enough to have a single expression index in each of the tables. Then the query either joins the tables and checks each column, or you can run a separate query for each of the tables and stitch together the results in app code.
> I think maybe one reason people shy away from things like Presto (and the above) is the uneven performance guarantees; waiting for an unoptimized Hadoop or Orcfile query by accident because you joined on something or another is fine for one-offs, but might become costly in prod workflows.
Right, so my question is: how is that solved with Hydra? Seems like you'd arrive at the same issue?
Looks neat, but wasn't this the promise of Presto? Presto didn't seem to really work out. From what I've seen it converged to a mostly analytical engine. It's still very useful, but I've never seen it used (successfully) in an OLTP workload. Maybe there's some difference in the intended product trajectory that I'm overlooking here?
Does anyone know if there is a way to monitor sudden changes in planning behavior for a given query? For example, I'd like to monitor/alert on this kind of situation: I'm executing the same query for many weeks/months, some table involved in the query is slowly growing over time, at some point the query planner responds to the growth by changing the plan.
Hi HN! Elastiknn is an Elasticsearch plugin that lets users store numerical vectors within documents and run exact and approximate nearest neighbor queries on them. It supports dense and sparse vectors with five similarity functions (L1, L2, Angular, Jaccard, Hamming).
It's a hobby project of mine, based on an application where I needed to to continuously update an index of vectors for approximate searches. Very few of the existing approximate search libraries satisfy this use-case; most require re-building the index from scratch.
Elastiknn lets you add/update/delete vectors without rebuilding the index and combine vector queries with existing Elasticsearch queries, all through the standard JSON/HTTP API, all in your existing cluster. You just specify a field type of `elastiknn_sparse_bool_vector` or `elastiknn_dense_float_vector`, specify some optional settings for approximate indexing, index vectors as JSON lists, and then use the `elastiknn_nearest_neighbors` query in your search requests.
Nothing is without tradeoffs. In terms of raw throughput, Elastiknn is slower than many existing solutions. If you need a high performance solution for offline batch jobs, it's likely not for you. There are some benchmark results at https://elastiknn.com/performance/.
My current goals for the project are: integrating with the ann-benchmarks project, releasing builds for multiple Elasticsearch versions, and creating some example applications.
All feedback is appreciated. At this point there are several teams evaluating the plugin. There's a pretty thorough test suite, but nothing beats real-world usage and feedback. Cheers!
The Lucene implementation seems early and slow-moving. Seems they are trying to create new storage formats and use graph-based search methods. OpenDistro wrapped a C++ binary that also uses a graph-based method. It works quite well, but only for L2 similarity and comes with the operational burden of running a rather large sidecar process completely disjoint from the JVM.
The approach I've taken is to support five similarity functions (L1, L2, Angular, Jaccard, Hamming), support sparse and dense vectors, implement everything inside the JVM with no sidecar processes and no changes to Lucene, and to use hashing-based search methods (i.e. LSH). IMO the last point has a clear advantage over using graph-based methods, because the hashes are treated just like regular text tokens which is clearly the optimal access pattern in ES/Lucene. Of course it will likely lose to a C++ implementation in terms of raw speed because it's the JVM, but IMO that matters less than making the plugin trivial to run and scale.
I don't think there's a definitively better approach yet. It's an interesting problem and it'll be interesting to see what ends up working well.