I ended up working for 2 years before I went to school, and it made a lot of things easier. When I graduated I was very certain that I wanted to go back to industry rather than go to academia. Definitely different jobs and skillsets.
Separately, there seems to be a ton of unused or broken or dead code sprinkled throughout — for instance, in the auth code, I can't tell if you're doing basic email/password auth or using Supertokens and a third-party login via Google. You have code for both and some routes seem dead or missing.
Also, I mentioned the lack of documentation for how to run Tegon locally because your docs are entirely insufficient. The main docs page is just a README template. https://github.com/tegonhq/tegon/tree/main/docs
Extremely anodyne — given that the author has maintained FreeDOS for longer than I've been alive, I was hoping for slightly more interesting ideas than "have a website and be nice to people." Not that I disagree with his advice!
Looks totally fine at first glance. I'm happily using https://github.com/gorilla/sessions to handle cookies right now — OP, is there any reason I should want to switch to your library instead?
Aside — the architecture diagrams on https://xeiaso.net/blog/xesite-v4/ are not displaying in-browser for me, but do transfer correctly and are viewable on my computer. Maybe another mime issue, as the network inspector shows them being transferred with type "octet-stream"?
Repro by visiting the URL in latest Firefox, Safari, or Chrome.
Maybe I missed it, but why are you embedding a zipfile of gzip streams rather than just embedding the static site's files with an `embed.FS` and serving from that?
I think a lot of new entrants to previously-well-explored spaces tend to label themselves as "modern" in order to signify that there is something new about their approach.
For instance, I have a migrations library that I descibre as "modern" because it is designed for a continuous deployment environment where you're automatically running migrations on container startup — there are tons of existing popular migrations libraries, but none of them work this way because they were written in the era that you'd manually run sql commands in prod. I say "modern" so that if anyone finds my library, they realize that it was created recently based on more recent dev/ops trends.
Maybe I should drop the "modern"? I do see a lot of people describe their code as "minimal" or "clean", which is pretty meaningless to me, so I get that "modern" could come across that way as well.
Maybe. Frankly this is a very impressive outcome that is attributable both to the Father's business and management practices, as well as the Son's business and management practices. No matter how good you are at running a web business, if you inherit a totally fucked business with poor practices, very unlikely you could engineer this outcome.
My condolences to the OP and I was happy to read such a well-written article describing a fantastic outcome for his father's company.
Enforce squash merging to main and move on with your life. Linear history on main, individual contributors can rebase and merge or format-patch or do whatever they want on their PR branches and it doesn’t matter. There are zero downsides to this approach.
One more thing you could do is put the real name of any human being with any track record of professionalism, anywhere on the website. Currently you're:
- commenting under a pseudonymous profile
- asking for emails by saying "please email me. contact at cyberscarecrow.com"
- describing yourself in your FAQ entry for "Who are you?" by writing "We are cyber security researchers, living in the UK. We built cyber scarecrow to run on our own computers and decided to share it for others to use it too."
I frequently use pseudonymous profiles for various things but they are NOT a good way to establish trust.
Has anyone here used this, or anything similar? This sounds phenomenal if it really works. Looks like “contact us” is the only way to try it or buy it right now, and the purported benefit (memorize facts up to the model training size, basically trillions of tokens of facts) is wild. I’d love to try a system running this way to understand the failure modes, like for instance how does it reliably infer which “facts” to use?
I have immense respect for the OP for writing up the story, and even more so for giving this preface. It's really useful to know what mistakes other people make, but can be quite embarrassing to tell others about mistakes you've made. Thanks, OP.
> JVector, the library that powers DataStax Astra vector search, now supports indexing larger-than-memory datasets by performing construction-related searches with compressed vectors. This means that the edge lists need to fit in memory, but the uncompressed vectors do not, which gives us enough headroom to index Wikipedia-en on a laptop.
It's interesting to note that JVector accomplishes this differently than how DiskANN described doing it. My understanding (based on the links below, but I didn't read the full diff in #244) is that JVector will incrementally compress the vectors it is using to construct the index; whereas DiskANN described partitioning the vectors into subsets small enough that indexes can be built in-memory using uncompressed vectors, building those indexes independently, and then merging the results into one larger index.
OP, have you done any quality comparisons between an index built with JVector using the PQ approach (small RAM machine) vs. an index built with JVector using the raw vectors during construction (big RAM machine)? I'd be curious to understand what this technique's impact is on the final search results.
I'd also be interested to know if any other vector stores support building indexes in limited memory using the partition-then-merge approach described by DiskANN.
Finally, it's been a while since I looked at this stuff, so if I mis-wrote or mis-understood please correct me!