I’ve been really impressed with OpenAI embeddings + vector searches for some full document searches compared to something like standard elasticsearch over a large body of text. Something I think I’m personally missing from the ChatGPT/GPT4/LLM conversation with regards to information retrieval is nested/graph hierarchies.
An example from a previous job where we used a hand tooled NLP system was querying for doctors/dentists/optometrists and being able to take something like “dentist near me who is available in the afternoons and speaks spanish”. We would parse this user query into a few different queries that would run against a search cluster and database to return the filtered result set, or the closest output.
What would be the ideal way to prepare or tokenize this data for querying with an LLM? It’s partially text (match dentist, speak Spanish), partially geographic (near me, doing a geo radius of N miles from providers location, and part filtering (who meets all those criteria and has availability in a time frame). Is this a use case for large token sizes to be able to take in all possible providers? Or parsing a query more easily from human language -> SQL/other data store query language? Or perhaps figuring out another way to encode this data?
Two different streams that have built up over time. I spend less than 10 hours combined weekly on them averaged over a month if i had to guess but both were independent full time jobs at one point
1) ($$m total, $$k/mo now) Server hosting! This started off as hosting game servers for friends in early high school. It expanded to friends of friends for awhile before I pivoted more towards crypto mining (yes I know, young kid with dubious ambitions). Lots of my first software experience was here. I wrote some software for switching the processes between mining/game hosting across the different boxes (raw processes on servers, the horror). Nowadays it’s winding down as just game hosting and some scientific computing rental to a few universities/their robotics clubs. It’s still slightly profitable but I have no interest in updating servers (CPUs from 2013-2015, GPUs mostly re-sold except for what a few people requested) and it runs everything via containers now.
2) ($$k/mo) Sports film review. I wrote the first version of this my first year of college which was a way to keep the stats book for basketball and football games and stitch the actions with the video footage. We had customers throughout ~20 states primarily high school but some colleges as well. In fact it still runs at a lot of them, but I’m not really connected with managing of it anymore. My co-founder still runs it and we rotate a few students from our alma-matter in as interns and occasionally juniors on it.
This became the basis for an esports version of the software that I created a few years ago. This time with CV to do all the gathering of stats and allowing for jumping around in videos and analyzing overall stats from the output. This started in Call of Duty for their then new professional league but expanded out to Halo, Rocket League and Valorant since. I still do some occasional retraining of the models but the product itself got acqui-hired by a larger company for which I still “consult”
EDIT: I’ve also had many more that cost me more money than they ever made, but I’m a big proponent of failing fast and iterating
IMO, the release of Warzone 1, Call of Duty’s Battle Royale mode that released a few months after Modern Warfare 2019, has completely changed the priorities and incentives for the company. The goal is to sell in game cosmetics more so than the game itself, as many other games are turning to as well.
The yearly release cycle, which may be ending soon, leads to bugs that re-emerge each year and features that appear and disappear. Sure, the different studios which produce the games need some room to innovate, but the inconsistent base set of features is incredibly frustrating. CoD games are one of the games I play the most, with the other being a game which is the complete opposite, Old School RuneScape, that has been built on for ~20 years.
I play the current game MWII with friends a few hours a day most days of the week. Multiple times per session my game crashes at random, something I can’t remember with any other major games with a top of the line PC. Like many other pieces of software, chasing other revenue sources seems to have made the quality of the product take a nose dive, with consequences yet to be seen. This is disappointing to me as someone who enjoys playing the game with friends, who has competed in open-bracket events at major tournaments over a span of a few years, and worked directly with the professional league and teams (CDL and CWL) for analytics and software.
I would love to see something akin to an M-based Mac as a dev board of sorts, but I’m fairly sure it would never happen. My home server has transitioned over time from various 4u rack mount options, to a 2u and an M1 Mac Mini, to a Mac Studio with a disk enclosure.
I even tried using a Snapdragon phone dev board for awhile for something more than an RPi
I switched jobs recently and became the defacto DevOps person so have been able to deploy mostly how I want. I’ve used kubernetes at multiple jobs, side projects and at home but for a cost and time constrained startup we are leveraging ECS/Lambda/Batch/Cloudfront. B2C application, mostly low traffic with nearly no traffic off hours. Occasionally we’ll get a big rush, 2 to 3 orders of magnitude more traffic than usual, from a marketing push and haven’t ran into any issues yet.
I still run KEDA at home for managing plex, home assistant, some game servers and other of my own projects. But being the only one who is using the cluster is a different use case than getting RBAC, ingress and management set up correctly for a production cluster IMO. I’ve never had the sole responsibility or permission over a cluster before, so it was a daunting step I decided not to take for my own sake
Yes Pokémon Legends: Arceus was a tipping point for me. The game does not look good at all to me, and across 3 different switches I tested, it felt like it was hitching and not loading correctly even with the low detail and resolution in docked mode. I’m handheld it’s a bit better, but that’s not really how I want to play the game.
On the other hand, my most played game is by far Old School RuneScape, which is a restore of a backup of RuneScape from 2007. The game has certainly evolved in style and content since the re-release, and I use a 3rd party client with an “HD” plug-in which adds more render distance, shadows, lighting and anti-aliasing.
Like the others have said, definitely seems to be hit or miss. I inherited an old cluster running on elastic cloud, migrated it to AWS managed and it reduced monthly costs by nearly an order of magnitude on top of going from a 5-15 minute outage per day for master node election to having 3 over 2 years.
We’ve rewritten most of the interactions with the older version of ES into a new service and use a self-managed OpenSearch cluster on Graviton instances and it’s the most stable Elasticsearch/Solr solution I’ve ever interacted with
Even better then I would have thought, thanks! I’ve primarily dealt with Elasticsearch/Solr in my career, so jumping into the way searches work in Postgres with various native support and plugins is, interesting to say the least. Maybe I just need to break my Lucene roots
Poking a bit at the documentation, I see like/ilike but are there plans for text search/trigram capabilities? Recently I’ve been working with lots of different entities that have searchable properties and exploring searching across different elasticsearch indexes to do “JOIN” like operations but have been exploring Postgres (and related) solutions for better “JOIN” support out of the box
To get around the separate directories/copies someone established a pattern of doing a for each environment at the top of each tf file and then all variables are stored as a map of each env, which I find interesting.
The pattern exists in one repo with the other being all the duplicates, they’ve been in “it just works” for long enough that no one has changed them. Workspaces look pretty slick, I’ll have to check them out
My manager is aware only because we have worked together several times, but he still fights for a yearly raise on the 5-15% scale of things without me asking, and does for other team members as far as I know. For the most part I work 5 by 5s but if there is a meeting or it’s my turn to be the on call engineer I don’t get to shove those duties to the side because they aren’t fun. I have followed and will continue to follow said manager for the foreseeable future due to the relationship we have
I have always seen the second monitor as “extra” space and not to be used equally. I previously used two 34” ultra wide (21:9) monitors with one centered and one off to the right. The left half of the extra monitor was basically reserved for slack and dashboards and the right half of the extra monitor was seldom ever used.
I still use that setup for gaming,
but I use a single Pro Display XDR for work now and I think the only time I might miss the extra screen space is probably Zoom
Absolutely this. My side of the engineering department has taken a hard stance on what our definitions of Data Analyst, Data Engineer, Data Scientist, Machine Learning Engineer and Applied Scientist are. We have had a few issues with people wanting a different job title while interviewing and we supply them with our description plus a path from the position we believe they are to what they want to be. The ones who have taken these offers have been some of our best hires.
On the other hand the analytics/sales teams have many DS and MLEs, a DA is basically a “junior” for them, but they routinely have to reach over to an engineering team to do pretty basic SWE skills that they are supposed to cover themselves
1) most of the team is on MBP 16 inch models. I know there is one and may be two on 13 inch models.
2 and 3 I’ll answer to the “best of my knowledge” but on the professional level it’s something my team doesn’t handle (I do wish I knew more!). Features get merged to dev/stage/main and trigger blue-green rolling deploys based on whatever is configured. The deploy is handled through AWS CodeDeploy (not my choice, also not my teams jurisdiction) which handles ticket validation/testing/deploying to whatever k8s cluster/manual deployment rollover if needed.
I believe you can build ARM images from Docker, or at least I have a vague recollection of doing so for a Raspberry Pi, on a normal x86 machine
We have a base Debian image maintained by our Platform/Security team (compliance) that we build our Flask apps off. Other than them forgetting to push an ARM compatible image when they update we haven’t ran into issues, but I don’t know what went in behind the scenes to get the image working if anything.
I would say I’m the bridge between the infra/DevOps teams and our back end team of 12 people and as far as everyone else on the team is concerned it “just works”. Still waiting for the first big time it doesn’t
From my POV now, the i3 instances should never be used unless you absolutely need the dedicated local storage in those amounts. The per vCPU performance is horrid in comparison to any of the modern instances (R5/M5/C5) let alone comparing those vCPUs to the actual CPU of R6g/M6g/C6g like you said.
Another advantage of the Graviton processors is 50% more dedicated storage compared to equivalent Intel/AMD instances. To get enough storage we would have had to bump up to r5d/r5ad.24x or metal which when testing we also saw more “jitters” in latency on the long tail. Despite the x86 instances being larger they traded blows in different tests we had, aggregates were one thing that x86 easily beat out ARM but a lot of our aggregates come from another data source so it wasn’t a deal breaker. Overall we are happy with performance, compared to old stack we are at around 10% of the cost and I think our savings was more than 2x compared to x86 after locking in some rates. R6gd.metal (16x) vs R5d.metal (24x)
We’ve been testing a few Graviton instances since before GA and have gone “all in” with our latest release targeted for EOY.
We went live with R6g (Redis) and R6gd/T4g (ES) instances at the beginning of the month. We were coming from an ancient cluster from AWS managed ES (2.3) and using their i3 elasticsearch instances so it’s been an Apples to Oranges comparison.
We are running T4g/C6g/R6g
K8s clusters for web servers.
We do still have Postgres on R5 instances but I’m interested to check out R6g there since we can do a direct comparison
If you want to get it set up and running most of the way without tuning a bunch of stuff it’s pretty good.
It also has significant cost overhead compared to self-managed and limited instance types.
My team now is working on migrating a 2.3 cluster from AWS ES to a 7.8 self-managed cluster (probably 7.9 now I guess). We do lots of indexing throughout the day and the i3 instances available are pretty weak, just switching to r5d/r6gd instances greatly reduced our indexing and search latency
B2B side has been moving slowly off IE11, but moving faster since the start of the year and with most of the rest of the customers targeting to be off of it originally by end of quarter. Currently sitting around 40% but if they go through with their plans to upgrade by end of quarter then will be around 5%
B2C side is much lower, the users who visit not very frequently or are newer that use IE is low single digits. On the other hand we have some long term continuous use users who use IE at a higher rate than the B2B side
I use a first generation iPhone SE every day in conjunction with my regular iPhone 11 Pro Max and I definitely wouldn’t call it unusable. The main pain point I have with it is loading a heavy webpage if I don’t have ad blocking on
An example from a previous job where we used a hand tooled NLP system was querying for doctors/dentists/optometrists and being able to take something like “dentist near me who is available in the afternoons and speaks spanish”. We would parse this user query into a few different queries that would run against a search cluster and database to return the filtered result set, or the closest output.
What would be the ideal way to prepare or tokenize this data for querying with an LLM? It’s partially text (match dentist, speak Spanish), partially geographic (near me, doing a geo radius of N miles from providers location, and part filtering (who meets all those criteria and has availability in a time frame). Is this a use case for large token sizes to be able to take in all possible providers? Or parsing a query more easily from human language -> SQL/other data store query language? Or perhaps figuring out another way to encode this data?