Yes exactly. My company has asked AWS if they will be adding support for pgvector for rds but they haven't been able to confirm if that will happen any time soon.
If the vectors are in the same database as the tabular/structured data then text to sql applications of llm's are so much more powerful. The generative models will then be able to form complex queries to find similarity as well as perform aggregation, filtering and joining across datasets. To do this today with a separate dedicated vector db is quite painful.
The hack to solve this is to embed each paragraph in your large corpus. Find paragraphs most similar to the user query using embeddings. Put the paragraphs and the raw user query into a prompt template. Send the final generated prompt to gpt3.
Create features for day of week, day of year, month of year, lagged values of y, lagged values of y for each period (eg: 1, 2, 3 weeks and years ago etc). You then predict forward 1 time step at a time.
This looks really cool! I really like the free form chart builder. I work at Simply Wall St and we also make financial data easy to use and understand. We’ve realised that a bigger portion of our effort should be spent on educating individual investors so it looks like we’ve come to the same conclusion as both of you.
If anyone is keen to checkout Simply Wall St (https://www.simplywall.st), we have a 7-day free trial and a free plan for ongoing light usage.
I was initially excited about this as I think it might solve our Redshift pain points and potentially avoid us having to deal with a migration to Snowflake but then I remembered when AWS account managers promised Athena and Spectrum would solve these same problems at a previous company I worked for a few years ago. I'm assuming the developer experience will still be terrible with lots of knobs to tune to actually get any decent cost/performance.
Having used Redshift considerably in the past and experienced significant pain with many aspects of it, I would stay away. There are other much better options, ideally Snowflake.
I read somewhere (a YC video or post I can't find now) that during a pivot, the problem you're working on shouldn't really change, but the solution might change completely.
I interpreted the superordinate goal as the higher level problem you want to solve and dedicate years/decades of your life towards. It might result in multiple startup attempts with entirely different products.
It seems that a superordinate goal which is something you truly believe is worthy is probably well-validated due to deep personal experience. This seems less likely to change easily.
I think one of the less discussed advantages of SQL and reasons why it might be so prevalent in the analytics world is that users don't really need to spend much time to set up their environment. I can send a query to a less technical person with my analysis for them to run. They can put the processed data into a spreadsheet and do whatever they want.
This is exactly my experience. I've found becoming proficient with Pandas to take much more time than other data manipulation libraries like dplyr or Pyspark dataframes. The Pandas way of doing things is just not memorable or intuitive.
Hi, I’m Bayan, author of the post. I would love to hear if you agree/disagree with the definition of a unicorn product analyst or you’ve met anyone who fits the mould. Keen to hear your thoughts.
It would be awesome if gpt4 could be made to edit the code and therefore, it didn’t have to regenerate it from scratch every time.
The regeneration also chews up a lot of the token limit, so it forgets crucial parts earlier on in the conversation.