I linked to the shared report so that it's obvious what is actually being produced, rather than just a description of it.
This is just an example summary of a single web page, you could imagine it producing much more compelling content, for example combining multiple pieces of information, along with a twist, just as many humans do when publishing on the web. FlowChai could search hundreds of documents to produce a report (using RAG), so in that sense it can go beyond what a person could do in a reasonable time frame.
While this makes some of what my startup https://flowch.ai does a commodity (file uploads and embeddings based queries are an example, but we'll see how well they do it - chunking and querying with RAG isn't easy to do well), the lower prices of models make my overall platform way better value, so I'd say overall it's a big positive.
Speaking more generally, there's always room for multiple players, especially in specific niches.
I have found success with chunking with 100 tokens, preceeded by the last 10 tokens of the previous chunk, and the first 10 tokens of the next chunk, 120 tokens total. I generate an embedding for each, then compare that to embedding(s) derived from the input query.
How to generate embeddings from the input query well is where one's focus should be IMO. An example: "don't mention x" being turned into filtering out / de-emphasizing chunks that align with the embedding for x.
I've been using these techniques along with pgvector and OpenAI's embeddings for https://flowch.ai and it works really well. A user uploads a document or uses the Chrome Extension on a webpage and FlowChai chunks up the content, generates embeddings, builds up a RAG context and then produces a report based on the user's prompt.
I hope that helps show a real world example. You're welcome to play with FlowChai for free to see how it works in practice at the application level.
Things like this needing to exist are one of the reasons we are working on https://flowch.ai/?ref=hn - the ChatGPT interface is really sub-optimal for work and keeping track of interactions over time. We have been building a much better way to organize LLM interactions, even basics like splitting things into projects and search make it a much better experience.
Very fair, we have demo videos, guides etc planned for the next week or so. As it's a tool that can do many things it's hard to describe. Still a lot to do :)
In terms of what makes the report different from Bing: this could be any source of data: scraped from the web, search, API upload, file upload etc, so there's a lot more power there. Also, it's not just one off reports, there's automation there which would allow for example a weekly report on the latest papers on GPT4 (or whatever you're interested in).
Interesting that they're still centered around Chat as the interface, with https://flowch.ai (our product) we're building it much more around projects and reports, which we think is often more suitable for businesses.
We're going after some of these use cases:
Want a daily email with all the latest news from your custom data source (or Google) for a topic?
How about parsing custom data and scores from your datasets using prompts with all the complicated bits handled for you, then downloading as a simple CSV?
Or even simply bulk generating content, such as generating Press Releases from your documents?
All easy with FlowChai :)
I think there's room for many different options in this space, whether that be Personal, Small Business or Enterprise.
Learning to leverage LLMs as part of software systems (distinct from LLMs for writing code, that'll be common soon as people catch on and it becomes acceptable to most).
As part of my testing of https://flowch.ai (which we are developing), I've been uploading multiple PDFs (including scientific papers). I then ask a question and ask it to cite its sources as part of the prompt and it'll give the PDFs it's using for the data / quotes it's using. Works well!
Its integration with public and private data along with layers of automation allowing its use way beyond chat bots. From individual tool to something that can power large parts of a business.
The open source solutions have mental overhead and potentially long term maintenance hassle. Even for people who could set it up, for many $20 is worth not having to deal with it.
GPT-4 via ChatGPT, mostly for coding and GPT-4 via https://flowch.ai (our tool) for anything that involves my own data (including my own code), or where I want automation, such as running a prompt on many pieces of data.
Overall this has at least 5x my productivity, possibly 10x.
https://flowch.ai (our project) does this and is currently free to use. It doesn't train a custom model but it gets good results.
A simple way to do this is to upload your files (PDFs, Wod docs, virtually any type is supported), then generate reports using prompts based on those uploaded files. You can go from uploading to results in around a minute.
I've had good results uploading a bunch of documents, then running the same prompt on each of the documents with a few clicks using the "Flow Reports" feature.
We're working on lots of stuff on top of this, like scheduled reports (daily summaries / analysis / newsletters) and automated web scraping and data upload.
Here's an example using a BBC News article that I just uploaded to FlowChai (prompt: summarize in 200 words):
A lot of LLM libraries seem incredibly overhyped. I've found that I'm able to do a lot with the underlying API calls.
If you're comfortable with doing API calls in your own code, I'd suggest trying to prototype your desired functionality without those libraries to start with.