We are experimenting with ways to use ChatGPT to get better answers more reliably, remove hallucinations, etc.
This little library will generate multiple draft responses and then use a second model to judge the answers and pick a winner, which is then returned to the user. Google's Bard uses this same approach.
With this library you can apply the pattern to gpt-3.5 and gpt-4.
Drafts are generated in parallel and all drafts are evaluated with a single prompt.
This will use a lot of tokens. For example to generate 3 drafts, you are at 3x + you need to feed those drafts into another prompt + get that response, so >7x.
What all these tools need to adopt is sending 10-20 requests out and finding the "best" response. I think it's incorrect that we try to get the tool to work right the first time. Auto-GPT has JSON parse errors 20-50% of the time. Instead, with enough parallel responses we can increase the likelihood one of them is "really good". The next challenge is figuring out which answer is really good and continuing with that.
The limitation is because of the word position embedding matrix size. This isn't a config issue, or an API limitation. This is a limitation on the size of a matrix that is part of the model and is decided on before training. You can't change it.
What does that mean?
For each token in your input or inference output it requires the model to have some understanding of what the position of the word means.
So there is the word position embedding matrix that contains a vector per position. The matrix has "only" 1024 entries in it for GPT2 or 4096 for GPT3. The size of each entry varies as well, containing a vector from 768 for GPT2 small and up to 12,288 for GPT3.
So the WPE (word position embeddings) for GPT2 is (1024x768) and for GPT3 (4096x12288)
Inference requires info from this vector to be added to the word tokens embedding for each token in the original prompt + each generated token.
Could you please explain this line of reasoning? It's not immediately clear why a middleman reduces chances of malicious behavior. Do you think the risk of malicious behavior could be reduced through individual confidence, judgement or leadership in the same way or differently?
I started a consulting business about 10 years ago. We're >80 people now and only do full team projects that include design, product management and engineering.
As others have said in this thread it's all about relationships and sales. Technical ability will help you retain clients and build bigger things over longer timescales, but won't help you get clients in the first place. Also, technical ability rarely leads to happy clients - they don't review your code - but a focus on helping the client achieve their mission is the key to success.
You need to start by getting 1 client. Then 2, and so on. You don't need a website, or even the name of a company to do this. I didn't know the name of my consultancy until about 6 months into it after we were 5 full time people.
Subcontracting with other firms when you start is possible, but is mostly based on relationships, not capability. When I started I had a relationship with a design agency and they subcontracted us their heavy engineering work. In reverse, I saw this as outsourcing sales and account management to them.
To grow the business I started a meetup.com group back when meetup was just starting. There were no other groups at the time and I was able to make a good name for myself in the community which lead to word of mouth business. Once a month I would have a different big name company in town host an event where I would bring 50-80 engineer types and do some tech talks for about an hour. It took about 5 hours to organize per month and barely any out of pocket costs since companies were happy to open their spaces to technical types and provide food.
You need to figure out what you're willing to sell. Staff augmentation? Time & Materials? Contracts based on scope? They're all very different in sales process and delivery.
You need to think about account management as a real thing. Developing the empathy and focus around happy customers over "best code" or "ideal features" is critical.
Developing your sales beyond word of mouth is the hardest thing to do. Hiring a sales person alone won't do it. There are way too many small consultancies reaching out to potential customers everyday and you can't distinguish yourself from the noise. If you have a clearly differentiated product offering or services approach that a sales person can leverage to make a clear pitch that's not just "we can do your projects!" then it can work.
This is scientifically verified and yet nobody does anything about it.