The name ‘late chunking’ is indeed somewhat of a misnomer in the sense that the technique does not partition documents into document chunks. What it actually does is to pool token embeddings (of a large context) into say sentence embeddings. The result is that your document is now represented as a sequence of sentence embeddings, each of which is informed by the other sentences in the document.
Then, you want to parition the document into chunks. Late chunking pairs really well with semantic chunking because it can use late chunking's improved sentence embeddings to find semantically more cohesive chunks. In fact, you can cast this as a binary integer programming problem and find the ‘best’ chunks this way. See RAGLite [1] for an implementation of both techniques including the formulation of semantic chunking as an optimization problem.
Finally, you have a sequence of document chunks, each represented as a multi-vector sequence of sentence embeddings. You could choose to pool these sentence embeddings into a single embedding vector per chunk. Or, you could leave the multi-vector chunk embeddings as-is and apply a more advanced querying technique like ColBERT's MaxSim [2].
You don’t have to reduce a long context to a single embedding vector. Instead, you can compute the token embeddings of a long context and then pool those into say sentence embeddings.
The benefit is that each sentence’s embedding is informed by all of the other sentences in the context. So when a sentence refers to “The company” for example, the sentence embedding will have captured which company that is based on the other sentences in the context.
This technique is called ‘late chunking’ [1], and is based on another technique called ‘late interaction’ [2].
And you can combine late chunking (to pool token embeddings) with semantic chunking (to partition the document) for even better retrieval results. For an example implementation that applies both techniques, check out RAGLite [3].
Where's the data that says Moore's law no longer holds? I see comments and articles asserting this but everytime with evidence. The data that I do find certainly still suggests Moore's law is doing fine.
Sounds great until the client realises they can hire someone else who does charge by the hour, saving them a massive 100k - 10k = 90k compared to your proposition.
Are you sure about that? It depends on how Cloudflare defines what a cold start is. It might well include the initial loading of your code, with imports and init.
My experience has been the opposite: building Docker images is much easier with conda than it is with pip. With conda you can start from miniconda3, copy an environment.yml, and then conda create it. With pip, you might need to take additional steps to install system dependencies like build-essential first, and you'll need different tools to manage your virtual environment and Python installation too.
I sympathize with your views in that a formalisation of meaning in life would seem to take away from it somewhat. However, your response also comes across as putting a stake in the ground and thereby closing your mind as to what the utility framework has to offer.
These are some of the problems with utility I took away from your comment:
1. Short term utility maximisation does not necessarily align with long term utility maximisation.
2. The utility is difficult to define because we don't know what can contribute to it and how much those components would contribute.
3. The utility, even if defined, can only be partially observed.
While these are certainly valid and interesting observations, that doesn't mean there isn't a latent utility that describes your lifetime's value. If you live a thousand lives and had access to a device to measure the (latent) utility, surely you could it to rank lives as more meaningful or valuable than others.
Looking at it this way, the problems you raise could be seen as opportunities: how should we align short term utility with long term utility? How can we find out which utility components there are and how much they would contribute? And how can we make the utility observable, so that it can be maximised?
Interestingly, attempting to maximise the utility may itself lower the utility as a result of spending time on the maximisation task and the dehumanisation of life as a result of its formalisation.
The implementation learns to play Battleship in about 2000 steps, pretty neat!