Great resource, definitely a good place to take the next step. As I looked into detail, the natural question came (based on software developing experience), how do I evaluate the correctness of output produced by LLM given the inputs. Clearly, unit test with fixed in/out pairs won't help so learning methods to evaluate as we develop iteratively will be very useful.
May be if you pick a real-world agent workflow (toy from your production experience, trim it down), and showcase how all these factors will come along in a project.
I am inspired by the simplicity of these 12 factors and definitely want to learn more with an example that embraces these factors.
I am wondering how libraries like DSPY [0] fits in your factor-2 [1]
As I was reading, I saw mention of BAML
> (the above example uses BAML to generate the prompt ...
Personally, in my experience hand-writing prompts for extracting structured information from unstructured data has never been easy. With DSPY, my experience has been quite good so far.
As you have used raw prompt from BAML, what do you think of using the raw prompts from DSPY [2]?
Yes, I love using the code and AI tools like Claude, Github CoPilot are great at it. But when solving a business problem, we still need to "see it" before start writing the code (at least this is my preferred way)
I have made one side project ever since 2013 (I am working from 2011 and made few failed project, but learnt a lot). In 2013, we had exactly same problem - tracking expenses and do personal budgeting. My wife was using spreadsheets and I created a software (using Python and Flask) and released a webapp. Later we realized that an app would be much more useful, so rewrote the backend (Java EE) and front-end(Objective C), but never released it. We have however 3 years worth of our own data (never did analysis or charts). Lately, we feel that since our app never made to App Store, it crashes every 7 days and I need to reinstall/use XCode to do this work (very painful). I plan to rewrite again in Scala (my latest favorite) and use React-Native to build the app. This time I would like to publish it
I do not consider myself as very productive (and still try to learn from others). Some things that has helped me are
1. Increased water intake/Eat Well. I drink 1st bottle (300ml) with black tea (usually Oolong) and then refill with hot water. I make sure I am drinking water while I work. I am vegetarian and I take almost no sugar. I have replaced sugar with raw honey in my chai :-). I try to have more plant-based diet.
2. Take breaks. When I am stuck with the problem, I take either water-refill break/pee break/short walk to get me away from the problem, but possible solutions find me where ever I am.
3. Start day early. I usually like to start my day between 5.30AM-6AM. But I do not jump on emails/office work. I have a separate to-do list to learn/read different things (mostly technical). I start my day with it. That's my kick.
4. Less scanning. I use Asana in the Calendar view. As soon as I reach the office, I make sure the world is not crashing. I add tasks that need my attention and close the email. Then I add things I like to do (and aren't listed in my today's task). I start hunting these tasks. If I have some dependency on others(Ops for example), I create tasks/email them and move on with my agenda
5. Exercise. Not doing very much these days, but try to run for 20-25 minutes every evening, to have physical workout > mental workout (This is a key to good sleep)
6. Sleep well. I try to get somewhere between 7-8 hours every night. I need to learn to improve this area.
At the end of the day, I prefer a fulfilled day -> A day where I produced value to my company Plus I learned something new. That produced happiness which leads to productivity. Iterate.