This regression seems to have happened in the past few days. I suspected it was hallucinating the run and confirmed it by by asking Gemini to output the current date/time. The UTC it was reported was in the future from my clock. Some challenging mathematics were generating wrong results. Gemini will acknowledge something is wrong if you push it to explain the discrepancies, but can't explain it.
My experience points to that until the last sorry is eliminated the math might look like it will fit together, but yet not ultimately all fit together.
Somewhat related: I'm surprised we don't see NonZero numbers (NzInts, NzNats, and NzReals, etc). This would seem to avoid a lot of division by zero problems at the type level. I realize this might "back up" checking for zeros into the origin of denominator values, but that seems like it would be an improvement to me.
I've been using LLMs to discuss fairly challenging topics and have had some success. Here are a few hints:
1) Assume that LLMs work like improvisational actors in that they are in the mode of saying, "Yes, and...". By that I mean that they will mostly agree with what you write without providing a real critique of your ideas unless so prompted. If you don't ask critical questions the AI will happily entertain crazy ideas. I bet I could get it to happily act as if it believed the moon is made of cheese with sufficient prompting effort.
2) Use multiple session to explore and discuss separate topics. Use your "main context" with purpose.
3) Ask a series of "warm up" questions. I've found that the quality of reasoning differs immensely whether the AI has been discussing a topic. Asking a question, then asking the AI to continue (several times), then asking it to reconsider it's responses seems to create the required knowledge graph network that is required for a pretty good response.
4) Some LLM's will eventually either stop working or start to slow down as their context becomes filled. I've found the best way around this is to ask one AI to generate concise prompts with minimal formatting to help another AI get up to speed. Don't let it get too meta though. Two AI's will happily "fluff" each other's ego and get nothing done over a series of dozens prompts.
5) Use "Less Meta." as a command when an AI talks about how to do something instead of actually doing it.
6) Pay attention when an AI says it cannot do something. It might start lying to you if you continue to push it.
It would be interesting to model open source contributions using something like this. Perhaps it would reveal something about the economic paradox about why people contribute to open source.
The use of this for setting configuration options is absolutely brilliant. I've been looking at cheap ways of doing this for years. That you would publish this patent-free is fantastic. Thanks!