> Whether you’ll get the high IQ or low IQ LLM on the next suggestion is a crapshoot
I have heard theories from people--whose ideas I don't ordinarily consider to be in the realm of purely uninformed speculation--that this can vary depending on system-wide GPU load, and is throttled accordingly based on real-time demand by the major LLM providers.
Sounds like conspiratorial thinking to come to terms with the fact that LLMs are fundamentally non-deterministic text predictors. You can get dramatically different responses to the same question, and that’s just how the system works.
I have heard theories from people--whose ideas I don't ordinarily consider to be in the realm of purely uninformed speculation--that this can vary depending on system-wide GPU load, and is throttled accordingly based on real-time demand by the major LLM providers.
Whether it's true, I have no idea.