Editing Philosophical Research:The LLM Olympics (section)

== Rules ==

Three very important rules will be followed in all of these tests:

<strong>1)</strong> Absolutely no online models will be used, only models that can be run entirely offline. This is mainly for the ethical concern of making sure that running the models does not use more computing power or rack space than a regular computer program. However, it also has the benefit of creating the simplest test cases with no external variables. If there is only 1 gigabyte of model or less and not 10 more gigabytes of model hiding out of view, it is easier to know the full range of behaviors of the model, and if nobody else is running the model, there will not be any external actions "the company" can take at the same time the test is running such as datamining conversations or inserting ads. All the causes and effects inside the test will be in one place.

<strong>2)</strong> No generated sentences will be directly copied onto any page. All the text on these pages is created manually. The longest quotations of generated text on these pages will be approximately three words long.

<strong>3)</strong> The LLM must not be given an unreasonable task, only tasks which fit within the boundaries of its known programming, bugs, and quirks. Each task will include several steps of "testing understanding" to make sure the LLM is getting the intended answers at every single step before then giving it harder questions requiring inference and not directly explained in the text. Unless the task proves to be truly impossible, the test will not stop until the LLM actually completes the task.