Conversation
|
It is currently not explicitly specified anywhere but I guess it should be clarified that solutions should implement a generally correct implementation of the algorithm in question. Ideally, test cases should be designed to make this inevitable. |
|
Haha; I wanted to suggest this; but I am glad someone else did. Although the prepacked versions demonstrates a practical solution when the problem space is known; I think we can agree they aren't really demonstrating language features 😬... (which was kind of the point I was trying to make). |
While I can relate to the "gaming the system" feeling, it's not all that straightforward: the prepack strategy is legit and could be used by real validators. Its true that it won't scale to the higher input values, but its ok to apply it in general. I am hesitant to approve this PR which changes the scenario tests for one reason: doing it will invalidate existing submissions. However, I have a counter-offer for you: What do you guys think of this alternative idea? Otherwise it might be not to late to ask authors of existing pre-packed submissions to update them for higher inputs. |
This adds harder tests for fibonacci open.
The "prepacked" solutions by aiken and plutarch basically hardcode all testcases of the benchmark. This seems to me to somewhat improperly gaming the system.
Moreover, to really evaluate the power of linear solutions, larger inputs need to be tested.
This commit adds such larger inputs. Note that the aiken solution fails on this, because it just hardcodes results up until 25. Note also that all other submissions fail on this because they essentially time out. The latter case should be correctly handled by uplc-cape (i.e. by restricting the budget of the UPLC machine)